Abstract:

Computer vision and natural language processing have placed significant emphasis on producing textual descriptions for images over an extended period of time. Numerous methods based on deep learning have been established to achieve this objective, among which Image Captioning is the most conspicuous. The key objective of this project is to generate descriptions for images submitted by the user. This is achieved through a Pythonbased implementation of caption generation using Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) models. These models are trained and tested using both human-annotated images and synthetic data generated by a Generative Adversarial Network (GAN)-based text to image generator. The task of collecting a substantial number of human-generated images along with their corresponding descriptive captions can be both costly and time-consuming, but our proposed method has overcome this limitation by using synthetic data. The evaluation of the models involved commonly utilized qualitative and quantitative analyses, which demonstrated a substantial improvement in the quality of the descriptions generated for real images when both real and synthetic data were used during the training phase.

Description:

GAN, LSTM, CNN, Corpus Text, Lemmatization, Deep Learning