Multimodal Code Generation
Building AI systems that generate code from multiple input modalities — combining natural language, visual diagrams, and structured specifications to automate software development.
Code LLMsVision-LanguageProgram Synthesis
PhD Research Scholar
Multimodal Code Generation
& Natural Language Processing
I am a PhD research scholar at IIT Bombay, currently working on Multimodal Code Generation — building AI systems that combine visual, textual, and structured inputs to generate code. I was formerly a Research Assistant at the CFILT Lab, IIT Bombay.
Interests
Multimodal Code GenerationNLPNeural Machine TranslationTransformersLow-Resource LanguagesNamed Entity RecognitionDeep LearningCode LLMs
scroll
About
My earlier research focused on neural machine translation for low-resource language pairs like Hindi–Chinese, applying romanization and data augmentation to bridge the gap where parallel data is scarce.
My work has been published at ICON 2024 and Springer, and I contributed to national multilingual AI initiatives under Bhashini/MeitY.
Education
PhD, Computer Science · IIT Bombay · CPI 8.33 · 2023–present
M.Tech, Data Science · JNU Delhi · 87.4% · 2020–2022
B.Tech, CSE · Rajasthan Technical University
GATE Qualified (2020, 2021, 2024) · UGC NET (2023)
Machine Learning & NLP
NLP · Machine Learning · Deep Learning · BERT · GPT · LLaMA · SFT · LoRA · XAI · Transfer Learning
Tools & Frameworks
PyTorch · TensorFlow · HuggingFace · NumPy · Anaconda
Platforms & Infrastructure
HPC Clusters (Slurm) · Docker · Python · MySQL · LaTeX
Research Areas
Active directions spanning multimodal code generation, agentic software engineering, mechanistic interpretability, and explainable AI.
Neural Machine Translation
Built encoder-decoder NMT systems for low-resource language pairs (Hindi–Chinese). Applied romanization (Romantra) to improve translation quality where parallel data is scarce.
Hi-Zh NMTRomanizationBack-Translation
Named Entity Recognition
Developed BioBERT-based NER models for biomedical entity extraction from COVID-19 corpora — identifying medical terms, symptoms, and treatments from clinical text.
BioBERTBiomedical NLPCOVID-19
Sarcasm & Sentiment Detection
Designed hybrid sarcasm detection combining rule-based, ML, and deep learning. Fine-tuned RoBERTa and fused contextual embeddings with LSTM for nuanced classification.
RoBERTaLSTMSentiment
Transformer Architectures
Working across BERT, GPT, and LLaMA variants with experience in fine-tuning (SFT, LoRA), parameter-efficient adaptation, and explainability (XAI) techniques.
BERT / GPTLoRAXAI
Multilingual AI (Bhashini)
Contributed to Ishaan and Vidyaapati multilingual AI projects under MeitY's Bhashini initiative — bringing AI capabilities to Indic languages at national scale.
BhashiniMeitYIndic NLP
Automated weekly tracker
Research Radar
High-priority recent papers across multimodal code generation, agentic software engineering, mechanistic interpretability, and explainable AI. Priority is boosted for watched venues, institutions, researchers, and stronger causal/evaluation signals.
Research radar data will appear after the first automated GitHub Pages update.
Publications
Peer-reviewed publications in NLP, Neural Machine Translation, and Biomedical AI.
ConferenceFeatured
2024Romantra: Optimizing Neural Machine Translation for Low-Resource Languages through Romanization
G. Soni and P. Bhattacharyya
Proceedings of the 21st International Conference on Natural Language Processing (ICON) · AU-KBC Research Centre, Chennai · NLP Association of India · pp. 157–168
NMTLow-Resource NLPRomanization
Book Chapter
2023BioBERT-Based Model for COVID-Related Named Entity Recognition
G. Soni
Advances in IoT and Security with Computational Intelligence · Springer · pp. 333–346
BioBERTNERCOVID-19
Book
2024Deep Learning for Extracting Biomedical Entities from COVID-19 Dataset: A Case Study
G. Soni
Springer · September 2024
Deep LearningNERBiomedical
Experience & Education
Work Experience
Jul 2023 – May 2025
Research Assistant
CFILT Lab, IIT Bombay
Former- Built and trained Transformer-based NMT models
- Applied back-translation and data augmentation
- Curated and preprocessed parallel/monolingual corpora
- Evaluated performance using BLEU and standard metrics
- Contributed to Ishaan & Vidyaapati (Bhashini / MeitY)
Oct 2022 – Jan 2023
Project SRF
IIT (ISM) Dhanbad
Auto-Evaluation of Tenders using AI/ML — OCR extraction from scanned documents and AI-based evaluation framework.
Jul 2020 – Dec 2020
Content Writer
GATE Coaching
Created technical content for competitive engineering examinations.
Education
2023 – Present
PhD, Computer Science
Indian Institute of Technology (IIT) Bombay
CPI: 8.33 · Research in Multimodal Code Generation and NLP for low-resource languages.
2020 – 2022
M.Tech, Statistical Computing (Data Science)
Jawaharlal Nehru University, Delhi
Aggregate: 87.40% · Thesis: Deep Learning for COVID-Related Named Entity Recognition.
B.Tech, Computer Science Engineering
Rajasthan Technical University
Foundation in algorithms, software engineering, and computer systems.
Qualifications
National Examinations
GATE & UGC NET
GATE Qualified (2020, 2021, 2024) · UGC NET Qualified (2023)
Contact
Open to research collaborations, academic discussions, and opportunities in NLP and multilingual AI.
govindksoni17@gmail.com
linkedin.com/in/sonigovind07
Google Scholar
View my citations
GitHub
github.com/sonigovind
Open to Collaboration
Research collaborations · Academic discussions · Industry partnerships