Here are my research papers and publications:
Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation
Authors: Vaibhav Seth, Ayan Sengupta, Arinjay Pathak, Natraj Raman, Sriram Gopalakrishnan, Tanmoy Chakraborty Abstract: Fine-tuning Large Language Models (LLMs) is resource-intensive due to their size. While low-rank adaptation is a common parameter-efficient fine-tuning approach, it is sensitive to hyperparameter choices, leading to instability. This paper introduces MonteCLoRA, a fine-tuning method that uses Monte Carlo estimation to achieve unbiased posterior estimation of low-rank parameters with low variance. MonteCLoRA improves stability and performance with minimal additional parameters (O(1)). ...