- nlp_reading_group - mailman.cs.vt.edu

[nlplab] Reading Discussion on May 3
by Yao, Barry 03 May '24

03 May '24

Hello everyone, In this week's reading discussion, we will talk about: LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement<https://arxiv.org/abs/2403.15042> Abstract: While many real-world applications still require fine-tuning to reach satisfactory levels of performance, many of them are in the low-data regime, making fine-tuning challenging. To address this, the authors propose LLM2LLM, a targeted and iterative data augmentation strategy that uses a teacher LLM to enhance a small seed dataset by augmenting additional data that can be used for fine-tuning on a specific task. You could attend the meeting in the Data & Decision Building, Room 366, or on Zoom (https://virginiatech.zoom.us/j/4506040955?pwd=TGFtTUVWR2tuYmVlS3JraTQzbjgrU…) at 4-5 pm on Friday. Note that the Zoom link is different from the previous reading discussions. Happy reading! Best regards, Barry Menglong Yao Ph.D. student in Computer Science Virginia Tech Homepage<https://barry-yao.netlify.app/>

1 1

Reading Discussion on Apr 19
by Xu, Zhiyang 18 Apr '24

18 Apr '24

Hi everyone, In the reading discussion this week, we will discuss: MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding (https://arxiv.org/pdf/2404.05726.pdf) The reading discussion is at 4-5 pm on Apr 19th this Friday. You may attend the reading discussion in Room 366 in the Data & Decision Building, or on Zoom (https://virginiatech.zoom.us/j/5961845863<https://www.google.com/url?q=https%3A%2F%2Fvirginiatech.zoom.us%2Fj%2F59618…>). Enjoy reading! Best, Zhiyang Xu

1 0

[nlplab] Reading Discussion on April12
by Shen, Ying 09 Apr '24

09 Apr '24

Hi Everyone, In the reading discussion this week, we will discuss: Eliciting Latent Predictions from Transformers with the Tuned Lens<https://arxiv.org/pdf/2303.08112.pdf> The reading discussion is at 4-5 pm on April 12 this Friday. You may attend the reading discussion in Room 366 in the Data & Decision Building, or on Zoom (https://virginiatech.zoom.us/j/5961845863). Happy reading! Best, Ying

1 0

Reading Discussion on Mar 28
by Liu, Minqian 28 Mar '24

28 Mar '24

Hi Everyone, In the reading discussion this week, we will discuss: Evaluating Large Language Models at Evaluating Instruction Following (ICLR 2024) (https://arxiv.org/abs/2310.07641) The reading discussion is at 4-5 pm on Mar 28 this Friday. You may attend the reading discussion in Room 366 in the Data & Decision Building, or on Zoom (https://virginiatech.zoom.us/j/5961845863<https://www.google.com/url?q=https%3A%2F%2Fvirginiatech.zoom.us%2Fj%2F59618…>). Enjoy reading! Best, Minqian Liu

1 0

Reading Discussion, Mar 21
by Qi, Jingyuan 21 Mar '24

21 Mar '24

Hi Everyone, In the reading discussion this week, we will discuss: RAFT: Adapting Language Model to Domain Specific RAG<https://arxiv.org/abs/2403.10131> This paper proposes a training recipe that improves the model’s ability to answer questions by ignoring those documents that don’t help in an "open-book" in-domain setting. The reading discussion is at 4-5 pm on Mar 1st this Friday. You may attend the reading discussion in Room 366 in the Data & Decision Building, or on Zoom (https://virginiatech.zoom.us/j/5961845863<https://www.google.com/url?q=https%3A%2F%2Fvirginiatech.zoom.us%2Fj%2F59618…>). Enjoy reading! Best, ======================================== Jingyuan Qi Ph.D. student, Department of Computer Science Natural Language Processing Lab Virginia Tech (0710) Blacksburg, VA 24061

1 0

Reading Discussion Mar 15
by Mishra, Indrajeet Kumar 14 Mar '24

14 Mar '24

Hi Everyone, In the reading discussion this week, we will discuss: Denoising diffusion probabilistic models : https://arxiv.org/pdf/2006.11239.pdf This paper proposes a generative image modeling approach that learns the underlying distribution and generates high fidelity images. I will also discuss the idea of how this led to the development of Latent diffusion Models. The reading discussion is at 4-5 pm on Mar 15th this Friday. You may attend the reading discussion in Room 366 in the Data & Decision Building, or on Zoom (https://virginiatech.zoom.us/j/5961845863). Enjoy reading! Thank you Indrajeet Kumar Mishra

1 1

Reading Discussion, Mar 1st
by Xu, Zhiyang 01 Mar '24

01 Mar '24

Hi Everyone, In the reading discussion this week, we will discuss: LLaMA Pro: Progressive LLaMA with Block Expansion<https://arxiv.org/abs/2401.02415> This paper proposes a new post-pretraining method for LLMs with an expansion of Transformer blocks. By tuning the expanded blocks using only new corpus, their method efficiently and effectively improves the model's knowledge without catastrophic forgetting. The reading discussion is at 4-5 pm on Mar 1st this Friday. You may attend the reading discussion in Room 366 in the Data & Decision Building, or on Zoom (https://virginiatech.zoom.us/j/5961845863). Enjoy reading! Best, Zhiyang Xu

1 1

Reading Discussion, Feb 23rd: How to Train Data-Efficient LLMs
by Yao, Barry 23 Feb '24

23 Feb '24

Hi Everyone, In the reading discussion this week, we will discuss: How to Train Data-Efficient LLMs<https://arxiv.org/abs/2402.09668> This paper is from Google DeepMind and investigates data-efficient approaches for pre-training LLMs, i.e., techniques that aim to optimize the Pareto frontier of model quality and training resource/data consumption. Their proposed Coverage sampling can recover the performance of the full data, while models trained on their ASK-LLM data consistently outperform full-data training — even when they reject 90% of the original dataset, while converging up to 70% faster. The reading discussion is at 4-5 pm on Feb 23 this Friday. You may attend the reading discussion in Room 366 in the Data & Decision Building, or on Zoom (https://virginiatech.zoom.us/j/4506040955?pwd=TGFtTUVWR2tuYmVlS3JraTQzbjgrU…). Enjoy reading! Best regards, Barry Menglong Yao Ph.D. student in Computer Science Virginia Tech Homepage<https://barry-yao.netlify.app/>

1 1

Reading Discussion, Feb 16th
by Beigi, Mohammad 15 Feb '24

15 Feb '24

Hi Everyone, In the reading discussion this week, we will discuss: “Multimodal Neurons in Pretrained Text-Only Transformers” This paper is about multimodal neurons in text-only transformer MLPs and how these neurons consistently translate image semantics into language. The reading discussion is at 4-5 pm on Feb 16ht this Friday. You may attend the reading discussion in Room 366 in the Data & Decision Building, or on Zoom (https://virginiatech.zoom.us/j/5961845863) Best Regards, Mo Link to paper: <https://openaccess.thecvf.com/content/ICCV2023W/CLVL/papers/Schwettmann_Mul…> [preview.png] Schwettmann_Multimodal_Neurons_in_Pretrained_Text-Only_Transformers_ICCVW_2023_paper<https://openaccess.thecvf.com/content/ICCV2023W/CLVL/papers/Schwettmann_Mul…> PDF Document · 16.9 MB<https://openaccess.thecvf.com/content/ICCV2023W/CLVL/papers/Schwettmann_Mul…>

1 0

Reading Discussion Feb 9th 2024
by Yeole, Nikita 08 Feb '24

08 Feb '24

Hi Everyone, In the reading discussion this week, I will be presenting: Transforming Free-Form Sentences into a Sequence of Unambiguous Sentences with LLM. The research focuses on addressing the factors associated with Natural Language which makes Natural language programming very challenging and building an intermediate system between the natural language and the code by converting the free form sentences into a sequence of simplified and clear sentences using Large language model. The reading discussion is at 4-5 pm on feb 9th this Friday. You may attend the reading discussion in Room 366 in the Data & Decision Building, or on Zoom (https://virginiatech.zoom.us/j/5961845863) Thanks and Regards, Nikita Yeole.

2 1