February 2024 - nlp_reading_group

Reading Discussion, Mar 1st
by Xu, Zhiyang 01 Mar '24

01 Mar '24

Hi Everyone, In the reading discussion this week, we will discuss: LLaMA Pro: Progressive LLaMA with Block Expansion<https://arxiv.org/abs/2401.02415> This paper proposes a new post-pretraining method for LLMs with an expansion of Transformer blocks. By tuning the expanded blocks using only new corpus, their method efficiently and effectively improves the model's knowledge without catastrophic forgetting. The reading discussion is at 4-5 pm on Mar 1st this Friday. You may attend the reading discussion in Room 366 in the Data & Decision Building, or on Zoom (https://virginiatech.zoom.us/j/5961845863). Enjoy reading! Best, Zhiyang Xu

1 1

Reading Discussion, Feb 23rd: How to Train Data-Efficient LLMs
by Yao, Barry 23 Feb '24

23 Feb '24

Hi Everyone, In the reading discussion this week, we will discuss: How to Train Data-Efficient LLMs<https://arxiv.org/abs/2402.09668> This paper is from Google DeepMind and investigates data-efficient approaches for pre-training LLMs, i.e., techniques that aim to optimize the Pareto frontier of model quality and training resource/data consumption. Their proposed Coverage sampling can recover the performance of the full data, while models trained on their ASK-LLM data consistently outperform full-data training — even when they reject 90% of the original dataset, while converging up to 70% faster. The reading discussion is at 4-5 pm on Feb 23 this Friday. You may attend the reading discussion in Room 366 in the Data & Decision Building, or on Zoom (https://virginiatech.zoom.us/j/4506040955?pwd=TGFtTUVWR2tuYmVlS3JraTQzbjgrU…). Enjoy reading! Best regards, Barry Menglong Yao Ph.D. student in Computer Science Virginia Tech Homepage<https://barry-yao.netlify.app/>

1 1

Reading Discussion, Feb 16th
by Beigi, Mohammad 15 Feb '24

15 Feb '24

Hi Everyone, In the reading discussion this week, we will discuss: “Multimodal Neurons in Pretrained Text-Only Transformers” This paper is about multimodal neurons in text-only transformer MLPs and how these neurons consistently translate image semantics into language. The reading discussion is at 4-5 pm on Feb 16ht this Friday. You may attend the reading discussion in Room 366 in the Data & Decision Building, or on Zoom (https://virginiatech.zoom.us/j/5961845863) Best Regards, Mo Link to paper: <https://openaccess.thecvf.com/content/ICCV2023W/CLVL/papers/Schwettmann_Mul…> [preview.png] Schwettmann_Multimodal_Neurons_in_Pretrained_Text-Only_Transformers_ICCVW_2023_paper<https://openaccess.thecvf.com/content/ICCV2023W/CLVL/papers/Schwettmann_Mul…> PDF Document · 16.9 MB<https://openaccess.thecvf.com/content/ICCV2023W/CLVL/papers/Schwettmann_Mul…>

1 0

Reading Discussion Feb 9th 2024
by Yeole, Nikita 08 Feb '24

08 Feb '24

Hi Everyone, In the reading discussion this week, I will be presenting: Transforming Free-Form Sentences into a Sequence of Unambiguous Sentences with LLM. The research focuses on addressing the factors associated with Natural Language which makes Natural language programming very challenging and building an intermediate system between the natural language and the code by converting the free form sentences into a sequence of simplified and clear sentences using Large language model. The reading discussion is at 4-5 pm on feb 9th this Friday. You may attend the reading discussion in Room 366 in the Data & Decision Building, or on Zoom (https://virginiatech.zoom.us/j/5961845863) Thanks and Regards, Nikita Yeole.

2 1