September 2022 - nlp_reading_group

[nlplab] Reading Discussion on Monday Oct 3
by Minqian Liu 30 Sep '22

30 Sep '22

Hi all, In next week's reading discussion I will talk about: Neuro-Symbolic Causal Language Planning with Commonsense Prompting <https://arxiv.org/pdf/2206.02928.pdf> This paper aims to resolve spurious correlations between tasks and action plans learned in LLMs, and proposes to elicit procedural knowledge from the LLMs with commonsense-infused prompting. I highly recommend you read the paper in advance and think about: 1. What's the motivation, novelty, and contribution of this work? 2. What we can learn from this work? Any interesting idea we can derive from this work? 3. What are the limitations and remaining challenges of this work? 4. Any idea to address the remaining challenges? We will hold the discussion on Monday 4-5 pm in Torgersen 3160A. You may also join via Zoom https://virginiatech.zoom.us/j/5961845863. Hope everyone has a great weekend. Happy reading! Best regards, Minqian Liu

1 0

[nlplab] Reading Discussion on Monday Sep 26th
by Barry Yao 25 Sep '22

25 Sep '22

Hello everyone, In next week's reading discussion, we will talk about: Generative Cooperative Networks for Natural Language Generation <https://arxiv.org/abs/2201.12320> *Abstract:* Generative Adversarial Networks (GANs) have known tremendous success for many continuous generation tasks, especially in the field of image generation. However, for discrete outputs such as language, optimizing GANs remains an open problem with many instabilities, as no gradient can be properly back-propagated from the discriminator output to the generator parameters. An alternative is to learn the generator network via reinforcement learning, using the discriminator signal as a reward, but such a technique suffers from moving rewards and vanishing gradient problems. Finally, it often falls short compared to direct maximum-likelihood approaches. In this paper, we introduce Generative Cooperative Networks, in which the discriminator architecture is cooperatively used along with the generation policy to output samples of realistic texts for the task at hand. We give theoretical guarantees of convergence for our approach and study various efficient decoding schemes to achieve state-of-the-art results in two main NLG tasks empirically. I recommend you read the paper in advance and think about: 1. Why does their method work? 2. How to apply their method to other tasks? We will hold the discussion on Monday 4-5 pm in Torgersen 3160A. You may also join via Zoom <https://virginiatech.zoom.us/j/4506040955?pwd=TGFtTUVWR2tuYmVlS3JraTQzbjgrU…> . Sorry for the short notice. Best regards, Barry Menglong Yao Ph.D. student in Computer Science Virginia Tech Homepage <https://barry-yao.netlify.app/>

1 0

[nlplab] Reading Discussion on Monday Sept 19
by Ying Shen 19 Sep '22

19 Sep '22

Hello everyone, In next week's reading discussion, we will talk about: Diffusion-LM Improves Controllable Text Generation <https://arxiv.org/abs/2205.14217> This paper proposes to improve controllable text generation via a new language model based on continuous diffusions. I highly recommend you read the paper in advance and think about: 1. What's the diffusion model? 2. What method does Diffusion-LM use to control the generation of text? 3. Compared to other works on controllable text generation, what is the main advantage of this work? Any interesting idea we can derive from this work? 4. What are the limitations and remaining challenges of this work? 5. Any idea to address the remaining challenges? We will hold the discussion on Monday 4-5 pm in Torgersen 3160A. You may also join via Zoom https://virginiatech.zoom.us/j/5961845863. Hope everyone has a great weekend. Happy reading! Best, Ying

2 1

[nlplab] Reading Discussion on Monday Sept 5
by Minqian Liu 02 Sep '22

02 Sep '22

Hi all, In next week's reading discussion I will talk about: Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? <https://arxiv.org/pdf/2202.12837.pdf> This paper analyses the effect of demonstrations on in-context learning, and I think it's very interesting. I highly recommend you read the paper in advance and think about: 1. What's the motivation, novelty, and contribution of this work? 2. What we can learn from this work? Any interesting idea we can derive from this work? 3. What are the limitations and remaining challenges of this work? 4. Any idea to address the remaining challenges? We will hold the discussion on Monday 4-5 pm in Torgersen 3160A. You may also join via Zoom https://virginiatech.zoom.us/j/5961845863. Hope everyone has a great weekend. Enjoy reading! Best, Minqian Liu

1 1