October 2022 - nlp_reading_group

[nlplab] Reading Discussion on Monday Oct. 31st
by Barry Yao 29 Oct '22

29 Oct '22

Hello everyone, In next week's reading discussion, we will talk about the following NeurIPS 2022 paper: Adaptive Multi-stage Density Ratio Estimation for Learning Latent Space Energy-based Model <https://arxiv.org/pdf/2209.08739.pdf> Abstract: This paper studies the fundamental problem of learning energy-based model (EBM) in the latent space of the generator model. Learning such prior model typically requires running costly Markov Chain Monte Carlo (MCMC). Instead, we propose to use noise contrastive estimation (NCE) to discriminatively learn the EBM through density ratio estimation between the latent prior density and latent posterior density. However, the NCE typically fails to accurately estimate such density ratio given large gap between two densities. To effectively tackle this issue and learn more expressive prior models, we develop the adaptive multi-stage density ratio estimation which breaks the estimation into multiple stages and learn different stages of density ratio sequentially and adaptively. The latent prior model can be gradually learned using ratio estimated in previous stage so that the final latent space EBM prior can be naturally formed by product of ratios in different stages. The proposed method enables informative and much sharper prior than existing baselines, and can be trained efficiently. Our experiments demonstrate strong performances in image generation and reconstruction as well as anomaly detection I* recommend you read the paper in advance and think about the followings:* 1, Why do they execute the density ratio estimation for multiple stages? 2, Why do they learn the energy-based model in latent space? 3, Could we apply density ratio estimation in image space with VAE? We will hold the discussion on Monday 4-5 pm in Torgersen 3160A. You may also join via Zoom <https://virginiatech.zoom.us/j/4506040955?pwd=TGFtTUVWR2tuYmVlS3JraTQzbjgrU…> . Hope everyone has a great weekend. Happy reading! Best regards, Barry Menglong Yao Ph.D. student in Computer Science Virginia Tech Homepage <https://barry-yao.netlify.app/>

1 0

[nlplab] Reading Discussion on Monday Oct 24
by Ying Shen 21 Oct '22

21 Oct '22

Hello everyone, In next week's reading discussion, we will talk about: Composable Text Controls in Latent Space with ODEs <https://arxiv.org/abs/2208.00638> This paper proposes a new efficient approach for composable text controls in the compact latent space of the text. We will hold the discussion on Monday 4-5 pm in Torgersen 3160A. You may also join via Zoom https://virginiatech.zoom.us/j/5961845863. Hope everyone has a great weekend. Happy reading! Best, Ying

1 0

[nlplab] Reading Discussion on Monday Oct 17th
by Zhiyang Xu 17 Oct '22

17 Oct '22

Hi guys, In the next week's reading discussion, I will talk about LARGE LANGUAGE MODELS ARE HUMAN-LEVEL PROMPT ENGINEERS <https://openreview.net/pdf?id=92gvk82DE-> which addresses the problem of automatic instruction generation and selection. Zoom discussion is available at https://virginiatech.zoom.us/j/5961845863. Best, Zhiyang Xu

1 1

Reading Discussion on Monday Oct 10
by Sijia Wang 08 Oct '22

08 Oct '22

Hi guys, Next week's reading discussion will be on the following paper Few-Shot Diffusion Models <https://arxiv.org/abs/2205.15463>. I may also cover some discussions from the survey paper on diffusion models A Survey on Generative Diffusion Model. Several questions might help you go through these work are: 1. What are the challenges for few-shot diffusion compared with general diffusion approaches? 2. What makes the proposed method work? 3. How can we extend this work to few-shot language generation? We will hold the discussion meeting in Torgersen 3160A, 4-5 pm on Oct 10. In case the room will be occupied, please get there by 4pm so that you can be informed of the new meeting location. Zoom discussion is available at https://virginiatech.zoom.us/j/5961845863. Enjoy reading! -- Sijia Wang Graduate Student, Department of Computer Science, Virginia Tech, Blacksburg, VA 24061 Phone: +1 (314) 665 8146

1 0