Draft Thesis_edit Thesis view Ongoing some baseline papers: Modeling Content-Emotion Duality via Disentanglement for Empathetic Conversation

NIPS 2023 check 22 24 27 papers Introduction Contrastive learning has recently emerged as a promising approach for learning data representations that discover and disentangle the explanatory factors of the data. (advantage)Learning to disentangle the explanatory factors of the observed data is valuable for a variety of machine learning (ML) applications. Contrastive methods (approximately) invert the data generating process and thus recover the generative factors. We aim to develop a unified framework of statistical priors on the data generating process to improve our understanding of CL for disentangled representations. (motivation and issue)

INSIDE: LLMS’ INTERNAL STATES RETAIN THE POWER OF HALLUCINATION DETECTION ICLR2024 image image A Mathematical Investigation of Hallucination and Creativity in GPT Models image

NIPS 2022 Outline Intoduction Method Experiment Conclusion

Real time series Machine learning for real-time aggregated prediction of hospital admission for emergency patients Real-time Forecasting of Time Series in Financial Markets Using Sequentially Trained Many-to-one LSTMs image image They use sequential training

Outline Introduction Approach Experiments Conclusion Introduction Previous works dropped into the paradigm of non-goal-oriented knowledge-driven dialog. They are prone to ignore the effect of dialog goal, which has potential impacts on knowledge exploitation and response generation.

Challenges and Applications of Large Language Models 8/1 Task To resovle Fine-Tuning OverheadThe additional computational and memory resources required to adapt a pre-trained Large Language Model to perform well on a specific downstream task. Limited Context Length The challenge of processing long inputs in natural language processing (NLP) tasks.

:::spoiler LLaMA LLaMA lit-llama LLaMA-Adapter LLaMA: Open and Efficient Foundation Language Models (paper) LLaMA 2 ::: :::spoiler ChatGLM ChatGLM-6B code

Outline Show papersA Pseudo-Semantic Loss Controllable Abstractive Summarization Constrained Abstractive Summarization KITAB Comparison papers A Pseudo-Semantic Loss for Autoregressive Models with Logical Constraints

Outline Introduction Latent Variable Distillation for Hidden Markov Model Latent Variable Distillation for Probabilistic Circuits Efficient Parameter Learning Extracting Latent Variables for Image Modeling Experiments Conclusion Introduction

2023/08/22 Outline Introduction Related work Guiding Autoregressive Generation with Tractable Probabilistic Models Efficient Probabilistic Reasoning with Hidden Markov Models Experiments Conclusion

Outline Introduction Preliminaries Bridging the Gap – a Unified View Experiment Conclusion Introduction Recent work has proposed a variety of parameter-efficient transfer learning methods that only fine-tune a small number of extra parameters to attain strong performance. While effective, the critical ingredients for success and the connections among the various methods are poorly understood.How are these methods connected?

Outline Introduction Approach Experiment Conclusion Introduction Prompt tuning (PT) prepends tunable continuous prompt vectors to the input, has emerged as a promising approach for parameter-efficient transfer learning with PLMs. We propose multitask prompt tuning (MPT), which first learns a single transferable prompt by distilling knowledge from multiple task-specific source prompts. We learn multiplicative low-rank updates to this shared prompt to efficiently adapt it to each downstream target task.We decompose the soft prompt of each source task into a multiplication of a shared matrix and a low-rank task-specific matrix.

ICLR 2022 Outline Introduction Related work Background Approach Experiments Conclusion and future work

Outline Abstract Introduction Related Work Plug and Play Language Models Experiments, Results, and Evaluation Conclusion Abstract Controlling attributes of the generated language is difficult without modifying the model architecture or fine-tuning on attribute-specific data and entailing the significant cost of retraining.

2023/3/2 PPLM Quark A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Models ChatGPT: Jack of all trades, master of none

Quark: Controllable Text Generation with Reinforced [Un]learning github Outline Information Quark: Quantized Reward Konditioning Experiments Model Ablations Related Work Conclusion

2023/2/15 Published as a conference paper at ICLR 2020 Reference PLUG AND PLAY LANGUAGE MODELS 1. LANGUAGE MODELING WITH TRANSFORMERS In this paper, we use a transformer to model the distribution of natural language.

2023/2/14 Published as a conference paper at ICLR 2022 Outline Abstract Introduction Background Value-Gradient weighted Model loss (VaGraM) Experiment: Model Learning In Low-Dimensional Problem

2022/12/27 Abstract The new framework is centered around a weak version of the concentrability coefficient that measures the deviation of the behavior policy from the expert policy alone. We consider a lower confidence bound (LCB) algorithm developed based on pessimism in the face of uncertainty in offline RL. Outline 1. Introduction 2. Background and problem formulation 3. A warm-up: LCB in multi-armed bandits