Skip to content

Latest commit

 

History

History
39 lines (34 loc) · 4.3 KB

README.md

File metadata and controls

39 lines (34 loc) · 4.3 KB
Raw

LLM-PaperReview

Review papers of NLP, mainly LLM. NLP / LLM 관련 논문 리뷰 레포입니다.

  • 스케일부터 메소드까지. ChatGPT의 상업적 성공과 여러 발군의 오픈소스들을 업고 LLM은 나날이 빠른 속도로 성장하고 있습니다.
  • 쏟아지는 LLM 관련 논문들을 깊고 넓게 공부하고 토론하고자 합니다. :)
  • 논문 선정은, Pond에 모아둔 논문들 중 하나를 발표자가 발표일 1주 전까지 선정하여 공지하는 것으로 이루어집니다.

Semesters

  • 2023 8/6 - 2023 10/19 Every Thursday(Finished).
  • 2023 11/23 - 2024 2/1 Every Thursday(Finished).

일정 및 선정 논문

2기

  Paper a.k.a Affiliation published date Speaker Youtube
11.23 REPLUG: Retrieval-Augmented Black-Box Language Models REPLUG Washington Univ. May. 2023 김한성 LINK
11.30 Prefix-Tuning: Optimizing Continuous Prompts for Generation Prefix-Tuning Standford Univ. Jan. 2021 임서연 LINK
12.7 QLoRA: Efficient Finetuning of Quantized LLMs QLoRA Washington Univ. May. 2023 이상민 LINK
12.21 Direct Preference Optimization: Your Language Model is Secretly a Reward Model DPO Stanford Univ. May. 2023 천재원 LINK
12.28 Efficient Streaming Language Models with Attention Sinks StreamingLLM MIT Dec. 2023 김가영 LINK
1.4 Efficient Memory Management for Large Language Model Serving with PagedAttention vLLM UC Berkeley Sep. 2023 이주형 LINK
1.18 Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks RAG FAIR April. 2021 신중현 LINK
1.25 Mixtral of Experts Mixtral Mistral.AI Jan. 2024 신혁준 LINK
2.1 Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functions - Oxford Univ. Oct. 2023 김현수 LINK

1기

  Paper a.k.a Affiliation published date Speaker Youtube
8.17 RoFormer: Enhanced Transformer with Rotary Position Embedding RoPE Zhuiyi Technology August. 2022 천재원 LINK
8.24 TRAIN SHORT, TEST LONG:
ATTENTION WITH LINEAR BIASES
ENABLES INPUT LENGTH EXTRAPOLATION		 ALiBi  Facebook April. 2022 이주형 LINK
8.31 Finetuned Language Models Are Zero-Shot Learners FLAN Google Sep. 2021 천소영 LINK
9.7 WizardLM: Empowering Large Language Models to Follow Complex Instructions WizardLM Microsoft Jun. 2023 박경택 LINK
9.14 G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment G-Eval Microsoft May. 2023 신혁준 LINK
9.21 SimCSE: Simple Contrastive Learning of Sentence Embeddings SimCSE Princeton Univ. May. 2022 김세형 LINK
10.5 LLaMA: Open and Efficient Foundation Language Models LLaMA Meta Feb. 2023 김가영 LINK
10.12 LoRA: Low-Rank Adaptation of Large Language Models LoRA Microsoft Oct. 2021 신중현 LINK
10.19 Training language models to follow instructions with human feedback InstructGPT OpenAI March. 2022 홍영훈 LINK