Top Important LLM Papers for the Week from 05/02 to 11/02

Stay Updated with Recent Large Language Models Research

Large language models (LLMs) have advanced rapidly in recent years. As new generations of models are developed, researchers and engineers need to stay informed on the latest progress. This article summarizes some of the most important LLM papers published during the Second Week of February 2024.

The papers cover various topics shaping the next generation of language models, from model optimization and scaling to reasoning, benchmarking, and enhancing performance. Keeping up with novel LLM research across these domains will help guide continued progress toward models that are more capable, robust, and aligned with human values.

Table of Contents:

1. LLM Progress & Benchmarking
2. LLM Reasoning
3. LLM Training & Evaluation
4. Transformers & Attention Based Models

Most insights I share in Medium have previously been shared in my weekly newsletter, To Data & Beyond.

If you want to be up-to-date with the frenetic world of AI while also feeling inspired to take action or, at the very least, to be well-prepared for the future ahead of us, this is for you.

🏝Subscribe below🏝 to become an AI leader among your peers and receive content not present in any other platform, including Medium:

1. LLM Progress & Benchmarking

1. StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback
2. PokéLLMon: A Human-Parity Agent for Pokémon Battles with Large Language Models
3. TravelPlanner: A Benchmark for Real-World Planning with Language Agents
4. Nomic Embed: Training a Reproducible Long Context Text Embedder
5. BlackMamba: Mixture of Experts for State-Space Models
6. OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models
7. Grandmaster-Level Chess Without Search
8. Direct Language Model Alignment from Online AI Feedback
9. Multi-line AI-assisted Code Authoring
10. Code Representation Learning At Scale
11. Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks
12. EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters
13. Scaling Laws for Downstream Task Performance of Large Language Models
14. CodeIt: Self-Improving Language Models with Prioritized Hindsight Replay
15. More Agents Is All You Need
16. An Interactive Agent Foundation Model
17. Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domains
18. In-Context Principle Learning from Mistakes
19. Memory Consolidation Enables Long-Context Video Understanding
20. Driving Everywhere with Large Language Model Policy Adaptation
21. Multilingual E5 Text Embeddings: A Technical Report
22. WebLINX: Real-World Website Navigation with Multi-Turn Dialogue
23. SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models
24. SpiRit-LM: Interleaved Spoken and Written Language Model

2. LLM Reasoning

1. K-Level Reasoning with Large Language Models
2. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
3. Self-Discover: Large Language Models Self-Compose Reasoning Structures

3. LLM Training, Inference, Evaluation & Optimization

1. Specialized Language Models with Cheap Inference from Limited Domain Data
2. Rethinking Interpretability in the Era of Large Language Models
3. Rethinking Optimization and Architecture for Tiny Language Models
4. LiPO: Listwise Preference Optimization through Learning-to-Rank
5. Shortened LLaMA: A Simple Depth Pruning for Large Language Models
6. BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
7. TP-Aware Dequantization
8. Hydrogen: High-Throughput LLM Inference with Shared Prefixes
9. Offline Actor-Critic Reinforcement Learning Scales to Large Models

4. Transformers & Attention Based Models

1. Repeat After Me: Transformers are Better than State Space Models at Copying
2. Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers
3. The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry

5. LLM Fine-Tuning

1. Fine-Tuned Language Models Generate Stable Inorganic Materials as Text

If you like the article and would like to support me, make sure to:

• 👏 Clap for the story (50 claps) to help this article be featured
• Subscribe to To Data & Beyond Newsletter
• Follow me on Medium
• 📰 View more content on my medium profile
• 🔔 Follow Me: LinkedIn |Youtube | GitHub | Twitter

Subscribe to my newsletter To Data & Beyond to get full and early access to my articles:

Are you looking to start a career in data science and AI and do not know how? I offer data science mentoring sessions and long-term career mentoring:

• Mentoring sessions: https://lnkd.in/dXeg3KPW
• Long-term mentoring: https://lnkd.in/dtdUYBrM