LLM-basic

Pytorch

• core

Transformer

MoE

• MoE on NPU
• MoE from scratch

DeepSeek

R1

• pre-train

  • GRPO R1-zero like recurrent from scratch on Qwen

• post-training

  • SFT for R1
  • Distill for R1[黑盒]

V3

• DeepSeek-V3 MoE from scratch

• MTP

  • MTP on Qwen

• MLA

Knowledge Distillation

• [白盒]

RL

pipeline

• PPO
• GRPO
• DPO

TRL

• ppo
• reward model

personal_chatgpt

• llama 源码阅读
• LoRA & PEFT
• DL