课程依赖 Verifiers、vLLM 等外部项目，需自行按章节安装；需额外下载 LiquidAI/LFM2-2.6B 模型权重；README 未给出环境依赖版本，需留意各章节说明
llm-rl-environments-lil-course
anakin87Apache-2.0更新放缓一门小课程，教你搭建强化学习环境来评估和训练语言模型，涵盖 GRPO、RLVR 和井字棋等实例。
🌱 A little course on Reinforcement Learning Environments for evaluating and training Language Models
暂无版本243star19fork同步于 2026-11-07 07:29 UTC
README 未提供安装命令，需从 chapters/01.md 开始按章节操作 硬件要求按 README 说明
README 未说明显存、CPU 或 GPU 型号要求；课程涉及 vLLM 推理与 LiquidAI/LFM2-2.6B 模型训练，但未给出具体硬件门槛。
支持 Apple Silicon
一门用井字棋环境讲 LLM 强化学习训练与评估的小型实战课程
一门小课程，教你搭建强化学习环境来评估和训练语言模型，涵盖 GRPO、RLVR 和井字棋等实例。
核心能力
- 用 Verifiers 库把 RL 环境写成软件工件
- 从零构建井字棋环境并用于模型评估
- 用合成数据做 SFT 预热再进入 RL 训练
- 把 LFM2-2.6B 小模型训练成井字棋高手
- 附失败实验复盘章节，讲清哪些做法没奏效
容易踩的坑
内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README
暂无 release 数据（可能没有发版，或尚未同步）。
用户点评
登录用户的真实使用记录，可按版本和硬件筛选共 0 条
还没有人点评。第一个说出真实体验的人最有价值。
LLM RL Environments Lil Course
A little course on Reinforcement Learning Environments for evaluating and training Language Models.
Unlike classic fine-tuning, RL environments let models explore and improve beyond what curated datasets can teach.
In this course, we'll build a Tic Tac Toe environment and use it to transform a Small Language Model
(LiquidAI/LFM2-2.6B) into a master player that beats gpt-5-mini.
➡️ Start here: Chapter 1 - Agents, Environments, and LLMs
🎥 Video walkthrough @ AI Engineer
🤗🕹️ Play against Mr. Tic Tac Toe
Who is this course for?
- AI Engineers: You are familiar with classic LLM fine-tuning techniques (Supervised Fine-Tuning) but have little to no experience with Reinforcement Learning.
- Traditional RL Practitioners: You know how RL works, but you want to learn how to apply it to Language Models.
- Curious Tinkerers: You keep hearing about "reasoning models" and RL post-training, and you want to see how it works under the hood.
Chapters
➡️ Start here: Chapter 1 - Agents, Environments, and LLMs
- Agents, Environments, and LLMs: mapping Reinforcement Learning concepts to the LLM domain.
- Verifiers: an open-source library to build RL environments as software artifacts.
- Developing a Tic Tac Toe environment with Verifiers
- Evaluating existing models with RL environments
- Training preparation and synthetic data generation for Supervised Fine-Tuning
- Supervised Fine-Tuning warm-up
- Reinforcement Learning training to teach our model Tic Tac Toe
- Reinforcement Learning pt.2: towards Tic Tac Toe mastery
- What did not work: a Tic Tac Toe Post-Mortem from my failed experiments
- What we have learned and the future
Technologies
This course is not affiliated with any of the following projects:
Course author
Stefano Fiorucci/anakin87
- 🏗️ AI orchestration by day (Haystack developer)
- Small Language Models post-training, RL tinkering by night 🌙
I built this course from hands-on experimentation. If you spot any errors, please open a GitHub issue.
Feel free to follow me on my social profiles: GitHub, LinkedIn, X, Hugging Face.