Avi Chawla Demonstrates Building a Reasoning Model with Open-Source Tools
Avi Chawla recently shared a LinkedIn post detailing the creation of a reasoning model, similar to DeepSeek-R1, utilizing open-source tools and local resources. This development leveraged Llama 3.1-8B as the Large Language Model (LLM) and the 'Unsloth' fine-tuning library to enhance reasoning capabilities. By employing LoRA, the team avoided the extensive computational demands typically needed for tuning all of the model’s weights. The model’s development focused on structured thinking by training it on the GSM8K dataset that contains math word problems, thereby encouraging step-by-step problem-solving. They also used custom reward functions to refine reasoning precision. A significant part of the training harnessed GRPO, a reinforcement learning technique that enhances performance without needing separate value functions unlike traditional PPO. Post fine-tuning, the model showed significant improvements in delivering accurate reasoning. In a bid to support educational initiatives in AI/ML, Avi shared the code publicly and offered a free 530-page PDF filled with over 150 lessons in Data Science and Machine Learning, emphasizing the potential of structured and innovative methods to enhance AI model reasoning.