Qwen2.5 Coder 1.5B Roblox

This model is built on Qwen2.5-Coder-1.5B-Instruct and has been fine-tuned exclusively on the official Roblox Luau corpus. It is designed to assist developers with code generation, completion, and understanding of Luau patterns commonly used in Roblox game development.

Key Features

Fine-tuned on quality-filtered Luau code from the official Roblox corpus

Optimized for Roblox-specific APIs and development patterns

Trained on TPU v5e-8 infrastructure

Includes interactive Colab notebook for immediate testing

Technical Details

The model uses Low-Rank Adaptation with target modules q_proj and v_proj, trained on filtered data (20-5000 character code samples) with a 90/10 train-validation split. Training was conducted with a batch size of 4, gradient accumulation of 32 steps, and a learning rate of 3e-5 using cosine annealing.

Access

Model: Interactive Demo: Available via Google Colab (link in model card)

The model is released under Apache 2.0 license and is ready for integration into development workflows.

Feedback and contributions from the community are welcome.