GitHub - Saivineeth147/lora-speedrun: Speedrunning LoRA fine-tuning: frozen task, frozen hardware, public wall-clock leaderboard. modded-nanogpt for fine-tuning.

▤ Summary

This article introduces a public GitHub project that benchmarks LoRA fine-tuning speed on a fixed task and fixed hardware setup. It uses a single L40S GPU on Modal and publishes wall-clock leaderboard results that anyone can reproduce and verify. The project focuses on parameter-efficient fine-tuning techniques such as QLoRA, DoRA, PiSSA, and Unsloth kernels. It also defines a strict submission and verification process so contributors can compare optimization ideas under identical conditions.

▦ Linked Companies

Modal

$1M to $5M