← SnapDragon-Coder

Rabbit holes

• Domain-REAP. If most of a mixture-of-experts model’s experts are idle on Go, prune to the ones that fire and serve something a quarter the size. Carves were built and scored on the frozen 77-task set. Dead end: the full base beat every carve (63 vs 34 vs 23), and the low scores were driven by the pruning criterion, not the data; the keep-set overlapped the reference pruning by 89.6%. Closed. ¹
• Community quantisation as the base. The 4-bit community build was the only way to fit the model on one card at first. The ablation that finally isolated it showed op-adoption 0.82 quantised versus 0.98 in bf16, with the adapter compounding the loss. Dead end for the quant; became the bench lane, which exists to measure this before trusting it. ²
• A frontier open model as teacher. Its weights were archived locally as a hedge. Running it at home managed about a quarter of a token per second. Dead end as a teacher (corpus-scale generation needs an API or rented multi-node serving); kept as an exact-reference path.³
• Two jobs on one GPU. A training run and a serving process collided on the same card. Became the GPU lease manager in fleet ops: heartbeat, release, and a read-only check before any GPU work.
• “Train small from scratch instead.” The handicapped-base finding raised the question of whether the fine-tuning was the problem. Became shellm.