shellm
- status
- archived · loss
- when
- 2026-07 → 2026-07
- came from
- SnapDragon-Coder
- led to
- Marilyn
Turn 2: a byte-level language model written in Rust, trained from scratch on a machine’s own man pages and scripts.
Why it started
SnapDragon-Coder showed that fine-tuning a big quantised base left it handicapped. The opposite bet: a tiny model, trained from nothing, on a narrow domain I could fully describe. Also a learning project. I wanted to have written a transformer, not just called one.
What it is
A nanoGPT-style byte-level model in the Burn framework on wgpu/Vulkan, trained on markov’s
card against a corpus mined from the box itself: man pages, shell scripts from every repo I
own, recursive --help output from the CLIs I use, and configuration-management code. The
orchestration was written in Python, then ported to Go against Temporal, which became the
pattern for everything after.1
Where it stands
Archived. It cleared the one milestone I set it, compressing better than gzip at 1.974 bits per byte, and then stalled: the base and the corpus were both far too small to be useful as a shell expert. The repo stays as the reference for the Go orchestration and the dataset builders.2
Lineage
- Came from: SnapDragon-Coder
- Led to: Marilyn. If small-from-scratch is going to work, it needs a real corpus, a real size, and a real question.
Sources
Numbers on this page are quoted from the lab's own records. The records are private; each note gives the record's date.