start sloppy, refine in motion.
Directional Ablation, and How Astra Fixed Qwen3.5-0.8B's Reasoning Loops
How an LLM edited 0.016% of Qwen3.5-0.8B’s weights to break its reasoning loops: directional ablation from the refusal-direction work, then pushed further with a strength α and 64 selected neurons per layer.
Fixing Softmax and Cross-Entropy: Two Numerical Pitfalls
Softmax Collapse and Softmax Cancellation: two floating-point pitfalls in Softmax and Cross-Entropy, and mathematically equivalent fixes.
Rotation Representations Part 1: Rotation Matrices, RoPE, and 6D/9D
Rotation matrices, complex numbers and RoPE, and the 6D and 9D rotation representations used in neural networks.
Why Training Loss Sawtooths Across Epochs
Why does training loss creep up within each epoch and drop at the next? Not the optimizer, but what the loss measures: the expected gap since a sample was last trained on follows a sawtooth, and it explains every sampling ablation.
From 1000 Steps to 1: Building Diffusion and Flow Models from Scratch
Clean, minimal PyTorch implementations of diffusion and flow models, from 1000-step DDPM to 1-step generators, on one backbone with one set of conventions.