liqq

Directional Ablation, and How Astra Fixed Qwen3.5-0.8B's Reasoning Loops

Wed, 04 Nov 2026 00:00:00 +0000

How an LLM edited 0.016% of Qwen3.5-0.8B’s weights to break its reasoning loops: directional ablation from the refusal-direction work, then pushed further with a strength α and 64 selected neurons per layer.

Fixing Softmax and Cross-Entropy: Two Numerical Pitfalls

Sun, 25 Oct 2026 00:00:00 +0000

Softmax Collapse and Softmax Cancellation: two floating-point pitfalls in Softmax and Cross-Entropy, and mathematically equivalent fixes.

Rotation Representations Part 1: Rotation Matrices, RoPE, and 6D/9D

Sat, 24 Oct 2026 00:00:00 +0000

Rotation matrices, complex numbers and RoPE, and the 6D and 9D rotation representations used in neural networks.

Why Training Loss Sawtooths Across Epochs

Thu, 15 Oct 2026 00:00:00 +0000

Why does training loss creep up within each epoch and drop at the next? Not the optimizer, but what the loss measures: the expected gap since a sample was last trained on follows a sawtooth, and it explains every sampling ablation.

From 1000 Steps to 1: Building Diffusion and Flow Models from Scratch

Fri, 09 Oct 2026 00:00:00 +0000

Clean, minimal PyTorch implementations of diffusion and flow models, from 1000-step DDPM to 1-step generators, on one backbone with one set of conventions.

About

Mon, 01 Jan 0001 00:00:00 +0000

I work on generative models. This blog is where I write up things I build to understand them.

2022 – 2026
Viggle AI

Generative Models
2021 – 2022
University of Waterloo

Master of Electrical and Computer Engineering
2015 – 2021
SenseTime

Computer Vision
2010 – 2015
The Chinese University of Hong Kong

Bachelor of Information Engineering