scott-k.dev
SYS / BLOG
Notes on building, thinking, and what I'm currently up to
20 posts
posts
~8 min
Building a Convolutional Neural Network From Scratch in C++
No Python, no PyTorch, no autograd. A full CNN with hand-written convolutions, pooling, and backpropagation using only Eigen and OpenCV.
~10 min
Learning NLP From the Ground Up
From tokenisation and one-hot encoding to skip-gram embeddings, positional encoding, LSTM, and attention — a complete bottom-up reconstruction of the ideas behind modern NLP.
~7 min
Predicting Rainfall and Flooding With a CNN-LSTM Hybrid
Using spatiotemporal deep learning to predict the next frame of a flood sequence, without ever seeing the correct answer first.
~8 min
A Personal Cautionary Tale: Training a DCGAN on Face Data
What I learned from building a Deep Convolutional GAN from scratch — five hard lessons about normalisation order, hyperparameter sensitivity, and why you should monitor the discriminator first.
~8 min
Teaching a Genetic Algorithm to See Heat: NEAT on a Raspberry Pi Thermal Camera
A from-scratch NEAT implementation that tunes its own colour space parameters to identify objects through smoke, no labels required.
~7 min
LBCGAN: Using Location Data to Generate Scene-Aware Images
A conditional GAN prototype that uses location coordinates as a conditioning signal, designed to overlay depth-aware scene generation on top of thermal camera contours.
~10 min
A Deep Dive Into Reinforcement Learning: Seven Algorithms From Scratch
Implementing and comparing policy gradient, actor-critic, and off-policy methods in PyTorch — from VPG through to SAC with PER.
~12 min
System Zero: Building a Self-Hosted Agentic RAG Platform From Scratch
A full-stack agentic AI platform running on bare metal — hybrid retrieval, ReAct reasoning, MCP tool calls, and what it takes to own the entire stack.
~8 min
Teaching a Drone to Fly With Reinforcement Learning
Building an autonomous quadcopter from scratch — simulation, deep RL, and a custom physical build with no flight controller.
~6 min
Going Below PyTorch: Learning Raw CUDA Kernel Development
Why I stopped treating the GPU as a black box — naive matmul, shared memory tiling, and what the 138× gap to cuBLAS actually means.
~10 min
Hand-Writing a Vision-Language Model, Part 1: Qwen3-4B's Attention Stack
RMSNorm, RoPE, and Grouped-Query Attention hand-written from scratch. Real architecture numbers read from the safetensors index before a single line of code.
~10 min
Hand-Writing a Vision-Language Model, Part 2: SwiGLU, DecoderLayer, and a RAM Lesson
The SwiGLU MLP block, the full DecoderLayer, and the complete Model class. Then immediately discovering that 4B parameters at fp32 is 16GB before you've done anything.
~10 min
Hand-Writing a Vision-Language Model, Part 3: The ViT (SigLIP) and the First Bridge to Qwen
The vision half of the project. A hand-written SigLIP ViT verified against the real checkpoint, and the Projector that maps 768-dimensional patch embeddings into Qwen's language space.
~10 min
Hand-Writing a Vision-Language Model, Part 4: Wiring the VLM Together, and What SigLIP's Training Actually Means
The full stack connected and forward-pass verified for the first time. Then the more valuable part: five wrong turns on the way to actually understanding what SigLIP's training does.
~8 min
Hand-Writing a Vision-Language Model, Part 5: Stage 1 Training, The Projector Actually Learns Something
The first time any weight in this project has been updated by gradient descent. Flickr8k dataset, hand-rolled preprocessing, MLflow tracking, and a loss curve that actually drops.
~8 min
Hand-Writing a Vision-Language Model, Part 6: Teaching the VLM to Actually Talk
A real autoregressive generation loop, a FastAPI endpoint for before/after comparison, and watching noise turn into "the dog" for an image of dogs.
~10 min
Hand-Writing a Vision-Language Model, Part 7: Stage 2 Kickoff, Mazes, Coordinates, and a Reserved Vocab Surprise
Stage 2 begins: maze navigation, coordinate tokens woven into reasoning text, and a discovery about what Qwen pre-reserved in its vocab before we even asked.
~12 min
Hand-Writing a Vision-Language Model, Part 8: Hand-Writing LoRA, and a Real Memory Bug That Was Not What It Looked Like
LoRA from scratch, a 1.24GB optimiser cost hiding behind a gradient hook, the tail distribution lesson, and stage 2's first real training run.
~8 min
Hand-Writing a Vision-Language Model, Part 9: Testing What Stage 2 Actually Learned
Correct syntax, correct endpoint, zero repetition: but no stop signal, and a suspiciously generic path. Loss going down and the model learning the task are different things.
~10 min
Hand-Writing a Vision-Language Model, Part 10: The Model Ignoring the Image, and Diagnosing Why
Three different mazes, one identical output. Measuring cosine similarity at each pipeline stage to find where the image's distinguishing information actually gets lost.