What I learned from building a Deep Convolutional GAN from scratch, and everything that went wrong along the way.
Generate realistic human faces from noise. That is the classic GAN benchmark: take a random vector from a latent space and produce an image that looks like a real face. Simple to state, genuinely hard to achieve.
I used the NVIDIA FFHQ dataset (Flickr-Faces-HQ), specifically the 128x128 thumbnail version containing 65,534 high-quality face images. The task: train a DCGAN to generate new faces that could plausibly belong to that dataset.
This is not a success story. It is a record of what I built, what broke, and what I would never do again.
The generator (generator_upsample) takes a 10x10 latent sample drawn from a standard normal distribution (mean 0, std 1) and upsamples it progressively to a 128x128 RGB image.
The head is a single Conv2d layer that projects the latent input into 1024 feature maps. The body is a series of upsample-then-convolve stages working through the channel hierarchy:
Latent (1, 10, 10)
Conv2d head: 1 → 1024 channels
Upsample x2 + Conv2d: 1024 → 512
Upsample x2 + Conv2d: 512 → 256
Upsample x2 + Conv2d: 256 → 128
Upsample x2 + Conv2d: 128 → 64
Upsample x2 + Conv2d: 64 → 32
Upsample x2 + Conv2d: 32 → 3 (RGB output, 128x128)Each stage uses LeakyReLU activations and LazyBatchNorm2d. The upsample-then-convolve approach was chosen over transposed convolutions to avoid the checkerboard artefact problem that transposed convolutions are notorious for in GAN outputs.
Weights are initialised from a normal distribution (mean 0, std 0.02) for Conv layers and (mean 1, std 0.02) for BatchNorm layers — the standard DCGAN weight initialisation that keeps activations stable from the very first forward pass.
The discriminator (discriminator_conv) takes a 128x128 RGB image and outputs a single scalar score indicating real or fake. It progressively downsamples through seven convolutional stages, mirroring the generator's channel hierarchy in reverse:
Input (3, 128, 128)
Conv2d: 3 → 32, stride 2, kernel 4
Conv2d: 32 → 64, stride 2, kernel 4
Conv2d: 64 → 128, stride 2, kernel 4
Conv2d: 128 → 256, stride 2, kernel 4
Conv2d: 256 → 512, stride 2, kernel 4
Conv2d: 512 → 1024, stride 2, kernel 4
→ scalar outputEvery discriminator layer includes a Gaussian_noise module that injects noise during training, plus LeakyReLU activations. The noise injection is a deliberate stability mechanism: it prevents the discriminator from becoming too confident too quickly, which would starve the generator of useful gradient signal.
The discriminator also includes a decay_gauss_std function that reduces the injected noise magnitude each epoch. Noise is helpful early in training when the generator is producing garbage, but should step back as the generator improves.
t_critic steps (set to 1, so both update together). An RMSprop variant with weight clipping was tried first (the Wasserstein GAN approach) before switching to Adam with BCEWithLogitsLoss.One of the most important things I added to this project was gradient flow visualisation. The plot_grad_flow function plots the average absolute gradient per layer across the network during training.
Running plot_grad_flow on both networks at regular intervals throughout training is what allowed me to diagnose most of the problems I hit.
The README for this repo describes it as a "personal cautionary tale when working with GANs." That description is accurate. Here is the full list of hard lessons.
The preprocessing pipeline does two things: normalise (zero mean, unit variance) and rescale (to [-0.5, 0.5]). I did these in the wrong order on the first attempt.
If you rescale first and then normalise, you corrupt the statistics that normalisation depends on. The data distribution going into the network was wrong, which meant the discriminator was learning from differently-distributed inputs than the generator was producing. The outputs were incoherent and the loss curves made no sense.
Correct order: normalise first, then rescale. Every time.
Early versions of the discriminator used kernel sizes that did not divide the spatial dimensions cleanly given the stride. This produced spatial dimension mismatches mid-network, requiring awkward padding fixes that introduced their own artefacts.
The kernel size of 4 with stride 2 is the standard DCGAN configuration for a reason: it halves spatial dimensions cleanly at each layer. Deviating from it requires careful arithmetic and usually produces worse results.
Most deep learning models are somewhat robust to hyperparameter choices — you can be within a factor of 5 on the learning rate and still get something reasonable. GANs are not like this.
Learning rate, kernel size, stride, padding, the rescale range, the latent space size — all of these interact. A learning rate that works perfectly with one kernel configuration breaks completely with another. There is no shortcut here. You have to run experiments, monitor gradient flow, and adjust systematically.
The generator's quality is almost entirely determined by the quality of the signal it receives from the discriminator. If the discriminator's gradient flow is poor, the generator will not learn regardless of how well it is architected.
The workflow I settled on: get the discriminator training stably on real data alone before introducing the generator at all. Verify gradient flow is healthy. Then bring the generator in.
It is tempting to throw regularisation at a struggling GAN early. Gaussian noise injection, dropout, spectral normalisation — all of these can help. But if the fundamental hyperparameters are wrong, regularisation just masks the problem without fixing it.
Get the hyperparameters right first. Verify gradients are flowing. Then add regularisation on top of a working baseline.
The generated faces show clear structure: skin tones, rough positioning of facial features, some colour coherence. They are not photorealistic. At 500 epochs with this architecture and dataset size, the generator has learned the broad statistical properties of faces but has not captured fine detail.
The gradient flow plots show a discriminator that is learning consistently throughout training. The loss curves for both generator and discriminator follow the expected adversarial pattern — neither collapses to zero, neither diverges. That alone represents a significant improvement over the early runs where the discriminator would win completely within the first few epochs.
Progressive growing. ProGAN's approach of starting at low resolution (4x4) and progressively increasing to full resolution produces dramatically better results than training at full resolution from the start. The network learns coarse structure before being asked to learn fine detail.
Spectral normalisation on the discriminator. It stabilises training more reliably than noise injection alone by constraining the Lipschitz constant of the discriminator.
Wasserstein loss with gradient penalty (WGAN-GP). The original WGAN used weight clipping which is crude. WGAN-GP replaces it with a gradient penalty that provides a much cleaner training signal and far more interpretable loss curves.
FID score tracking. Fréchet Inception Distance is the standard quantitative metric for GAN output quality. Training for 500 epochs without a proper metric means flying blind — loss curves tell you about training stability but not about output quality.
Full source: github.com/Kikumu/DCGAN-with-facedata