How I trained a photo colorizer that doesn't redraw faces, for about $50

October 2026 · 9 min read

Ungray is a one-person project: a colorizer for old black-and-white photos, trained on rented GPUs for about $50. Here's how it works, what went wrong along the way, how it measures up, and where it still makes mistakes.

Why not just ask a chatbot?

ChatGPT and Gemini will colorize a photo for free, and the results can look great. But they do it by generating a new image from yours. Every pixel is drawn again, the picture comes back at the model's own size (around 1,000 pixels), and details can drift. You might get a face that's almost, but not quite, your great-grandmother's.

Ungray takes a different approach. A photo can be split into lightness and color. Lightness is every bit of light and shadow, and it holds all the detail: faces, wrinkles, text, film grain. Color is two more channels on top. A black-and-white photo has the lightness and none of the color. Ungray's model predicts only the color and lays it over the original lightness, which stays untouched, pixel for pixel, at full resolution. When it guesses wrong, the worst it can do is pick the wrong shade. It can't move an eye or add a button.

The same design has a limit. Because the lightness is never changed, the model can't correct what the film got wrong. Early "orthochromatic" film was almost blind to red, so red lips and red cloth print nearly black. A colorizer that keeps the original lightness can't turn that dark gray into a bright red.

The model: DDColor's design, trained from scratch

I didn't invent a new architecture. For this job the best open design is still DDColor (ICCV 2023, Apache-2.0), and I forked its code. DDColor's own tests show where its quality comes from. Its color decoder is worth about 2 points of FID (a standard image-quality score, where lower is better). Making the backbone four times bigger gains only 0.46. That made the small "tiny" version the right choice on a budget:

• Encoder: ConvNeXt-T, starting from weights pretrained on ImageNet-22k.
• Decoders: a multi-scale pixel decoder, plus a color decoder with 100 learned "color queries".
• Output: the two color channels of the Lab color space, recombined with the original lightness. The model has about 55 million parameters.
• Training signal: a pixel loss (weight 0.1), a VGG16 perceptual loss (5.0), a PatchGAN discriminator (1.0) and a small colorfulness term (0.1).

The model learns at 256 pixels and is then fine-tuned at 512. When you upload a photo, a 512-pixel copy is used to predict the color. A guided filter then scales that color up to your scan's full size, following the edges in the full-resolution lightness, and the result is laid over the original. Color changes slowly across a picture while the detail lives in the lightness, so a 600 dpi scan comes back at 600 dpi.

My own graphics card is a GTX 750 with 1 GB of memory, which can't train this model even one image at a time. All the training ran on rented cloud GPUs; the main runs used an RTX 4090 at about a quarter of a second per training step.

The data: 1.4 million openly licensed photos

My first model learned from about 190,000 photos from COCO and a colorization challenge dataset, which come under a mix of Flickr licences. That's fine for a hobby but murky for a product. So I trained a second model from scratch on Megalith-10m, a collection of Flickr photos marked public domain or CC0. I used 160 of its 959 shards and kept about 94% of the images after filtering out gray, sepia and tinted ones. After a second clean-up (more on that below), training used 1,411,016 photos at 256 pixels and 239,933 at 512.

One honest caveat: like nearly every image model, the encoder started from ImageNet-22k weights, and ImageNet's terms are aimed at research. The colorizing itself was learned from Megalith. Photos people upload to Ungray are never used for training.

The main run was a 30,000-step warm-up without the discriminator, then 400,000 steps with the full loss. It took about 33 hours on one RTX 4090.

The sepia problem

Colorizers are usually scored on modern color photos that have been converted to gray, because then the right answer is known. On that test my model looked good. Then I tried it on real old photos.

I set aside 300 genuine black-and-white photos that were never used in training, and sorted each result into one of three groups: a single warm, brownish tint ("sepia"), left nearly gray, or properly colored. Midway through the main run, 24% came out sepia. That's one family photo in four.

Three things fed it:

• The training data. Some "color" photos are really old prints scanned in color, or modern photos with a vintage filter, and some Flickr accounts post mostly archive scans. They teach the model that an old-looking photo should come out brown.
• Averaging. When the model can't tell whether a coat was blue, green or red, a muddy brown is the safest average. Old photos are exactly where it is least sure.
• Fading. The most faded third of the test photos came out sepia or gray 22% of the time, against 7% for the least faded third. Blur made no difference.

I fixed it in two rounds:

1. Cleaner data. I removed 64,141 photos: warm single-tint images, plus every photo from accounts that mainly post black-and-white archives. I also raised simulated aging (fading, grain, blur, dust, JPEG damage) from 30% to 50% of training inputs. Sepia dropped to 8% shortly after and ended the main run at 11.7%.
2. An old-photo fine-tune (30,000 steps at 512 pixels, about 3 hours, $3.07):
   • Gray inputs made the way old film would have recorded them, including orthochromatic film.
   • Aging on 60% of inputs, now including low-resolution scans.
   • A "real black-and-white" branch. At every step the model colorizes some of about 4,700 genuine old photos (never the 300 test photos), and the discriminator learns to call those results fake. That pushes the model away from brown and gray answers.
   • A one-sided colorfulness reward, which only penalizes results that are duller than the real photo.

Result: 5.3% sepia, 1.3% left gray, 93.3% properly colored.

One thing that didn't work: automatically removing any single-color cast after the model runs. It erased the sepia completely, but it made colors on modern photos noticeably less accurate, and it sometimes swung a portrait teal. I dropped it.

How it compares with DDColor-T

I ran DDColor-T's official weights through exactly the same pipeline as my own models, on the same photos and at the same resolution. That matters, because published FID scores aren't comparable: DDColor's own score has been reported anywhere from 3.92 to 17.96, depending on who measured it.

Read it like this. The base model beat DDColor-T on FID, a score measured on modern photos. The old-photo fine-tune gave that lead back (4.22 against 4.21 is a tie) in exchange for much better results on real old photos, which are what people actually upload. DDColor-T's low sepia rate comes with one old photo in five left gray. The live model is also slightly more colorful than real photos (about 105%), so the app offers a "Subtle" style at 85% color strength next to Natural, Vivid and Warm.

What it cost

That leaves out one expensive mistake. An early warm-up run froze, and the rented machine sat idle for about 45 hours, which cost roughly $40.

• What broke: the training library's averaged copy of the model (its EMA) averaged the weights but not the buffers that sit next to them. Its outputs grew to around 10¹⁶, so the FID score became NaN, and the math routine that computes FID never returns on a NaN.
• What changed: every run now has a stall watchdog and a hard time limit.

Running the model is cheap. The GPU worker scales to zero when nobody is using it. A single photo that has to wake a GPU costs about half a cent, and a photo under steady traffic costs about $0.0003. In return there's a wait. A photo takes about 8 seconds when a GPU is already running, and a median of about 14 seconds when one has to start. Roughly one cold start in ten waits a minute or two for a free GPU.

Where it still gets things wrong

• Things that could be any color. Sky, skin, trees and water usually come out right. Clothes, cars and painted doors get plausible guesses, not facts.
• Small faces in group photos. Colors are picked on a 512-pixel copy of the whole photo, so in a big group each face is only about 35 pixels wide. At that size the model can't make out the lips and tends to paint mouths too pink. The site now corrects this after the model runs: a face detector finds each mouth and keeps the lips within a natural distance of that face's own cheek color, and a second check catches pink spots on skin where no face was detected. On the 300 old test photos, faces with a pink mouth went from 11 to 1. If one still slips through, crop the photo to a few faces before colorizing; on bigger crops the model gives normal lips.
• A golden lean. A few scenes still come out a little warm.
• Old film's blind spots. As described above, a red that the film recorded as near-black stays dark.

When you know a color, use Recolor. Tap the area or paint it with the brush, pick the right color, and only the color changes. Every fold and shadow stays exactly as photographed. If grandma's coat was blue, make it blue.

Try it

Ungray is in a free open beta. There's no account, you can colorize 10 photos a day at full resolution, and uploads can be up to 50 MB. Uploads are deleted automatically after 24 hours. There's more on the sepia fix in Why AI colorizers turn old photos sepia, and on generated "restorations" in Colorize, don't redraw.