Making Freedom Fly
WWII veterans, speaking in the voices they had during the war
Context
A series of digital exhibits for the Canadian Warplane Heritage Museum in Hamilton, Ontario: an immersive dome, life-size MetaHuman portraits of the veterans that visitors can talk to, and a curated cinema. I was freelance, working with Parallel World Labs, and had my hands in most of it: the voice and face R&D, the technical setup, the graphic design, the lighting and environment design, and the UX of each exhibit. Then I travelled out to the museum for the physical install.
The challenge
The museum had recordings of the veterans as they are now, elderly. The stories were about who they were during the war. The portraits needed to speak in the voices those people had then and still sound unmistakably like themselves. The off-the-shelf options failed: StarGAN-VC came out robotic, ElevenLabs could clone a voice but not re-age it, and Tacotron-style training wanted hundreds of hours of clean audio. I had about 60 minutes per person, and each line had to generate in under two seconds.
How it was built
I split each voice into separate sound envelopes (pitch, timbre, formants, phonetic character) and tuned the modulation on each one on its own, testing weight combinations until the thin, gravelly artifacts of ordinary pitch-shifting were gone. Then I trained on a blend of each veteran's original audio and the re-aged version, adjusting the ratio until the voice sounded younger but was still theirs. The faces started from as little as a single blurry black-and-white photo from 1962. I upscaled, landmarked and relit them, built base meshes with a custom Blender plugin, and finished them in Unreal.
What surprised meThe fix wasn't a bigger model. It was the mix: putting each veteran's real voice back into the training data is what kept them sounding like themselves.
Outcome
Three exhibits installed at the museum: the dome, the portrait portals and the cinema. Each voice was trained on about an hour of source audio, where standard approaches need hundreds of hours, and generates in under two seconds.
Why it belongs here
It's the project that uses everything I do in one place: R&D on a problem with no off-the-shelf answer, art direction, environment and UX design, and being in the building for the install.