My last devlog was about the UI. This one is about how the multi-scene part actually works under the hood, since this was one of the more difficult parts to implement in my project.

The problem

I wanted one video to be able to have multiple scenes (multiple code snippets and narrations) instead of just one. The hard part about this is that each scene’s typing animation has to stay perfectly synced with its own narration audio.

How I solved it

Instead of trying to build one giant video and line up all the audio across scenes, I make each scene into its own small video first. Each scene gets its frames timed to its own narration, then gets turned into a scene_0.mp4, scene_1.mp4, and so on.

Then at the very end, all those mini-videos get stitched together into the final video using ffmpeg’s concat feature:

with open("scenes.txt", "w") as f:
    for i in range(len(scenes)):
        f.write(f"file 'scene_{i}.mp4'\n")

subprocess.run([
    "ffmpeg", "-y", "-f", "concat", "-safe", "0",
    "-i", "scenes.txt", "-c", "copy", "output_path"
])

ffmpeg’s concat needs a text file listing every video to join, so I made a scenes.txt file with each line being one scene and then I told ffmpeg to read it.

A bug I hit

At first all the scenes were saving their frames into the same folder, so scene 2 was overwriting scene 1’s frames and the final video only showed the last scene. I fixed it by giving each scene its own frame folder (frames_0, frames_1, etc) so they don’t clash.

Why this approach good

Because each scene is already a finished, synced video on its own, stitching them just plays them back to back. There’s no complicated math needed to merge multiple audio clips with one video, instead it is just merging videos which I can do really easily.

Cleanup

Since this makes a bunch of temporary files (mini-videos, frame folders, voice files), I made the program delete them all automatically after the final video is built, so you’re just left with the one video you actually want.

Stay tuned for more!