Devlog #6: Motion Smoothing

Motion smoothing! Live photos are usually about 30 fps, so when something moves fast across the frame, a long exposure doesn’t get a smooth streak, and you can see the individual frames. I wanted to try to address this by smoothing the gaps.

Smooth Motion

I basically wanted to synthesize extra frames between pairs of real frames to fill in the gaps. I had to figure out the pixel’s movement from frame to frame, so I computed dense optical flow with Vision’s VNGenerateOpticalFlowRequest. It returns a vector field that tells me how each pixel traveled (direction & distance). I run this once per import at a low res.

Once I have the flow field, I do a warp operation to generate a frame between A and B. I push frame A forward halfway towards B, and I push frame B backwards halfway towards A. Then I cross-fade the two results. This is repeated between each pair

All of this runs inside a metal compute shader. The kernel warps each synthesized frame and accumulates it into the running min, max, and sum buffers the blend already uses from the exposure stuff. This basically means that motion smoothing works with the other adjustments for free. Frame alignment and exposure normalization both are applied in the same way

---

I’m almost done with the app, I just have to keep tuning my values for exposure and motion smoothing then im going to release version 1.1.