Devlog #8: Onboarding & Settings

My onboarding and setting screens are now functional, although I kind of want to redo the UI on my settings screen.

I’m still working on my UI rework for the editting page. I decided on a category dropdown format.

Almost ready to release v1.2!

Devlog #7: Timelapse Video

I wanted to be able to export a video of the long exposure actually building up, so I added an export that renders a short timelapse .mov of the composite building up, ending on the finished exposure. I attached an example of the buildup, I’m still playing around with speed and stuff, it looks kinda ugly rn. 

The Build-Up

I built an incremental acumulator on the blend engine that I’m already using for the other features. It holds the min, max, and sum textures, and each step basically folds one new frame forward into them.

Video frame 1 is the blend of just the first source frame, video frame 2 is the blend of the first two, and so on until the last video frame is the full exposure. The last frame stays for about a second before the end. Usually, it ends up being around 4 seconds long. 

Writing the File

For the video itself I used AVAssetWriter with H.264. Each output frame resolves the current composite to a CGImage, draws it into a bgra pixel buffer, and appends it. The finished .mov goes to a temp file, then to the share sheet and save to the phone’s photo library.

I now have a lot of different features and settings, and they are taking up most of the app. The long exposure itself looks really small right now, so I need to rework the ui.

Devlog #6: Motion Smoothing

Motion smoothing! Live photos are usually about 30 fps, so when something moves fast across the frame, a long exposure doesn’t get a smooth streak, and you can see the individual frames. I wanted to try to address this by smoothing the gaps.

Smooth Motion

I basically wanted to synthesize extra frames between pairs of real frames to fill in the gaps. I had to figure out the pixel’s movement from frame to frame, so I computed dense optical flow with Vision’s VNGenerateOpticalFlowRequest. It returns a vector field that tells me how each pixel traveled (direction & distance). I run this once per import at a low res.

Once I have the flow field, I do a warp operation to generate a frame between A and B. I push frame A forward halfway towards B, and I push frame B backwards halfway towards A. Then I cross-fade the two results. This is repeated between each pair

All of this runs inside a metal compute shader. The kernel warps each synthesized frame and accumulates it into the running min, max, and sum buffers the blend already uses from the exposure stuff. This basically means that motion smoothing works with the other adjustments for free. Frame alignment and exposure normalization both are applied in the same way

I’m almost done with the app, I just have to keep tuning my values for exposure and motion smoothing then im going to release version 1.1.

Devlog #5: Exposure Modifications

I finally finished my exposure modifier. The big change from before is that i changed it from three buttons (average, lighten, darken) to a slider, so now the user can pick the right amount for each photo. 

Exposure Modifier

The slider has lighten and darken on each end. The lighten side keeps the brightest pixel across all your frames at each point in the image. This is supposed to improve the appearance of light trails in dark long exposures. Darken does the opposite. It keeps the darkest pixel, which is kinda niche and really only works to a small scale.This was built with a metal compute shader to track a running min, max and average across all the frames at once in linear light. The slider just picks how much to mix in the min or max, so moving it doesn’t require re-blending anything.

To do: i still want to add some sort of motion blur to smooth in the diff between frames a little bit

Devlog #4: README

I put together my readme! I used diagrams.net to make the how it works diagram, then everything else is pretty standard

Still working on exposure stuff, almost done

Devlog #3: Landing Page

Today, I built my landing page for the app. It was basically meant to show the different features of the app + how the app worked.

Link: https://long-exposures.jameshou28.com

IMPORTANT NOTE:

this is what I’m attaching as my demo, but it’s just the landing page not the actual app - I don’t have an apple developer account so the only way to download the app will be through a .ipa later

Devlog #2: Frame Alignment

The hard part of long exposures is that you need to hold your camera steady for the entire duration (usually with a tripod). If your camera is not steady, the picture becomes blurry.

frame alignment

To address this, I used apple’s VNTranslationalImageRegistrationRequest. It basically works by taking pixel buffers and returning a 3x3 matrix that describes how the frames map each other. I took the translation parts from this and used it to move the frames for the alignment part

Everything is aligned to the center frame, so basically every time you move the sliders, it has recalculates it

I’m still working on the exposure modifying, will be done soon

Devlog #1: Long Exposures

A few weeks ago when i was on vacation, I was standing on a bridge at 1 am playing around with some long exposures on my dslr, and I realized that iphones don’t have real long exposures. Without a third party app, the closest that iPhones can get to long exposures is through converting live photos, which is very limited in what it can do.

live photos

Basically, live photos consist of the cover photo and a ~3 second video. I used photokit and avfoundation to extract raw pixel buffers from the video. I then used a metal compute shader to average the pixel values across all of the frames of the video together to make the long exposure. At this point i had basically reconstructed the built-in long exposure from apple.

long exposure editting

The big thing that I wanted to do was take these long exposures and edit the frames that went into them (what you can’t do on apple). This lets me make long exposures with fewer frames and lets me go around apple’s built-in long exposure limitations. To create this feature, I added a range slider to select what frames go into the long exposure (letting me choose the start and end frame)

library

I implemented a simple library feature that basically lets you save your long exposures into the app (and also into your photo library on your phone)

I also started a frame alignment feature and an exposure modifier, but i’m still working on those so i’ll save it for another devlog

I spent at least 10 more hours on this before i realized that i didn’t have hackatime connected to my xcode :(