9h 52m 8s logged

mesh importing and refraction

ima start off by saying that this isn’t true refraction, since that requires path tracing, which is really computationally intensive

instead, i went with screen space refraction offset approximation, which is a good, efficient refraction approximator that i still think looks pretty cool

material data for refraction

the value that determines the transparency of an object is called transmission, and the value for refraction is index of refraction (or IOR)

the current filesystem for materials that i have is .mtl files, because thats the standard filetype for exporting materials with objs from programs like blender. however, .mtl is a filesystem from the 1990s that was originally created for phong systems, meaning it doesnt really contain transmission values

so, i created a json sidecar system, that generates a better, more robust, human readable material file for every model. so now, when you import an obj/mtl pair:

it checks for a json sidecar file
if there is one already, just load from that and finish!
if there isn’t one already, now read the .mtl, extract as much info as you can from that, and write it to a new json file

multi pass rendering

in order to correctly render transparent meshes, you need at least 2 render passes (render the opaque ones first, then the transparent ones)

also, for the refraction to work, it needs an image of everything behind it, so in between the opaque and transparent pass, it quickly copies everything on the screen to an offscreen framebuffer

how the refraction works

1. get the view vector (v):

this one’s simple, just evaluate the view direction from the fragment position to the camera position

also, adjust the normal (n) if rendering backface

2. refraction vector (snell’s law):

the refraction vector is actually easy too, glsl has a built in refract intrinsic so you just drive that with a clamped ior value

3. deviation mapping:

to determine how much to bend the background scene texture, just calculate the deviation vector between the refracted ray and the view ray:

vec3 rayDeviation = refractDir - (-V);
vec2 distortion = (push.viewProjMatrix * vec4(rayDeviation, 0.0)).xy * 0.05 * transmission;

4. uv sampling + composite blending

offset the screen coordinates, but in order to prevent the edges from exploding, it needs to be clamped

also, sample the background buffer to blend lighting and make a fresnel rim

other stuff i did:

i also made a full custom persistent ui system (this took a while), and a command system, because hardcoding the ui to do stuff is bad

so instead of the ui code directly running logic, clicking a button just sends a command to the command system

this meant creating another render pass, meaning i’m up to 3 render passes per frame now

the only command so far is file.import_mesh, which is also hooked into the ui (you can see in the video below)