shadows

this one is really cool, i feel like it really takes it from looking like its from 1995 to like… 2015 maybe? it doesnt look that good but i think its a massive improvement!

right now, its a 4096x4096 shadowmap with an extent of 24 units, a random noise filter to reduce pixelation, and 3x3 pcf to make it look somewhat smooth

it is currently orthographic light projection only, but I think i’ll work on getting perspective lighting soon :)

shadows did reqire implementing a whole new imageview, descriptor set, render pass, all that, but i’m getting used to it atp lol

serialization is also completely broken right now, so I do need to fix that…

texture loading!!!

absolute yipee moment

 
this took like a lot of thinking to figure out how to implement for reasons that i think are obvious, but some of them are:

• how tf do you load an image
• what does the loaded image data mean
• how do you map the texture to the model’s uv coords
• how does normal mapping work
• how does the shader know where on the image its reading
• and a lot more that i prolly forgot lol

this whole thing basically meant a whole shader overhaul again (even though i alr did that last devlog but its whatever loll)

 
you can now map albedo (color), normal (height/offset), orm (occlusion, roughness, metallic), and emmision (glowiness)

 
emission only kind of works rn lol, but i will get to it soon

 
also, you currently have to manually set filepaths to the textures you want in the .json sidecar for that model, i need to add filepicker stuff to the inspector but i will get to it ok

 
the textures in the video below were manually baked, packed, etc at 4096*4096 in blender

 
i think another feature i should work on soon is procedural material generating, because texture mapping will always have a resolution limit (even 4k textures look kinda bad when mapped to larger models)

 
i also want to change how reflections work a little bit, or maybe add anti-aliasing because right now the reflections look deep fried (as you will see in the video below)

shader overhaul

so i noticed that the pbr shader lowkey looked kinda ugly and my microfacet solver was buns af so i js spent a while fixing it up and basically ended up going down a rabbit hole of tonemapping and completely rewrote the whole thing

 
i switched from some random generic tonemapping to khronos standard neutral pbr mapping, which i think looks wayyyyyy better, the colors actually feel accurate now

 
i also spent a little while setting up the shader to support texture colors when i get that all working

 
it is wayyy better now, especially metallic things

i also started setting up for texture imports but right now its only albedo and it does NOT work in the slightest lmao

 
i think i will hopefully have texture importing and mapping done by next devlog, but uhhhhh…. no promises

physics

aaaaaaaaaaaaaaaaa

 
it works concerningly well and it actually makes me really happy to see it working

 
i legit jumped for joy when i first saw stuff dropping due to gravity

the actual underlying physics engine is jolt physics, but all the init and handling and all that is custom from me

 
its all linked to the inspector and allat, so you can tweak values real time, such as body type, bounciness, friction, and mass

 
its now a physics playground instead of just a glorified obj viewer

 
scaling is very broken as far as colliders go, ill probably fix that soon

 
you can start and stop the physics sim with f5

working scene serialization (saving)

if you hit ctrl + s (or hit the gui button) then it creates a .slate file, containing all the data of every object in the scene

 
this .slate file is human readable (and editable), written in a format very similar to toml

 
heres an example .slate file:

[scene]
name = "SlateScene"
version = "1.0"

[[entity]]
index = 0
name = ""
mesh_path = "models/test_obj.obj"
position = [-0.310723, 0.927019, 1.47743]
rotation = [0, -0, 0]
scale = [1.9916, 2.28848, 3.38984]
[entity.material]
albedo = [0.183333, 1, 0, 1]
roughness = 0.5
metallic = 1
ior = 1.5
transmission = 0

[[entity]]
index = 1
name = ""
mesh_path = "models/emerald_monkey.obj"
position = [1.29034, 0, -5.64025e-08]
rotation = [8.61461, 6.18611, 2.71029]
scale = [3.07852, 3.38535, 3.86504]
[entity.material]
albedo = [0.050432, 0.800023, 0.302281, 1]
roughness = 0
metallic = 0
ior = 2.418
transmission = 1

next, ill probably start working on physics

scene serialization

okay so this is akward but it doesnt actually work yet, but in my defense serialization is really confusing to me since i generally don’t do a whole lot with file writing and loading, especially in a custom formatted and extensioned .slate file

 
it basically just takes a bunch of data by looping through scene objects and records them, then it also has to do that backwards, then i also have to implement a command and gui system to trigger the logic

 
and i also want the .slate to be easily readable, so i need to format the text in a way that makes sense (i’m taking inspiration from TOML)

 
hopefully its done next devlog, but no promises lol

color picking!

i noticed that setting color in the inspector with 3 rgb values was really annoying, so i made a whole new ui class (ui_color_picker.hpp/cpp), that can hook into any other ui

 
it supports:
 

• S/B box + hue bar
• RGB input
• CMYK input (mildly broken)
• hex input (very broken)

 
it actually works pretty well, there is some bug where an object can never be fully white but i think thats a command issue not color picker or inspector

 
next, it might be time to work on physics

bugfixes and writing docs

i spent a good 4 hours just going through some code, finding anything that might be wrong and fixing them. surprisingly, i didnt find that much, which means that i’m better at coding than i thought
 

i found a few places where i just forgot to add something to a cleanup function, which could have led to a small memory leak over time, but nothing major

 
some bugs i fixed:

• the background of any dropdown menu was rendering behind the viewport
• selecting stuff in the hierarchy wouldnt sync with the inspector properly
• i forgot to add screenPointToRay to my cleanup function
• some other things that i forgot
   

i also added a few std::couts in some spots that needed them

i also decided to start writing some docs because my brain was fried
 

i basically just dug through my ~3000 lines of vulkan init and rendering code, and took notes throughout and tried explaining my confusing mental processes in a .md file
 

this kinda made me realize that i do not have enough annotation in my code but thats a problem for future me :)

hierarchy panel and scrolling

the hierarchy panel itself was actually not that hard, the hard part was coming up with a reliable system that can detect and list every object currently loaded in the scene without turning your pc into an explosive
 
the other hard part was modifying the dockspace layout to be able to handle multiple different panels, because previously, it was just one window per dockspace panel, but now it can be as many as you want
 
also, the scrolling system is new, it is its own class ui_scrollable_container, and it is basically doing what the dockspace used to do, but it also lets you scroll. so now, the dockspace just loads a scrollable container, and the scrollable container does all the arrangement and sizing logic.

the hierarchy works exactly as you would expect it to, it lists current scene objects and then you can select them from it
 
eventually, ill prolly add support for parenting, which should be pretty easy

its kinda crazy to see my custom persistent ui system slowly turn into something useful as i add more features to it
 
so far ive just been adding features when i need them, i havent just thought of features i might need and added all of them

inspector updates and dropdowns

the inspector now has IOR and transmission values that you can change, ui updates, and dropdowns
 
this took literally like 45 epochs because i was running into a materialgpu indexing issue where the inspector boxes would update the wrong values and my peanut brain didnt even think to check if i set the indices correctly
 
dropdowns also took a good 3-4 hours, but it wasnt too bad, mostly just figuring out the correct logic
 
in total, the dropdown class (ui_dropdown.cpp/hpp) is at 169 lines, and the inspector class (ui_inspector_panel.cpp/hpp) is at 470 lines

next, i think ill either work on ui rounding or a file hierarchy viewer

ui inspector

the inspector is by no means done, but its a start!
 
right now its just supposed to be purely functional, but i plan on making it look better in the future
 
the input boxes are their own ui class, meaning i can have those input boxes everywhere
 
making the ui system from scratch is very fun, but it at times i do regret not using qml or im gui, but overall, i dont regret making it from scratch too much
 
the main issues with it right now is that the sensitivity values in the input boxes feel wrong, i think the position needs less sensitivity and the rotation needs more
 
also, the color ones are a bit broken, theres a little bit of discrepancy between what the ui shows and what the actual .json material sidecar has written, so i’ll have to figure out how to fix that

i think at some point, i want to add support for dropdowns, so each row can be expanded into larger input boxes if needed, or so the entire transform or material section can be collapsed
 
i also need to add input boxes for IOR and transmission, i jus forgot to do that lol, should be pretty easy tho

rotation gizmo, multi-object support

you can now switch between the translation gizmo, and the rotation gizmo, and click between objects to change what one has the gizmo active
 
the gizmo also stays locked to the selected meshes’ local axes, i might add switching between local and global axis orientation at some point, but for right now local only is good enough
 
all of the gizmo mesh is generated from code, not as an obj, making it very efficient and fast and robust, but making the logic that generates vertices in the right position was very annoying lol

next, ill probably work on an inspector panel where you can edit an object’s location, rotation, scale, material data, etc
 
or maybe saving the scene to file

but now you can move and rotate any amount of objects to your heart’s content

translation gizmo

this was like
 
miserable to make lol
 
it took me sooooooo long to get translation working, basically i had this samurai battle with a scaling issue for the longest time, i accidentally told my code that the viewport was smaller than it actually was and it just took forever to find that out
 
this issue caused a really annoying issue where the actual location of the rendered gizmo was different than the clickable area
 
there was also this other issue where the gizmo would move twice the speed of the mesh it was translating, but it was just a flaw with the mesh class, the gizmo was basically constantly recieving its own center as well as the meshes’ center which made it translate double

but it works now, i think it works pretty well
 
its all based off of the modular command system, so anything can call translatemeshcommand and it translate meshes
 
next is prolly working on rotate and scale and stuff, all the other gizmos that we need
 
the gizmos also meant a new .vert and .frag pair, and a whole new vulkan render pipeline lol

grid generation

this took forever, and was much harder than i expected
 
basically, i needed to create a whole new rendering pipeline and a whole new glsl shader pair, as well as coming up with logic so the triangle-focused renderer can actually render some super thin infinite lines
 
the grid does generate dynamically around the camera view, so it is endless
 
i think it does look pretty good tho

next, ill prolly work on adding some move and rotation and maybe even scale tools

mesh importing and refraction

ima start off by saying that this isn’t true refraction, since that requires path tracing, which is really computationally intensive
 

instead, i went with screen space refraction offset approximation, which is a good, efficient refraction approximator that i still think looks pretty cool

material data for refraction

 
the value that determines the transparency of an object is called transmission, and the value for refraction is index of refraction (or IOR)
 

the current filesystem for materials that i have is .mtl files, because thats the standard filetype for exporting materials with objs from programs like blender. however, .mtl is a filesystem from the 1990s that was originally created for phong systems, meaning it doesnt really contain transmission values

 
so, i created a json sidecar system, that generates a better, more robust, human readable material file for every model. so now, when you import an obj/mtl pair:
 

• it checks for a json sidecar file
• if there is one already, just load from that and finish!
• if there isn’t one already, now read the .mtl, extract as much info as you can from that, and write it to a new json file

multi pass rendering

in order to correctly render transparent meshes, you need at least 2 render passes (render the opaque ones first, then the transparent ones)
 
also, for the refraction to work, it needs an image of everything behind it, so in between the opaque and transparent pass, it quickly copies everything on the screen to an offscreen framebuffer

how the refraction works

1. get the view vector (v):

this one’s simple, just evaluate the view direction from the fragment position to the camera position
 
also, adjust the normal (n) if rendering backface
 

2. refraction vector (snell’s law):

the refraction vector is actually easy too, glsl has a built in refract intrinsic so you just drive that with a clamped ior value
 

3. deviation mapping:

to determine how much to bend the background scene texture, just calculate the deviation vector between the refracted ray and the view ray:
 

vec3 rayDeviation = refractDir - (-V);
vec2 distortion = (push.viewProjMatrix * vec4(rayDeviation, 0.0)).xy * 0.05 * transmission;

4. uv sampling + composite blending

offset the screen coordinates, but in order to prevent the edges from exploding, it needs to be clamped
 
also, sample the background buffer to blend lighting and make a fresnel rim

other stuff i did:

 
i also made a full custom persistent ui system (this took a while), and a command system, because hardcoding the ui to do stuff is bad
 
so instead of the ui code directly running logic, clicking a button just sends a command to the command system
 
this meant creating another render pass, meaning i’m up to 3 render passes per frame now
 

the only command so far is file.import_mesh, which is also hooked into the ui (you can see in the video below)

sketching

all of this was just spent making some concept art (tbh i kinda just wanted an excuse to do some art and log it lol)

my art process is:

• blender concept model with matcap
• sketch in krita
• shade the sketch
• coloring
• shade the coloring

i have not started doing any actual game stuff or coding, because slate engine isnt far enough in development yet lol

pbr materials

pbr math

reflectance:

this is the main equation that a cook-torrance pbr system, all the other equations lead back to this one at some point

𝐿𝑜⁡(𝑝,𝜔𝑜) =𝐿𝑒⁡(𝑝,𝜔𝑜) +∫Ω𝑓𝑟⁡(𝑝,𝜔𝑖,𝜔𝑜)⁢𝐿𝑖⁡(𝑝,𝜔𝑖)⁢(𝐧⋅𝜔𝑖)⁢𝑑⁢𝜔𝑖

this equation essentially just determines what light does after hitting a surface with specific parameters

we approximate the integral with two different light interactions:

1. diffuse (⁢𝑘𝐷): any light that penetrates the surface, scatters internally, and re-emerges
2. specular (𝑘𝑆⁢): light that reflects directly off the micro-surface

in the code itself, the total outgoing radiance ($L_o$) is calculated with:

vec3 Lo = (kD * albedo / PI + specular) * radiance * NdotL;

cook-torrance brdf:

the specular component is calculated using the cook-torrance microfacet brdf:
 
⁢## 𝑓cook-torrance =𝐷⋅𝐺⋅𝐹 / 4⁢(𝐍⋅𝐕)⁢(𝐍⋅𝐋)⁢
 
where 𝐷⁢, 𝐺, and ⁢𝐹 represent:

1. normal distribution (𝐷):

   the NDF statistical distribution calculates how many microfacets are aligned with the halfway vector ($⁢𝐇=𝐕+𝐋 / abs(𝐕+𝐋⁢))

𝐷GGX⁡(𝐍,𝐇,𝛼) = 𝛼^2 / 𝜋⁢((𝐍⋅𝐇)2⁢(𝛼2−1)+1)2⁢

```glsl
float DistributionGGX(vec3 N, vec3 H, float roughness) {
    float a = roughness * roughness;
    float a2 = a * a;
    float NdotH = max(dot(N, H), 0.0);
    float NdotH2 = NdotH * NdotH;

    float num = a2;
    float denom = (NdotH2 * (a2 - 1.0) + 1.0);
    denom = PI * denom * denom;

    return num / max(denom, 0.000001);
}
```

1. geometry (𝐺): smiths schlick-ggx

   microfacets can cast shadows and obscure light from the camera too, so the geometry function calculates the self shading factor using smiths method:
       

𝐺⁡(𝐍,𝐕,𝐋,𝑘) =𝐺1⁡(𝐍,𝐕,𝑘) ⋅𝐺1⁡(𝐍,𝐋,𝑘)

     
where 𝑘⁢ maps roughness for direct lighting using ⁢𝑘 = (roughness+1)^2 / 8:

```glsl
float GeometrySchlickGGX(float NdotV, float roughness) {
    float r = (roughness + 1.0);
    float k = (r * r) / 8.0;

    float num = NdotV;
    float denom = NdotV * (1.0 - k) + k;

    return num / denom;
}

float GeometrySmith(vec3 N, vec3 V, vec3 L, float roughness) {
    float NdotV = max(dot(N, V), 0.0);
    float NdotL = max(dot(N, L), 0.0);
    float ggx2 = GeometrySchlickGGX(NdotV, roughness);
    float ggx1 = GeometrySchlickGGX(NdotL, roughness);

    return ggx1 * ggx2;
}
```

1. fresnel equation (𝐹⁡): fresnel-schlick approximation

   the fresnel factor calculates the percentage of light reflected versus absorbed based on the viewing angle (⁢𝜃):
      

𝐹⁡(𝜃,𝐹0) =𝐹0 +(1−𝐹0)⁢(1−cos⁡𝜃)^5

     
nonmetals reflect around 4% of light (𝐹0 =0.04), while metals (conductors) use their tinted base albedo color as 𝐹0:

```glsl
vec3 fresnelSchlick(float cosTheta, vec3 F0) {
    return F0 + (1.0 - F0) * pow(clamp(1.0 - cosTheta, 0.0, 1.0), 5.0);
}

// lower, inside main():
vec3 F0 = mix(vec3(0.04), albedo, metallic);
vec3 F  = fresnelSchlick(max(dot(H, V), 0.0), F0);
```

other stuff:

a surface cant reflect more light than it recieves, so a simple equation is used to make sure this doesnt happen:
       
⁢## 𝐤𝐃 =(1−𝐤𝐒) ⋅(1−metallic)
     
right now, all the material data is pulled from a .mtl file

i also added bindless ssbo storage so it doesnt explode when you have a lot of models

next, i might keep working on the custom persistent ui system

custom obj file loading

all of the obj loading is done using tinyobjloader, which is a popular header file that im using to get the raw vertex coords of any .obj file

however, all tinyobj does is pass the raw vertex data to my code (technically it can do some basic triangulation, but thats not its main purpose so its not perfect)

this means that any model with n-gons (complex polygons) will not work and will look super weird

technically you can tell blender to explicitly triangulate a file when you export it, but i think that it should be able to take any obj that you can throw at it without needing external setup

so i also added a popular header file called earcut, which at a basic level, just cuts any complex polygons into 3-vertex triangles (which my index pass and renderer accepts).

this now means that it can load any custom obj file

i want to add auto color loading (from the obj data) or texture loading next, or maybe work on the ui or add ambient occlusion

entered the 3rd dimension

wasd movement

made a whole separate Camera class (look at me keepin things organized )

• this means that now the vulkan_renderer class has model, projection, view, and transform matrices:
  • the model matrix tracks the transformations
  • the view matrix essentially tracks where the camera is
  • the projection matrix sets a perspective frustum in order to make things actually have perspective and not just be flat
• the lerping is also based on deltaTime, which means smooth movement no matter ur fps
• the mouse tracking is locked via SDL_SetWindowRelativeMouseMode, and can be unlocked with the escape key

vertices and index buffers

• right now, the vertices are being set manually in my code… however, importing obj (and maybe gltf and stl) files is next on the feature list, i’m planning on using tinyobj
• all the geometry is indexed, which basically means that instead of wasting a ton of memory repeating data for 36 raw vertices, you can just define the 8 corner vertices, then use an index buffer pass to tell the gpu how to connect those 8 vertices into 12 triangles

window resizing

• this was easy, basically just had to tell it to make new image views and framebuffers based on the current aspect ratio (w/h) when the size of the window changes

so now, its a spinning cube in a resizeable window that you can move around with wasd

the window opens (!!)

so, why did it take me 6 hours to get a window to open?
 
it kinda just comes down to the fact that vulkan is vulkan
 
on other apis such as opengl, the graphics driver just does stuff for you. you can say sumn like “open a window and make it pink” and it just does the memory, thread syncing, and goofy platform stuff for you
 
vulkan on the other hand, wants none of that. if you dont specifically ask for something, it wont happen. this does make it significantly harder (and much easier to create a gpu-ending vram overflow) to code with, but it does have the benifit of being much more powerful, having no overhead, and 100% control. its not gonna do anything that you dont tell it to

so what do you have to do to open a beautiful grey window with vulkan?

• os bridge (sdl3): instead of writing a whole abstraction for wayland/x11, win32 and metal, you just have to tell sdl3 that you’re making a vulkan window, and it does some magic stuff

• vulkan instance (VkInstance): this is how you get into the vulkan sdk. you kinda just have to tell vulkan everything it needs to know and ask sdl what extensions you need

• window surface (VkSurfaceKHR): vulkan is kinda dumb and doesnt know what a window actually is, so this is where sdl tells vulkan where it can actually write pixels

• pick a physical device: scan the current hardware, loop through every available device, and then just pick the best one (eventually ill probably add a way to select a custom one)

• logical device (VkDevice): the physical device is the actual doohickey in your computer that runs stuff, the logical device is the software’s interface to it.

• swapchain (VkSwapchainKHR): vulkan can’t actually write images directly to the screen (and that would look bad anyway), so you make a swapchain. my code does double buffering (basically vsync), which means that the screen displays image a, while the gpu is rendering image b.

• render pass and framebuffers: basically just telling vulkan what to do (in this case, clear the screen, then draw the color). the framebuffer just binds the swapchain image views to the render pass

• synchronization and command subsystem: believe it or not, your gpu’s clock speed is usually around (or more than) half of your cpu’s clock speed, so you have to make sure that your code waits for the gpu is ready before running the next c++ line. if you don’t, your gpu ram and cache will overflow.

 
overall, this took a total of 764 lines of code just to get a grey window
 
the nice part is that the progress that i’ll make is exponential, since the very beginning stuff takes the longest

i also made a logo (the current project page banner)