Code quality and CI improvements

Finally did the long overdue main loop refactor. The main loop is now way shorter, more readable and more testable. I just split off some of the main loop’s work into it’s own modules, which made them way better looking and also, since they are now standalone modules, actually testable. A big win for repo structure.

In the process, I also replaced the input handling’s long if chain that made each character do it’s job with a switch statement. It’s more readable and theoretically faster, although since I compile it with -O3, it probably made no difference or an unnoticeable one at best.

I also finally fixed the super long CI times. Until now, CI used to take ~50 minutes (depending on how fast the GitHub runner was) because of cppcheck. Everything else finished in under 10 minutes, but cppcheck stalled it for an hour. It was stalling in mesh_assimp.cpp (the assimp wrapper) for most of it’s runtime. I always thought it was because assimp is a very large library. So I experimented with it a little. I included everything I include in the wrapper into a test file with a little code and ran cppcheck on that. It finished in less than 10 seconds. Eventually, I found out it was somehow taking that long because the main loader function was too long and handled too much by itself. So i just refactored it into 3 separate functions, without changing the behavior at all. Somehow, I still don’t get why, it reduced the whole cppcheck run to 9 minutes. a more than 5x improvement, just because I split a function into more functions. I’m not complaining, but it’s definitely weird. CI is now way faster and adding new things is gonna be way smoother now.

Miscellaneous fixes and improvements

• Fixed two races in platform code (one MacOS, one Windows)

• Removed build flags that disabled some compiler security hardening at the cost of a little performance

• Fixed the test framework to not freak out on NaN in near assertions

• Improved thread launching and handling in loaders and renderer

So overall just boring fixes and improvements, nothing special.

Refactored the platform layer

The whole platform layer was just one file until now. Since I did a lot of work on it recently, it grew to over 2400 lines of code. It was super confusing and disorienting. So I decided to refactor it (split it) into 8 files, based on what they handle:

• console.h - saving/restoring terminal state
• control.h - termination signal handling
• file.h - file operations (64bit)
• input_reader.h - handling input (not parsing, that is it’s own file)
• shared_memory.h - shared memory handling for the kitty protocol
• terminal_io.h - writes output to the terminal
• terminal_query.h - querying the terminal for things like graphics capability, etc.
• timing.h - frame timing and sleeps

Last pre-release checks and cleanup

I’m finally ready to release v0.1.0-alpha.2. Only notable thing I changed here is that I changed the default thread count: It now defaults to hardware concurrency everywhere. Other than that, I of course did final documentation checks and fixes and now I’m just waiting for CI to pass so I can tag it and let the release workflow do the last step. So this is hopefully the last devlog before my ship!

Did some polish and cleanup of the platform layer

Did final polish and cleanup of the platform layer since I want to actually finally cut a release relatively soon and I just did a lot of work there. Fixed things like useless tests, outdated or wrong comments and some platform specific quirks. Probably actually gonna cut a release soon (this time for real). Also added early rejections for BVH and CSM files, since they contain no model geometry (they can’t currently be displayed at all) and documented the change. The code is basically done (hopefully, I really don’t want to find another big gap that’s gonna take a week to close), I just want to correct anything wrong in the README, changelog, manpage or help menu and maybe make them look a little better before cutting a release and shipping.

Working on SIGSTOP and SIGCONT handling

SIGSTOP and SIGCONT are POSIX signals for stopping and resuming a running process. I’m working on adding handling for these, so you can safely pause and unpause rasterminal at any time. This is a lot of work, since until now, we just set up the terminal on program start and cleaned up when the program exited. There’s a lot of quirks and weird scenarios that I have to handle, for example if another signal gets sent while cleaning up the terminal for SIGSTOP. I’m trying to catch as many edge cases by just doing random things while using the program and also with the help of AI to come up with potentially uncaught scenarios.

Added handling for SIGINT, SIGTERM, SIGQUIT and SIGHUP

I don’t really have much to say here, I just added the full per-spec handling for all these. Only remaining POSIX-only gap is now SIGSTOP/SIGCONT (stop/resume process). I’m also starting school now, so I’ll be working on this less.

Fully fixed and merged the first POSIX cleanup item

Well, would you look at that! That took significantly less time than the Windows fix. It is partly because it was a smaller fix and only part whole POSIX fix, while the Windows one was the whole thing and also the foundation had already been laid out from the Windows work and I just integrated the POSIX path into that, but it is also very much because Microsoft is a horrible, terrible company that can’t make good software. I mean, look at GitHub, they have been going downhill ever since they got acquired by Microsoft.
Now that I got that out of my system, here’s what was fixed:

• If the program cannot enable raw mode at startup, it safely fails instead of continuing and corrupting the terminal state

• If the cleanup gets interrupted, it retries cleanup so it doesn’t leave the terminal state corrupted after one failed attempt

Onto the next POSIX fix: handling SIGQUIT and SIGHUP.

Merged the full Windows console restoration branch

My nightmare is over. I hopefully won’t have to deal with Windows for a while now. Summary of what is now fixed/handled on Windows:

• Ctrl+C, Ctrl+Break and Q exits fully recover the terminal’s state

• Ctrl+C and Ctrl+Break don’t leak mouse escape sequences into the next prompt if you terminate while actively dragging a model

• A bunch of other minor Windows or cmd.exe specific quirks

Only thing that isn’t handled (didn’t find a good way to handle it) is task manager “end process”. It kills the program without cleaning up and sometimes freezes the shell.

I HATE WINDOWS

Windows just doesn’t work. It doesn’t work. Nothing works on Windows. Everything breaks. I don’t get it.
THE UNIT TESTS PASS ON WINDOWS CI BUT WHEN I ACTUALLY RUN THE TESTS ON WINDOWS IT FAILS????? HOW IS THAT EVEN POSSIBLE.

The release is now definitely getting delayed by at least a week because I keep running into more and more and more and more and more and more and more and more and more issues. I hate Windows. Microslop is the only company that could mess up a basic terminal emulator.

And I also have to reboot my PC every time I fix something and push it and then reboot to get back to linux. this is torture.

At one point I tried to do it on a Windows VM on linux but that was too slow so I just gave up.

Found a bunch of terminal cleanup gaps

Most of them are Windows or cmd.exe specific (nothing ever works on Windows), but some are present on POSIX too. The Windows specific ones are:

• cmd.exe for some unknown reason doesn’t turn off the raw input sequences, so arrow keys don’t work like they should, even though I explicitly turn it off (????????) (I hate Windows, they always have to do something unexplainable for no reason just so your code doesn’t work as it should)

• when you terminate the program with ctrl+c or ctrl+break while actively dragging the model around, it sometimes leaks mouse escape sequences into the next prompt

• bunch of other weird small stuff that nobody cares about

And the few on POSIX are mostly ctrl+z suspend/resume behaviour not recovering the terminal and the same thing with ctrl+\ (SIGQUIT).

I’m still working on fixing these. Most of the windows ones are already fixed, but I haven’t even started with the linux/POSIX ones.

I have postponed the planned release to when I fix these + any other issues I find.

Cutting a release

The repo is in a pretty complete and polished state, so I’ll cut a release here. This will also probably be the last release before I ship, so I’m making sure that nothing is broken and everything works on every platform.
I’ve still got a lot of work to do though, even after the end of stardance, like animations, glTF PBR, format completeness for every format, GPU acceleration, etc.

Merged the full assimp integration into main

I really don’t have much to say here, I’ve already put all the info about the branch into the past devlogs. All the formats work and the wrapper is very sturdy. I’ve found more bugs in assimp itself than my code (devlog on that soon). So, rasterminal can now load basically any 3D model you throw at it. Some of the formats assimp supports are so niche, I’ve never even heard of them.

Fuzzing my assimp wrapper

I’m trying a bunch of hand-crafted models from all the formats that I can load and trying to find things that dont load or render properly. Also had AI write me a simple fuzzer in python and I’m just letting it run for a few hours. Also auditing the assimp loader paths against my wrapper to find any format specific behaviour that should be handled. I’ve already found a bunch of things and I’m still finding more.
Assimp has a lot of weird quirks and issues that I keep running into. Honestly kinda regretting the decision to integrate it, but since I’m already so deep in, I’ll just keep going.

Still working on the assimp intgration

Currently trying various models from all the different model formats that assimp loads. So far found a segfault, some weird behavior and integration mistakes that I made. Also found a small bug in assimp itself, for which I’ve submitted the PR with the fix to their GitHub repo. For anyone interested, here’s the PR: https://github.com/assimp/assimp/pull/6799

Working on assimp integration

Yop, I’m adding the Open Asset Import Library as a fallback for any model formats that aren’t natively supported with their own dedicated loader (only ones with dedicated loaders are glTF2, PLY, STL and OBJ). I’m still keeping the dedicated loaders, because they are more feature-complete, to-spec and wayyy faster (especially the glTF one).

I added it because I feel like for a 3D model viewer, support for many formats, even niche and obscure ones, is pretty important, even though nobody might use them.

This roughly doubled compile time and almost quadrupled binary size, which also made CI take way longer (of course). It somehow made the MacOS runners take 50 minutes to compile (from the previous 3 minutes), but I fixed that by bounding the number of processes CMake can spawn to the number of threads on the runner (previously was using -j which spawns as many as it wants and it was spawning about 230 of them).

I’m still testing how it handles various model formats and trying to find any reachable segfaults.

Cleanup and restructuring changes

After I just merged three big feature branches (kitty and sixel graphics protocols and the high-res performance branch), I needed to do some overdue cleanup and refactoring. And so I did:

• Collapsed the super long changelog entries into way more readable and legible points

• Restructured the src directory into multiple subdirectories because it had like 50 files already

• Sadly completely removed the Make build system, so now you can only build with CMake (CMake is very much evil and even more evil but it’s the only one of the two that supports windows and I was tired of managing two build systems)

• Removed the –graphics-scale flag, so now everything renders at native resolution

• And some small docs and comment cleanup

Finally merging the performance branch

It’s finally time to merge the performance branch. I basically redesigned the whole renderer and rasterizer around high resolution workloads, where it would still choose the old high-poly paths where they were faster.

Even snuck in some SSE2 SIMD code (picture), although I will probably be switching to a dispatch SIMD library like Google’s Highway. Not entirely sure it will be Highway or even if I will do that yet, that depends on the code complexity and before/after benchmarks.

Still optimizing…

Spent the first ~7 hours trying to find microoptimizations and each one was either a wash or worse somehow. The compiler works in mysterious ways.
At some point, I just gave up and let Claude have a go at the optimizing. Surprisingly it actually came up with a lot of interesting optimization techniques and solutions that I definitely couldn’t have come up with on my own. But it found so much, that the whole diff adds 5k lines of code. So now I’ll be cleaning up, testing, and maybe further optimizing that diff…