devlog: impatient engineering, skipping the wait, and running humaneval live

the project recap

quick context if you’re catching up: my initial submission for thalamus got rejected because the UI looked like a boilerplate wrapper. instead of a superficial skin-deep fix, i decided to overhaul the entire backend engine and train a custom, hyper-specialized 32B model optimized strictly for heavy coding logic—named thalamus-thor-1. conversational filler was stripped entirely from the datasets to keep the weights lean; my existing gemini agent team handles live web scraping to cover general knowledge blind spots.

the resource situation is a major scramble. puter.js was too limited for my 9-agent orchestration, so i abandoned it entirely. i’m burning through the absolute last of my AWS credits while waiting on approvals for google cloud and cheap amd gpu clusters. plus, school starts in a couple days, so i’m building a lightweight remote execution layer to trigger test suites and monitor terminal logs from the school computer lab during our free periods. earlier today, after a brutal 13-hour marathon fighting out-of-memory (OOM) errors and kaggle’s session killers on deepspeed ZeRO-3 initialization spikes, i finally pushed the parameters under the threshold and successfully compiled the weights on their tpu v5e-8 kernels.

here is the live update from 1.5 hours later.

the midnight deployment: saying screw it

the plan from my last post was to sleep on the weights and stand up the evaluation harness tomorrow morning. i lasted twenty minutes. when you spend 13 straight hours fighting initialization crashes just to get a model cooked, you don’t leave the weights sitting cold on a drive. the curiosity and momentum make it impossible to step away.

i went ahead, pulled down the evaluation scripts, configured the isolation wrappers, and slammed the model directly into the testing execution loop.

right now, thalamus-thor-1 is running a live trial by fire. it is chewing through openai’s standard HumanEval dataset in real-time. as of typing this, the model has already plowed past 90 out of the 164 total programming problems in the suite.

the screenshot attached is the raw stdout window of my evaluation terminal. you can see the script compiling generations, feeding function signatures, and executing validation unit tests back-to-back.

what the logs are proving

seeing the terminal scroll through 90+ consecutive problems without dropping a syntax exception or throwing infinite token loops is an immense relief. when you compress a 32B model down to tight FP8 and 4-bit configurations while scaling the context matrix toward an 800K target using RoPE and YaRN embeddings, the mathematical margin for error is razor-thin. one wrong scale factor and the model outputs broken brackets or repeats strings indefinitely.

instead, the model is executing functional correctness cleanly. it’s handling complex array manipulations, algorithmic logic, and nested structural filters completely raw. because the weights are completely unburdened by chat filler, it behaves less like a chatbot and more like a lightning-fast logic compiler—reading the docstring and immediately piping clean python code directly to the testing harness.

once it clears problem 164, the system will output the final pass@1 percentage. that single number will tell the definitive truth about the token economics i calculated earlier. if the pass rate holds up against these unit tests, we are looking at a model that can genuinely compete with sonnet 4.6 on pure engineering tasks, while operating at an estimated internal compute cost of just $2 per million input tokens and $6 per million output tokens—giving my agent pipeline high-tier logic execution at the price of haiku 4.5.

i’m staying up until the script hits 164. final data drops the second the testing loops finish executing.