devlog: thalamus-thor-1 is trained. recap, kaggle breakthrough, and humaneval

context recap

if you haven’t been tracking: my first ship attempt for thalamus got rejected because the UI looked like a generic boilerplate wrapper. instead of a superficial patch, i decided to completely overhaul the core engine and train a custom 32B model optimized strictly for raw code logic—named thalamus-thor-1. general knowledge was stripped out during data prep to keep weights lean; my existing gemini-led researcher team handles live web scraping instead.

infrastructure-wise, it’s a massive scramble. puter.js proved way too limited for my 9-agent pipeline orchestration, so i’m entirely stranded there, burning through the absolute last of my aws credits and waiting on gcp/amd approvals. also, school starts in two days, so i’m building a lightweight remote pipeline to trigger test suites and monitor logs straight from school lab machines during free periods.

---

the kaggle breakthrough

yesterday, i was stuck in deepspeed ZeRO-3 initialization hell. every time the notebook tried to load the base parameters on kaggle’s free tpu v5e-8 clusters, a massive compute spike would trigger their aggressive backend session killers, immediately dropping the kernel.

i spent hours adjusting chunk prefills, gradient accumulation steps, and stripping background overhead until the initialization sequence squeezed just under the termination threshold. the run finally executed, completed its QLoRA fine-tuning passes at FP8, scaled the context toward an 800K target using RoPE and YaRN with 4-bit compression, and successfully compiled the weights.

the screenshot attached to this update is the actual kaggle notebook page of my AI training notebook showing the setup.

initial sanity checks

compiling weights is only half the battle. when you aggressively compress a 32B model down to FP8/4-bit formats while applying long-context scaling embeddings, the positional math can warp. if it breaks, the model turns into a hallucination engine.

i spun up a local instance to run direct logic checks, and the structural syntax held together perfectly. it didn’t choke on deeply nested loops, accurately referenced parameters defined deep inside large codebase contexts, and skipped conversational fluff entirely to jump straight into code execution.

tomorrow’s trial: the humaneval protocol

vibe checks aren’t enough. tomorrow morning, i am setting up an automated evaluation harness to run thalamus-thor-1 through the full OpenAI HumanEval dataset (164 coding problems) to calculate the raw pass@1 rate.

this will prove the token economics math i did earlier: whether this custom setup can genuinely compete with sonnet 4.6 and opus 4.6 on pure engineering tasks while running at an estimated cost of $2/1M input and $6/1M output tokens (top-tier intelligence at the price of haiku 4.5).

hosting is still the looming issue. permanently serving this with an extended context window requires a dedicated 80gb VRAM prompt server configuration, which i’m trying to source while my teammate builds out the social media presence to push links.

on a side note, fighting package conflicts during this nightmare inspired a lightweight side-utility: a clean CLI tool for coders that natively resolves dependency hell. the core engine is working perfectly, and you can track it here: https://stardance.hackclub.com/projects/6323

evaluation metrics drop tomorrow morning. let’s see what the numbers show.