Engine improvements

ughhhhhhh i spent hours making “spikes” (is that what they’re called?) and it wasn’t counted cause slint live preview/editor = not hackatime-d sighhhh

Oh well btw, hi, sorry for the random 1 month ish gap

I come back, better at Slint, and with fresh motivation.

okay so august 30 was my last commit and it is now october.

in my defense i did not touch this once in that whole time which is either very bad or very healthy idk. but something clicked today and i opened it back up and immediately went “ew who wrote this” (me. it was me.)

cough cough better engine

so the editor had this whole situation in doc.rs where Document::new would parse the markdown into an ast, and then i had this convert_blocks function that walked through the blocks AND a second function called find_block_range that literally scanned the source bytes by hand looking for double newlines to guess where each block started and ended. like manually. byte by byte. offset += whatever. i genuinely cannot believe past me looked at that and went yeah ship it

the thing is the parser already KNOWS where every block is. it has the offsets. it was just throwing them away and then i was reconstructing them with a worse, slower, buggier version downstream. so i added parse_blocks_offsets to the markdown parser so it just hands back (block, range) and now doc.rs is like. map over them, pull the range straight off, extract spans, collect. that’s it.

find_block_range is gone. convert_blocks is gone. good riddance :3

also killed the edita-core dependency because it turns out i wasn’t actually using it for anything, it was just sitting in the Cargo.toml being decorative

net result is like 1290 lines deleted vs 793 added which is my favorite kind of commit. deleting code that you wrote and were briefly proud of is so good for the soul hahaaaaa

anyway the engine is better now (cough cough not)

Component library + live typing preview

two things this commit

first

I pulled all the Slint components out of penumbra-ui into their own crate, penumbra-component.

it’s a pure-markup library, no Rust logic, just .slint files: the Theme, all the block components (ProseBlock, HeadingBlock, QuoteBlock, CodeBlock, TableBlock, FootnoteBlock, HtmlBlock), the CanvasSurface, the TopBar, and PillButton (which used to be NewNotePill but I generalized it so it takes a text prop instead of hardcoding “New note”). there’s an index.slint that re-exports everything and now the app imports from @penumbra-component/index.slint like a real library.

also why I did this:

the live preview. I set up a slint library path in build.rs (and a .vscode/settings.json so the preview resolves @penumbra-component), which means I can open any single component file and preview it in isolation instead of booting the whole app. given that basically all my UI work these days IS sitting in the preview nudging things, having the components be individually previewable is a massive quality-of-life win. also just cleaner, the editor-panel.slint used to have all seven block components copy-pasted inline at the top, like 200 lines of it, and now it just imports them. the VMs (BlockVM, NoteCardVM) moved to a data.slint too so the struct definitions aren’t tangled with the theme.

second thing, and this is the fun one

live preview while you type. before, the active block was just a raw text field, you’d type # Heading and see # Heading, and only once you clicked away did it render. now as you type, a little preview well under the input shows what the block WOULD render as, live. type # and it previews as a heading, type > and it’s a quote, type - and it’s a list. there’s a whole set of live_ functions (live_kind_name, live_display_text, live_heading_level, estimate_live_height) that route the currently-typed text through the same display logic the inactive blocks use, so you get instant feedback on your markdown without committing it.

But it’s broken and weird and not working (see previous image)

gosh might take a break from this and think about how to structure it again, Je n’aime pas ce désordre

Better editor

the editor got a real display layer, and honestly most of the actual work here happened in the Slint live preview, just tweaking things until they looked right, so the hours logged don’t reflect it. same story for the last few commits, tbh, a lot of “sit in the preview and nudge” that Hackatime doesn’t see. oh well.

anyway

the thing that makes an editor feel like an editor is that the active block shows raw markdown but every other block renders nicely, and the piece that does that rendering is the new display module. it turns each block into the text you actually want to see.

lists keep their markers because those go through the real markdown renderer. and tables, tables actually render as an aligned grid now, padded columns and a --- separator row, which looks so much better.

there’s also scroll targeting

given the active block, it works out where to scroll the editor so that block is comfortably in view. first block sits at zero, later blocks scroll proportionally, long bodies don’t leave you editing something off the bottom of the screen.

wrote a chunk of tests for all of this

the map got a zoom-to-card thing too

click into a note and the camera zooms so the card fills the viewport width

it’s great that’s what i wanted since the beginning tbh like that animation played in my head

Implement platform-specific modules approach like Saikuro

okay so the ui/penumbra-ui/src/lib.rs had gotten GROSS. it was this soup of #[cfg(not(target_family = "wasm"))] and #[cfg(target_family = "wasm")] blocks sitting right next to each other, two versions of half the functions, the SharedState struct declared twice, the boot path forked inline. every time I added a thing I had to add it twice with the right cfg gate and it was exactly the kind of mess I swore off on day one.

so I stole the pattern from Saikuro

a platform module. now there’s src/platform/mod.rs that declares the shared interface, plus platform/native/mod.rs and platform/wasm/mod.rs behind one cfg switch at the top. the rest of the app just calls into platform:: and doesn’t care which one it got. all the two-versions-of-everything nonsense is now contained in exactly two files that never both compile at once, and lib.rs reads like a single coherent app again instead of a choose-your-own-adventure. it’s so much cleaner.

while I was in the editor I also killed a dumb duplication: penumbra-editor had its OWN BlockKind enum that was just a lossy copy of the one in penumbra-markdown, with a whole block_kind() function to convert between them. why. they’re the same concept. so now the editor re-exports markdown’s BlockKind and clones it straight through, and that whole translation layer is gone.

Broken but a working ish editor!!!

you can WRITE in it now. like actually type words into a note and have them save. it’s janky but it works and I’m hyped.

How

the editor is block-based, which was the whole gamble. instead of one big textarea, a note is parsed into blocks (paragraphs, headings, code fences, etc) and you edit one block at a time. the active block shows its raw markdown so you can mess with the syntax, everything else renders. new session.rs in penumbra-editor holds all of it: EditorSession owns the list of BlockEdits, tracks which one’s active, and each block has a mode, Prose (renders as markdown, Enter splits into a new block), Heading (single line, Enter drops the tail into a paragraph below), and Raw (code, Enter is a literal newline). splitting a block at the cursor, merging, undo/redo with a 64-deep snapshot history. it’s a real little document editor.

uhhh

is it good? no! the commit message says “broken” and I mean it. the split logic has edge cases, the cursor does weird things, switching blocks is finicky. but you can open a note, type, and it round-trips to the markdown file on disk, which is the bar I set and it clears it. barely. hehe.

yeah state is hard

to make the editor and the layout worker both able to touch the graph, Universe’s graph is now an Arc<Mutex>. graph() hands back a guard, graph_handle() hands back the Arc for things that want to hold their own lock. had to sprinkle lock_graph() everywhere but it’s the honest cost of sharing state between the UI thread and the physics thread.

oop forgot about this

also implicit links persist now! the auto-associated links used to evaporate on restart because only explicit wikilinks got rebuilt from the files. now they save to their own file and get restored on open (skipping any whose notes are gone). so the constellations you didn’t draw by hand survive a relaunch too.

eeh

and the TODO… got GUTTED. so much got checked off I almost don’t believe it. Slint UI: done. map canvas v1 with pan/zoom/grid/edges/culling: done. spring animation so cards drift to their spots instead of teleporting: done. local neighborhood layout: done. pinned notes as fixed stars (pin via right-drag, right-tap to unpin, and left-dragging one nudges it back toward its pin): done. positions cached with throttled save + save-on-exit: done. physics running off the UI thread on the tokio runtime so the map doesn’t stutter while it’s solving: done. that whole “Slint” section of the roadmap is basically green now.

new-note-pill.slint got added too, a little floating “new note” thing that’s partially broken.

broken editor, but a broken editor that WORKS, which three months ago I could not have said about any part of the UI.

we’re so back :3

neighborhood steps actually mean something now

so way back, I wrote step_neighborhood as a function that… just called step(). full graph. every time. with a little // Note: functionally equivalent to step() comment that was basically an “it’s fineeee” for now.
it’s been sitting there mocking me… for months
(no joke i had it at #1 on the mental todo after deleting Dioxus)

not anymore! it does the real thing now. when you nudge one note, only the notes within ~400px of it recompute. everything past that radius stays frozen and acts as an anchor so the local cluster settles against the rest of the map instead of the whole universe re-jiggling every frame. this is the “atomic graph updates” idea from the original plan thing, finally real: touch one note, its neighborhood adjusts, the far-off constellations don’t even notice.

Efficiency

to make that not-slow I built a spatial hash. uniform grid, bucket nodes by cell, “give me everything within radius R” only checks the overlapping cells instead of all N nodes. the CPU layout uses it for the localized force pass (repulsion from local neighbors, springs to actual link partners even if they’re outside the neighborhood, gravity), and I also went back and ripped the O(n²) all-pairs loop out of the regular collision resolver and put it on the spatial hash too. that one was quietly quadratic this whole time and would’ve melted at a couple thousand notes. rebuild the hash each pass so pushed nodes land in their new cells. much better.

Collisions?

the collision resolver got a neighborhood-scoped twin as well, so a local step also only de-overlaps locally. nodes inside the neighborhood push each other apart normally; a node that’s an anchor outside it doesn’t move, so the inside node just gets shoved twice as hard away from it. felt like the right call.

PERSISTENCE?!

also taught storage about pins, separate workspace.pins.json file (refactored the workspace/pins file openers to share a state_file helper), and note cards carry a pinned flag now.

Oh lines

and there’s a whole new map module in the UI crate that generates the actual SVG-ish path strings: bezier edges between linked cards in screen coordinates, viewport culling so off-screen edges don’t get drawn, selected-neighborhood highlighting (the selected note’s edges render in the “selected” layer, everything else in “base”), and a grid path that anchors to the camera and clamps its own density so zooming way out doesn’t try to draw ten thousand grid lines.

the map handling a lot of notes without choking was the thing I was VERY, VERY worried about.
Not as much of a problem as I thought! :D

the map is alive

---

You might wonder, why is bro devlogging so often? Wellll, i had this whole plan i wrote down a bit ago to commit when so that it’s not a 10000 line commit (shudders in that happening before) but I severely overestimated how long these things would take

Most of the time not logged was testing, sigh i wish i could devlog that lolll

---

hehe okay so the notes actually live on a map now. like they have positions, computed by the real layout engine, and you can drag the whole thing around and zoom. it’s a canvas!! a real one!!

what and how

so on boot (native), it spins up the LayoutEngine, drops every note in as a node, restores their saved positions if there are any, feeds in the links, and runs the layout to settle everything. then it builds two things for the UI: note cards (with x/y from the layout) and edges (lines between linked notes, computed from the endpoint positions). both get shoved into Slint models and the map renders them. add a new note and it gets added to the layout, re-stepped, and the cards + edges refresh. positions save back to storage when you close the app.

the map-canvas.

slint got the fun part: pan and zoom. a TouchArea catches pointer down/move to drag the camera around, scroll wheel zooms between 10% and 500%, and every card positions itself as (card.x - camera) * zoom. threw in viewport culling too so cards way off-screen just don’t render (visible: only if it’s within ~200px of the edges). and a little “N notes | zoom: X%” readout in the corner because I like knowing things. (it’s broken like most of this, we ignore that for now, polishing is later)

my foolishness

the whole thing is cfg-split for wasm because the GPU layout engine isn’t wired up on the browser side yet, so wasm gets a fallback that just lists the cards without real positions for now. desktop gets the full living map, browser gets the placeholder, both compile, nobody’s angry. (the wasm map is a TODO-shaped hole but it’s an silly one.)

also had to expose universe.storage() so the UI can reach load_positions/save_positions, and added a small EdgeVM struct to carry line endpoints into Slint.

lock in

(the UI exists now. for real this time.)

hehe okay. LOCK IN.

this is the one where Penumbra stops being a really nice backend with no face and becomes an actual app you can look at. Slint UI, real components, and I wrote my own text editor because of course I did.so the editor. new crate, penumbra-editor. I could’ve just shoved a <textarea> equivalent in and called it a day but nooo, I wanted a proper block-based editor like the good note apps have. so it’s a document model made of blocks, each block has styled spans, and you edit it through commands: InsertText, DeleteRange, SplitBlock, MergeBlocks, SetMark, MoveCursor. every command goes through a Journal that’s just a command list + a pointer, which means undo/redo falls out for free (truncate on new push, walk the pointer back and forth). there’s a cursor/selection model and a view_model that snapshots the doc into something the UI can render without touching the guts. it’s BROKEN AND WEIRD NOW BUT IT’LL GET BETTER I PROMISE

the layout engine grew a CPU brain.

it was originally GPU-only via that vibe-graph-layout-gpu crate, which is great until you’re in an environment with no WebGPU (like… a lot of places, including some wasm setups) and then it just returns zero displacement and everything sits in a sad pile. so now there’s cpu.rs, my own Barnes-Hut quadtree force layout in plain Rust, and the engine tries GPU first and falls back to CPU when there’s no backend or the GPU step fails. no more “works on my machine with a discrete GPU only.” graceful degradation, hehe.

and then the actual Slint UI

which is the thing. it’s great.

it’s split into real components now: theme.slint for the tokens, top-bar.slint, map-canvas.slint for the graph, note-card.slint for the cards, editor-panel.slint for writing. app.slint wires it together. this is the part that broke my brain in Dioxus for literal months and in Slint it’s just… a file with a live preview and it looks how I tell it to. I keep waiting for the catch. haven’t hit it yet

tests too

NO JINXING ME (please)

I don’t want to jinx it but. this feels like the first time Penumbra is actually the thing I pictured back in that architecture-day devlog. backend I’m proud of, a UI that doesn’t fight me, my own editor, works on GPU or CPU, desktop or browser.

I locked in :3

working wasm version and candle

it runs in the browser now!! like, actually. wasm target, Slint rendering to a canvas, the whole thing. I’ve been dreading the wasm build this whole project and it just… went. mostly.

the big win here is model caching, because downloading a ~130MB embedding model every single page load is obviously insane. so there’s a model_cache module now that’s the same two functions (get/put) with two backends behind cfg: native writes the bytes to a temp dir, wasm shoves them into the browser’s Cache API. first launch downloads and stashes, every launch after just pulls from cache. same idea as before but now it works on both sides.

to make that clean I split CandleEmbedder::load() into two pieces: from_bytes() that takes raw model/config/tokenizer bytes and builds the thing, and load() (native only) that downloads then calls from_bytes. so the wasm path can grab bytes from cache (or fetch them itself) and hand them straight to from_bytes without the reqwest download dance. Candle running on CPU in wasm, no GPU, producing the same 384-dim vectors. genuinely did not expect that to just work.

the wasm caching code is… a lot of JsFuture and dyn_into and web_sys ceremony. open the cache, build a Request, match it, pull the arraybuffer, convert to a Uint8Array, to_vec. and the put side builds a fake 200 Response with the bytes as the body and stuffs it in. it’s verbose but it’s the standard, nothing clever.

the UI crate is a cdylib now (plus rlib) so it can compile to wasm, tokio features got split per-target (no rt-multi-thread in the browser, obviously), and there’s a tiny index.html that just imports the generated js and calls init(). Slint paints onto a <canvas id="canvas">

also wrote an inference benchmark test… all you need to know

so yeah. desktop AND browser, same codebase, real ML embeddings on both.

the “true cross-platformity” thing from the very first devlog is… actually real now?

that rule I set on day one about no #[cfg] in the interfaces paid off HARD here, the only cfgs are tucked inside the cache and download internals where they belong.

Vault mode (aka I made it Obsidian now, basically)

okay this is a big (foundational) one

so up until now Penumbra stored notes as a big pile of JSON blobs plus a graph.json that held all the links.

which works but it’s… a black box? you can’t open your notes in anything else, the links live in a separate file, it’s very “my app owns your data.” and that bugged me.

so now it’s a vault with markdown files. one .md per note, filename = title, YAML frontmatter for the id/timestamps/tags/pinned/archived, and the body is just… markdown. you could point Obsidian at the folder and it’d basically work. that’s the goal. well. the other way around but still :P

Also, links aren’t stored anymore, they’re derived. [[wikilinks]] in the body get resolved against filenames at scan time and become the explicit edges in the graph.

no more graph.json. the files ARE the source of truth, and if the app crashes mid-write the next scan just repairs everything. I like that a lot. it means if you go edit a file in some other editor and add a [[link]], Penumbra picks it up next time it scans.

and then I fell down the rename rabbit hole.

because if filename = title, renaming a note means renaming the file AND rewriting every [[old title]] across the whole vault to [[new title]].

so propagate_rename walks every note that linked to the old title and rewrites it.

and the rewriter is suuuuuuper careful: it skips inline code, fenced code blocks, indented code, handles [[Old|alias]], matches case-insensitively but keeps whatever case you typed. wikilinks inside `code` should NOT get rewritten and now they don’t, and I have a test that proves it (with the same link four times in different contexts, only the live one changes). very satisfying.

filenames can have slashes and colons and emoji and whatever, so there’s a sanitizer that strips the illegal stuff, collapses whitespace, caps length, falls back to “Untitled” if you somehow title a note with just dots. and if two notes want the same filename you get “Same Name” and “Same Name 2” like every sane file manager.

oh and frontmatter tags vs inline tags are kept separate on purpose. if you wrote #tag in the body, that stays inline, it doesn’t get hoisted up into the YAML just because Penumbra touched the file. only frontmatter tags round-trip through YAML. respecting the user’s file, basically.

the parser also got some unicode love while I was in there. tags can be #Ünïcodé now and nested like #projects/penumbra, and the whole split_text_for_custom walk steps by char instead of byte so it stops potentially panicking on multi-byte stuff. should’ve been like that from the start but oh well.

on the UI side: there’s a folder picker now! rfd native dialog, “Choose your Penumbra vault”, and it remembers your choice in a config file so it doesn’t nag you every launch. also reads a PENUMBRA_VAULT env var if you want to override. getting the macOS dialog to behave was its own thing because rfd has to run on the thread that owns NSApplication, so it marshals onto the Slint event loop and sends the result back over a channel. the browser build just uses OPFS for now, showDirectoryPicker slots in later (TODO left in the code, future me’s problem).

wrote a whole pile of tests for all of it too, but that’s booooriinnnngggggggggggggggggg

Basic template

so. Slint.

I was gonna do the whole “design it in HTML first then port it” thing that I very wisely declared last time and then, naturally, did not do at all.

instead I saw Slint had a live preview and its own markup language and yeah. here we are. and it’s actually fun??

like it has a real preview, you edit the .slint file and it just updates, no recompiling the entire universe to nudge one thing.

after the Dioxus months this feels illegal.

anyway before touching any actual UI I made penumbra-app, which is NOT the UI, it’s the brain. a Universe struct that holds the graph + storage + event bus and does all the note stuff: create, save, delete, restore on open, persist after every change. the whole point is it doesn’t know Slint exists. or Dioxus. or anything.

it just… runs the app, and whatever draws on top pokes it and listens for events. so if I rage-quit Slint in three weeks (I won’t! probably.) the brain doesn’t care.

the actual Slint crate (penumbra-ui) is barely anything yet. cargo.toml, a build.rs because Slint compiles the markup at build time, a main, one app.slint. it’s the hello-world stage. but it launches! natively! no webview! I cannot stress how nice it is to not have a webview.

oh also storage stuff got nudged, everything lives under a Penumbra/ folder now instead of just yeeting files into the shared app-data dir like a menace. and added a with_dir() so tests stop writing to my real data. speaking of, wrote a pile of Universe tests: make a note, drop the whole universe, reopen from the same folder,
is the note still there? yes.
edit it, reopen, still edited?
yes. delete it, reopen, gone? gone.

persistence actually persists, turns out.

months since the last commit lol. I saw the date and just sighed. but the backend was sitting there perfectly fine this whole time waiting for me to stop fighting UI frameworks, and now I’ve got a clean core library and a toolkit that doesn’t actively hate me! Gasp.

good place to be back :3

Revert Dioxus

That’s the sound of me… giving up

Giving up on Dioxus cause it’s so
SO

difficult to use

I’m using Slint now, maybe, we’ll see if i even remember what I was doing

Sorry for the like bad devlog, but it needed to be done

v0.1.0

Well Stardance told me this:

Warning
You have 1h 14m logged that you haven't posted a devlog for. If you ship without posting a devlog, this time will be lost!

IDK why, but i was planning to devlog my silly anyways so:

---

five commits. all tagged v0.1.0. all fixing things that should have been done before I said “the project is ready.” the classic “wait I forgot” sequence.

---

“Oops” - CI was trying to build docs without running npm ci first. added the install step. three words of actual code change. (this was wrong)

“Huh. where did that come from.” - the pages deploy was creating a docs/ directory but the docs copy was still commented out. the demo was deploying fine but docs were going nowhere. fixed the mkdir.

“Oooops guess who forgot to uncomment some code” - uncommented the actual docs copy in static.yml. also fixed the docs URL from nisoku.org (wrong domain) to nellowtcs.me/UTAU.js/docs. details.

“You never saw that coming did you /sarc” - added all the npm publish metadata to package.json. license, author, repository, homepage, bugs URL, and keywords. the keywords are: utau, synthesis, singing, voice, speech, audio, dsp, lf-glottal, source-filter, formant. so if you npm search for any of those things, maybe you’ll find this. someday. (NPM release failed lol, that’s why I did this)

“Badges hehe” - npm version badge, CI status badge, MIT license badge at the top of the README. because a project isn’t real until it has badges. everyone knows that. (/j)

---

five commits to go from “done” to actually done. four of them are CI fixes. like i just said, this is software development.

CI is green. docs are deployed. npm metadata is set. badges are shiny. v0.1.0 is tagged.

---

so. that’s UTAU.js.

six days. from “idk my voice tho” in an Obsidian doc to a parametric singing synthesizer with three languages, 14+ import/export formats, a Svelte demo app, a docs site, 160+ tests, and a v0.1.0 tag.

it sounds mid. like, genuinely. the vowels are recognizable but the consonants are strange and the whole thing has this uncanny valley quality where you can tell it’s trying to be a voice but isn’t quite there. it’s really adorable when it tries :3

but it makes sound. from math. in a browser. and that’s the whole point of v0.1.0. it doesn’t have to be good. it has to exist.

it exists now.

:)

the docs exist and the README isn’t “Web-Template” anymore

so uh. the README has said “Web-Template: My typical web dev starter project” for the entire life of this repo. six days. dozens of commits. a full synthesizer. and the README was still introducing it as a starter template.

fixed that. also wrote an entire documentation site. in one sitting. because apparently that’s how I do things now.

---

the README

133 lines replacing the 3-line placeholder. features list, quickstart code (Node.js WAV rendering + browser streaming in like 10 lines each), API overview table with every public export, development instructions, and a Mermaid architecture diagram.

the mermaid diagram is a mindmap and it renders SO PRETTY on GitHub. multicolored branches for Core DSP, Languages, Synth, Voices, and IO. each branch fans out into its components.
it’s AMAZINGGGGGGGGGGG
I spent way too long picking which mermaid diagram type to use and I regret nothing.

---

the docs site

ten markdown pages built with DocMD. custom Ruby theme with blue/cyan gradient accents, dark mode support, semantic search via HuggingFace transformers, and a proper navigation structure:

getting started: quickstart (5-step guide with code), installation (npm/CDN/source), core concepts (source-filter model explanation with a Mermaid flowchart showing glottal source -> formant cascade -> output)

guide: architecture (module layout, data flow diagram, design decisions), renderer (6-step pipeline walkthrough from lyric to audio), languages (built-in language docs + how to register custom languages + cross-language aliasing)

API reference: full export table, DSP primitives with code examples, VoiceConfig reference (264 lines of parameter documentation with perceptual effect tables for every single field. “openQuotient 0.3 = pressed tense voice, 0.5 = neutral modal voice, 0.7 = breathy soft voice.” that kind of detail.), import/export format support table

the VoiceConfig page alone is probably the most thorough documentation of a formant synthesizer’s parameter space I’ve ever seen in a JS library. I might be biased. I’ve not seen many others. We don’t talk about that.

---

also

added UST and TSSLN to the export format list (they were import-only before). fixed the export function to handle generators that return arrays instead of single Uint8Arrays. CI workflows now build and audit the docs workspace too.

the TODO is almost empty. just “README: quickstart, API overview, architecture diagram” which is now done. the project has: a synthesizer, three languages, 14+ import formats, 14+ export formats, a demo app with a piano roll, voice presets, undo/redo, settings, file import/export, WAV export, tempo control, buffer management, 160+ tests, JSDoc on everything, a full documentation site, and CI that’s green.

six days :O

the documentation commit (part 1)

---

JSDoc everywhere

this is the kind of commit where the line count looks impressive and the actual behavior changes are zero. 118 lines became 216 lines in types.ts alone. every interface, every field, every function got documented.

some highlights:

(don’t ask why i did a double dash. i was being silly and just noticed somehow idk how)

• openQuotient: “fraction of the glottal cycle the vocal folds are open. Higher values produce a breathier, softer sound (0.2–0.9)”
• speedQuotient: “ratio of the opening phase to the closing phase. Higher values give a faster closure, increasing brightness (0.3–3.0)”
• FormantCascade: “cascade of anti-resonators followed by resonators, modelling the vocal-tract transfer function. Anti-resonators (zeros) come first, followed by resonators (poles), matching the source-filter model: S(f) = G(f) * Z(f) / P(f)”
• LFGlottalSource: “Liljencrants-Fant (LF) glottal pulse source with jitter, shimmer, aspiration noise, and DC blocking. See Fant (1986), Liljencrants (1985)”
• renderNote: documents the full pipeline: “lyric -> phoneme symbols -> phoneme lookup -> LF glottal pulse -> formant cascade -> noise mix -> amplitude envelope -> normalisation”

every DSP class, every language module, every player method, every import/export function. the person reading this code in six months (probably me) will thank me.

---

docs soon

Docs next time, I will not be long heheeee

---

TODO cleanup

the languages section is fully checked off now. English CMUDict (124,911 words), Japanese EDICT2 (224,808 kanji-to-kana entries), Mandarin with full pinyin decomposition, and phoneme alias maps for cross-language compatibility. all done. all in previous commits I apparently forgot to devlog. oops.

reorganized the remaining items: Documentation (JSDoc done, README and docs site still needed) and Future (scroll-to-playhead, seek, loop, Web Worker rendering).

the TODO is getting short. that’s a weird feeling for a project that’s six days old.

567,211 lines of “code”

so about that line count. I added the CMU Pronouncing Dictionary as a git submodule. 134,000+ English words with their ARPABET pronunciations. the cmudict repo is… large. like, genuinely enormous. the diff says 567k lines and it’s almost entirely a text file of words and their phoneme sequences.

the actual code changes across these four commits? probably 280 lines. the submodule? 567,131 lines. software engineering.

---

the real G2P

Build/data/cmudict is now a submodule pointing to cmusphinx/cmudict. this is THE pronunciation dictionary. the one that every TTS and speech recognition system uses. “BEAUTIFUL” -> “B Y UW1 T AH0 F AH0 L”. “SQUIRREL” -> “S K W ER1 AH0 L”. every word, every pronunciation variant, every stress marker.

the 100-word hand-rolled dictionary was cute. this is the real thing. (chunk size warning in vite went from 500 to 13000 because bundling the dictionary is… yeah.)

---

glottal pulse fix

the LF glottal source had an audible click on every single cycle. the return phase (after the glottis closes) was a bare exponential decay: -exp(-epsilon * x * period). the problem: it starts at -1 with non-zero slope, so at the boundary between cycles there’s a discontinuity. click. every. cycle.

fix: multiply by sin(pi * x). the sine term makes the slope match the open phase at t=te (C1 continuity) and reach 0 at t=tc. smooth transition. no click. this is how the actual LF model paper describes it but I’d skipped the sine modulation the first time around because I thought it was just cosmetic. it was not cosmetic.

also killed the 0.8/0.2 smoothing filter. it was supposed to prevent clicks but was actually just low-passing the glottal pulse and adding phase distortion. the DC blocker handles drift on its own. coefficient went from 0.995 to 0.99 for faster response.

---

renderer performance

formant coefficient updates were happening every sample. that’s setFormants() with trig functions 44,100 times per second. formants don’t actually change that fast. throttled to every 5ms (221 samples at 44.1k). no audible difference, meaningful CPU savings.

cascade reset came back at phoneme boundaries. I removed it two commits ago thinking filter state carryover was smoother. it is smoother, but it also causes wideband clicks when the coefficients jump. the reset is the right call.

---

aspiration lowpass

aspiration noise on vowels was full-spectrum white noise. real breathy voice has most of its noise energy below 4kHz. added a one-pole lowpass at 4kHz on the aspiration path. sounds warmer and less harsh. also re-added parallel filter resets at phoneme boundaries (matched the cascade reset fix).

---

signal quality tests

new test category: signal quality. a long 8-beat note must stay in [-1, 1], be finite throughout, and not be dominated by white noise. the “not white noise” check uses a crude HF proxy: ratio of RMS(sample-to-sample difference) to RMS(signal) in the steady state portion. pure white noise scores ~1.4, a clean tonal signal scores ~0.2. test asserts between 0.05 and 0.8.

wrote these tests because the glottal fix could have easily introduced NaN propagation or DC drift and I wanted to catch it automatically.

four commits, and failure

I did not intend to do all of this in one sitting. and yet.

---

player batching + buffer pool

the player was creating a new AudioBuffer for every single chunk. for a 100-note score that’s 200 buffer allocations (stereo) plus 200 BufferSourceNodes. the garbage collector was not happy.

rewrote the scheduling layer. chunks now accumulate into batches (~0.5s each) and get merged into a single AudioBuffer per channel before scheduling. acquireBuffer() checks a pool of previously used buffers by sampleRate and length before allocating new ones. onended releases them back to the pool. pre-buffer threshold bumped to 1.0s for more runway.

the result: way fewer Web Audio nodes, way less GC pressure, smoother playback on longer scores.

---

japanese UTAU compatibility

imported UST files from real UTAU projects were breaking because Japanese has a bunch of special lyric conventions I didn’t handle:

• っ (sokuon): geminate consonant marker. maps to a new sil (silence) phoneme.
• ※ (pronunciation alias): text after ※ is the actual pronunciation. あ※ka uses “ka” not “あ”.
• dot-prefixed lyrics (.sil, .S): UTAU rest/silence commands. treated as empty.
• ＠ and ％: stripped. ＠ also acts as a note-join marker in the renderer: if present, the initial consonant gets suppressed so the vowel carries through from the previous note.

also removed the cascade.reset() between phonemes. filter state now carries through for smoother formant transitions instead of clicking at every phoneme boundary.

---

settings panel

new SettingsPanel.svelte. modal overlay, dark theme, closes on Escape or click-outside. currently has one setting: auto-scroll toggle (for a future scroll-to-playhead feature). settings gear icon in the header bar.

---

undo/redo

integrated undora for undo/redo history. 50-state capacity. snapshots are saved on mouse-up in the piano roll, delete key, pitch point changes, and lyric/note-number edits (on blur, not on every keystroke).

Ctrl/Cmd+Z to undo. Ctrl/Cmd+Shift+Z or Ctrl/Cmd+Y to redo. Undo2 and Redo2 icons in the header bar, disabled when there’s nothing to undo/redo. history clears on language switch and file import.

---

the formatting commit

ran prettier again. removed a dead MIN_POOL_SIZE constant and an unused import. cleaned up a few TODO items. the usual.

---

scrollToNotes also got a fix: it was centering on the midpoint of all notes which looked weird when the first note started late. now it clamps to the start of the earliest note so you always see the beginning.

---

also I tried to fix the hissing and static and clicking but to no avail. i tried fixing a LOT of stuff. but. sadly ir only made things worse, so I had to retry. and retry. and retry. and retry. with no luck

I’m not sure what the problem is tbh. Let me use Satori to debug some stuff ig

making it sound less terrible (two commits, one mission)

these two commits are about one thing: the output was robotic and buzzy and I was tired of it. every change here is about making the synthesizer sound more like a voice and less like a modem.

---

the renderer got smarter

cross-note co-articulation. renderNote() now returns { chunk, finalFormants }. the stream passes the previous note’s final formants into the next note’s renderer, and the first phoneme interpolates from those formants instead of jumping cold. gaps between notes reset the chain. notes that follow each other seamlessly now blend their formants across the boundary.

per-phoneme envelopes. the old global 5ms attack / 10ms release is gone. replaced with getPhonemeEnvelopeSamples() which gives each phoneme type its own envelope: plosives get 2ms attack / 15ms decay (sharp burst), consonants and vowels get 5ms / 3ms. every phoneme segment fades independently.

diphthong formant sweeping. PhonemeDef got an endFormants field. if a diphthong has both formants and endFormants, the renderer sweeps between them over the phoneme duration. AY now actually glides from /aa/ to /ih/. EY glides from /eh/ to /ih/. OW glides from /oh/ to /uh/. they sound like diphthongs now instead of static vowels.

vibratoOverride. was defined on Note but never read. now it is. per-note vibrato control works.

---

everything got retuned

glottal source. added shimmer (per-cycle amplitude variation driven by jitter, so the volume wobbles slightly like a real voice). aspiration noise is now high-pass filtered (subtract a lowpass from the raw noise) so it’s airy instead of muddy. aspiration gain bumped from 0.1 to 0.15.

plosive bursts. noise envelope for plosives changed from symmetric fade to a fast 12ms exponential decay. “pa” now sounds like a burst instead of a pop.

formant data for everything. Z, ZH, V, DH, Y, W, HH, JH in English all got formant targets. same for z, h, y, w, j in Japanese. consonants that were previously just noise bursts now resonate through the vocal tract. the difference is huge.

vowel bandwidths tightened. defaults went from 80/100/120 to 70/90/130 Hz. narrower bandwidths = sharper resonant peaks = more vowel-like quality.

voice presets retuned. male voice: lower open quotient (0.4), lower speed quotient (0.65), higher tenseness (0.65), less aspiration (0.05). sounds less breathy, more chest voice. female: formant scale 1.18. gender slider in scaleVoice now affects speed quotient and has a gender-dependent tenseness base.

pitch accent. Japanese got resolveAccents() implementing heiban pattern (low first mora, high rest). the stream groups consecutive notes into phrases, calls resolveAccents, and applies the offsets as constant pitch shifts. it’s basic but it makes Japanese phrases have some melodic contour beyond what the score provides.

---

four TODO items checked off in one go: co-articulation, phoneme envelopes, diphthong sweeping, vibratoOverride. pitch accent too.

it still doesn’t sound human. but it’s starting to sound like it’s trying. and that’s a big step from where it was.

---

if you can identify the song in the editor image, good job, you’re cool :D

CI arc (three commits, one story)

three commits that are really one story: getting CI from “permanently red” to green.

---

the setup

“wrote” (aka copied and modified) six GitHub Actions workflows in one go:

ci.yml: lint + test + build on push/PR. matrix tests Node 20 and 22. builds the library first, then typechecks the demo. (docs build is commented out because docs don’t exist yet. they will. eventually.)

test.yml: dedicated test runner. same Node 20/22 matrix. runs jest in the Build workspace.

release-npm.yml: publishes to npm on GitHub release. strips private/scripts/devDependencies from package.json, copies README and LICENSE into Build/, publishes with --provenance. has a workflow_dispatch with a dry-run option so I can test without actually publishing.

security-audit.yml: runs npm audit --audit-level=high on all three workspaces (root, Build, Demo). daily cron plus on push when package files change.

static.yml: reworked the GitHub Pages deployment. it was pointed at Build/dist (wrong, that’s the library output). now it builds the library, builds the demo, and deploys Demo/dist as the pages root. docs will go in pages-root/docs/ when they exist.

single-file.yml: was still referencing Web-Template (the old template name). fixed to point at Demo, output file is now UTAUjsEditor.html.

also integrated Updato (my own auto-updater library) into the demo. on load it checks the current build hash against the latest commit on main and shows an update notification if there’s a newer version. the build hash gets injected at build time via __BUILD_HASH__ in the vite config.

cleaned up the TODO: removed all the completed checkboxes (they were cluttering the file), added detail to the remaining items.

---

the fixes

CI was 0/3 passing. then 1/8 passing. then 2/8. then eventually 8/8. the classic experience.

the single-file and updato workflows needed the library built before the demo (workspace dependency). added npm ci at root level and a “Build Library” step before the demo build. also added vite-plugin-singlefile and cross-env for the build:single script.

second fix commit added ts-node and unrun as dev deps because the ESM config loading was unhappy without them.

three commits to go from red to green. could be worse honestly but whatever