Starweb dev log #16

DevTools! F12 or Cmd/Ctrl+Shift+I opens a dock on the right side of the page.

Elements works: a picker button lets you click something on the page to select it, or walk the DOM tree directly. Selecting a node shows its computed styles and box model, with real layout padding rather than just the CSS value.

Console works too. Whatever you type evaluates in that tab’s own Lua engine, with history on up/down.

Network, Sources, and Metrics are just tabs for now, not built yet.

Im going to be working on design now

Starweb dev log #15

Added live search and SVG rendering.

Type in the sidebar or the home search, and a dropdown fills in as you go. That needed a new input event for page scripts, they only got click before. Queries are percent-encoded before they hit the URL, with a small urlenc() since this runtime has no encodeURIComponent.

Also added recent pages on the home screen, capped at 3, deduped, ordered so two clicks in the same instant land right.

Starweb dev log #14

Analytics got a range picker, and the drawing underneath it got a lot cheaper to run.

The panel graphs now go from 1h up to 1y, not just a fixed 14 days. That needed a second counter: stats_min, one document per domain per minute, which only exists to answer the sub-day windows and ages out on a TTL index. The old daily stats collection stays forever so the all-time total is still exact. Past a month the buckets widen to weeks then 30-day chunks, since 180 daily points would just read as a smear at chart width.

The nebula background got refactored for speed: prep() runs once per canvas resize and does all the expensive math, square roots, atan2, walking every line segment, into a flat array; lit() runs per frame and just reads that array plus whatever actually changes frame to frame (the shine position, the cog’s rotation). Same picture, way less work per frame. Hex color parsing got a fast path too, since a canvas setting fillStyle per dot was hitting the full color parser thousands of times a frame for the one form it ever uses.

Canvas itself grew a few things: fillRect/strokeRect take an optional radius now, measureText reads the actual font the page pushed instead of the UI default, and hoverX/hoverY are readable off a canvas element. The ctx method lookup also switched from a chain of strcmp to a real table, since a heavy canvas loop calls into it constantly.

requestAnimationFrame self-paces now. A callback that runs long gets a gap after it sized to what it just cost, capped at 500ms, so one expensive canvas can’t eat the whole frame budget. It also only fires for the visible tab and stops while the window’s minimized; timers and fetches keep going regardless, same as a background tab in a real browser.

Starweb dev log #13

Added analytics and worked on the domains page.

The resolver bumps a counter every time it answers a lookup: one document per domain per day, not per query. It’s best-effort: if the analytics write fails, the answer still goes out. Missing days are zero-filled, so a new domain shows a flat line instead of a gap.

That feeds two chart types in Lua, chartBars and chartLine, just canvas: fillRect bars or a stroked polyline. Zero draws nothing, and only a real value gets a 2px floor. The home page gets one combined trend line across every domain, the domains list gets a sparkline per row.

Also fixed a flex-grow input because it didn’t fill its row width. Inputs and textareas now stretch to match, so form rows line up.

Starweb dev log #12

I started redesigning the panel. Signed in, you land on a sidebar with tabs (home, domains, analytics), a search bar, and a home page. (Cloudflare copy ik lol)

The sidebar icons are Lucide SVGs. Those are drawn on a 24x24 grid and use way more of the SVG path language than my login art did: relative commands, H/V, arcs, closepath, plus <circle> elements. The browser has no SVG parser yet (but ig it’s unavoidable at this point), so I wrote a script (shapes.py) that reads each icon once and flattens it into plain points, arcs, and all. Those points get baked straight into the page’s Lua as a table, so the page itself is just normal HTML and Lua, no different from any other page, and the script just loops over the points and draws them.

Rendering also looks sharper now. Layout and drawing account for the framebuffer scale, so the vector art doesn’t go blurry on a hi-DPI window anymore.

Starweb devlog #11

Nebula login page design.

I don’t have much to talk about today, so enjoy the video.

Starweb dev log #10

Starweb has domains now. There’s .web, with a DNS behind it.

StarDNS (name is temporary) is an authoritative server for the .web zone. It answers over UDP, falls back to TCP when a reply won’t fit, and reads records out of MongoDB. Three parts in one process: the resolver, a panel to manage names, and a CA that issues star:// certs.

the panel

Sign in, and you get three domains, each a single label under .web. Records are A, AAAA, CNAME, TXT with a TTL from 60 to 86400. The rules a resolver relies on are enforced on the way in: a CNAME can’t share a name with another record or sit at the apex. @ is the domain itself, *.dev is a wildcard, and a CNAME target with no dot is relative. It’s served with the starweb package, so you browse it in Starmap like any page. Every action is also a JSON route, so it’s scriptable from starweb post.

certificates

“Issue certificate” makes a P-256 leaf covering mysite.web and *.mysite.web, signed by the StarWeb root, valid 825 days. Stored in Mongo and written to dns/issued/; the key never touches the database. The root is name-constrained to .web now, so it can’t issue for a public name; ca.issue("www.google.com") fails its own verify, and a test holds it there.

resolving

The browser and client resolve .web themselves. A host in the zone gets asked straight to StarDNS over UDP (src/common/resolver.hpp), never getaddrinfo, which would leak the lookup to a public DNS. No fallback: if StarDNS can’t answer, the load fails instead of quietly asking the public resolver. Anything that isn’t .web still goes to the system resolver. It chases CNAMEs, queries A and AAAA separately, and caches by TTL so a page’s subresources cost one lookup. Point it elsewhere with STARWEB_DNS, or STARWEB_DNS=off to fall back.

Next is the look of the website.

Starweb dev log #9

Media streams now instead of downloading the whole file first.

A big MP4 used to sit there until every byte landed. Now there’s a MediaSource: a seekable view of the remote file that fills in as it’s read, so the decoder starts on the first megabyte while the rest is still downloading.

It runs on Range requests. The server advertises Accept-Ranges: bytes and answers bytes=a-b, bytes=a-, or bytes=-n (trailing bytes, how a player finds an MP4’s moov atom) with 206. Bad or multi-range requests get 416. It streams the file out in chunks rather than loading it into memory, so the server side is cheap too.

On the browser, chunks land in a sparse cache file in 64KB pieces, and a read only blocks on the chunks it touches. A .ranges sidecar tracks what’s present; once it’s all there, the file is the cache entry. Seeking doesn’t wait behind the sequential download; hint_seek steers the prefetcher to where you jumped. Both backends got wired in: FFmpeg through custom IO, macOS through AVAssetResourceLoader.

Cache pruning counts real bytes on disk now and cleans up the .ranges sidecars, and there are range tests on the Python side (bytes=a-b, EOF, suffix, 416, clamping).

Starweb dev log #8

Page scripts can fetch() now.

fetch("/api/time", function(err, res) ... end) runs on a worker thread and calls you back when it’s done, so the page never freezes. You get res.status, res.ok, res.body, res.headers, and res:json(). POST works too: fetch("/api/echo", { method = "POST", json = { hi = "there" } }, cb). There’s a json global with it (json.encode/json.decode).

Limits: 6 fetches at a time, 1MB request body, 8MB response, caps on headers, URL length, and JSON depth. Cross-origin requests are CORS-gated by browsers; a fetch to another host only returns a body if it sends Access-Control-Allow-Origin. The Python server got a per-route cors= option to match.

Also refactored the fetcher: one perform_request with RequestOptions (method, headers, body, timeouts, max size), and perform_fetch is a thin wrapper for navigations on top. Scripted fetch and the address bar share code now instead of two separate socket loops.

StarWeb dev log #7

You can write StarWeb backends in Python now.

Everything so far has been C++. Writing a page that did anything meant static files or a Lua script in the browser. So there’s now a starweb Python package that speaks STWP, in two halves.

The server half looks like Flask. Make an App, decorate functions with @app.route("/api/time"), return a dict, and it goes out as JSON, or a Response for control over status and headers. Routes take params (/api/greet/<name>), the request has a query dict, and app.mount_static("/", "www") serves files with routes winning over static. app.run(scheme="both") hosts moon:// and star:// at once, thread per connection, same as the C++ server.

The client half looks like requests. starweb.get("moon://localhost/"), .post(...), a Session to reuse a connection, and TLS for star:// with each failure getting its own exception (TLSVerificationError, MixedContentError, ALPNError).

it only speaks STWP

The interesting part is what it refuses. Send the server an HTTP request line, and it answers 505 and hangs up; curl and a browser can’t wander in by accident. The client trusts only the StarWeb root CA, never the system roots, and it’s pinned to TLS 1.3 and nothing else. ALPN is required: no stwp/1.0, connection dropped.

the CLI

starweb get moon://localhost/index.html fetches a URL, -v for headers and TLS info. starweb serve app.py runs a file defining an App, with --scheme, --no-tls, --log. curl and a dev server in one command.

It’s a proper pip package too: README, license, pyproject.toml, pip install starweb and you get a starweb command. And there are tests: interop against the C++ server, URL and message parsing, schemes, isolation, which caught a couple of parser mismatches between the two.

StarWeb dev log #6

StarWeb has secure connections now: star://.

star:// is to moon:// what https is to http: same STWP messages; the transport underneath is just TLS 1.3 (OpenSSL 3) instead of plaintext TCP. Plaintext moon:// stays on port 8090, encrypted star:// runs on 8490, and the server serves both at once. There’s a tools/make_certs.sh that spins up a local root CA and a server cert so you can test it. The CA only signs localhost, .local/.star, and private IPs (10.x, 192.168.x, etc.), so it works on your LAN but not the public internet. (I it will be public after i make DNS)

The address bar got a lock icon. Click it, and you get a cert info popup: who issued it, who it’s for, that kind of thing. If the handshake fails, verification fails, cert’s self-signed, expired, wrong host, you get a full-page interstitial instead of the page, same as a real browser yells at you.

the security rules

A star:// page can’t pull in moon:// content. Mixed content is blocked; no loading plaintext resources into a page you’re being told is secure.

Downgrades are blocked too, and stricter than the normal web; here, a script can’t navigate you from star:// to moon://. You can still type a moon:// URL yourself; that’s your call, but a page can’t quietly drop you off TLS.

TLS also enforces ALPN and checks the hostname, and there’s a session resumption cache so reconnecting doesn’t redo the whole handshake every time.

Also fixed URL parsing to handle IPv6 addresses in brackets ([::1]), which it just choked on before.

StarWeb dev log #5

Lua support!

Every tab gets its own ScriptEngine, a sandboxed Lua 5.4 interpreter. A custom allocator caps its memory, an instruction hook kills it when it runs past its time budget, and require, dofile, and loadfile don’t exist in there. It’s untrusted code from whatever page you loaded.

Scripts get a DOM API: getElementById, querySelector, innerText, get/setAttribute, addEventListener, console.log, alert, setTimeout/setInterval, requestAnimationFrame. There’s a location object too, so scripts can navigate, but only to moon:// URLs.

Yes, there is <canvas>. getContext("2d") gives you fillRect, paths, arcs, fillText, but scripts never touch the screen directly; every call gets recorded into a list of CanvasOps and the renderer draws the list. To test all of it, I wrote a raycaster (www/game.lua), a small maze you can walk around in, running entirely as a page script.

external scripts and keyboard input

<script src="..."> works, not just inline scripts. The fetcher downloads them so the engine sees them in document order. Keyboard input works too; keys get polled from GLFW and sent to the active tab, so document.addEventListener("keydown", ...) does something. That’s how the raycaster gets its controls.

The main loop also stopped redrawing at 60fps when nothing’s happening. It sleeps until something needs a frame: a fetch, a playing video, a pending timer, or requestAnimationFrame. Good for the battery. (My MacBook 2015 was on fire when the browser ran. That’s how I noticed lol)

Also, from now on I will be posting more devlogs because im done with the core stuff. Rupnil () said not to make them that big and commit more.

StarWeb dev log #4

CROSS-PLATFORM SUPPORT AHHH!!! (BOTH MICRO SL.. WINDOWS AND LINUX! and yk it was working before on MacOS)

The socket code was the first problem. Everything was written against BSD sockets, which Windows doesn’t speak. I pulled all of that behind a net.hpp wrapper: a socket_t type, a kInvalidSocket constant, and helpers like net::close, net::is_valid, and set_recv_timeout that pick BSD sockets or Winsock2 depending on platform. Small thing, but an invalid socket is -1 on POSIX and an unsigned INVALID_SOCKET on Windows, so you can’t just reuse the same sentinel value everywhere like I was doing before.

Media playback was the other one. AVFoundation isn’t available outside macOS, so VideoPlayer now has two backends: AVFoundation on macOS, and a FFmpeg + miniaudio backend for Windows/Linux. Same play/pause/seek/volume API.

Alongside that: a CMake build, a BUILDING.md with setup steps per platform, GitHub Actions building targets on push, and a LICENSE + third-party notices file, and that needed to be documented properly instead of just… not. (FFmpeg licensing is so confusing)

Then: flexbox, and input types

Once the cross-platform stuff was done, I went back to layout, which had been held together by manual cursor-position math since day one. I imported Yoga (the flexbox engine Meta built for React Native) and wired it into the renderer through a compute_flex_layout call, so display: flex, gap, and the flex-* properties are now working.

The parser also handles text a lot better now. Before, it just dumped every raw character into text_content, so something like & showed up on the page as & instead of &. Now it decodes entities and splits text into its own #text nodes instead of gluing everything onto the parent tag.

Forms are also better now because of new input types: name, min, max, step, checked, and gave the renderer form controls instead of the placeholder boxes, plus calendar and clock widgets for date/time inputs. Submitting one actually resets and collects values now.

And we know what happens now…

StarWeb dev log #3

Finally made a media player!

The fetcher now actually looks at what it downloaded instead of assuming everything’s HTML. It checks the Content-Type header first, falls back to the file extension if the server didn’t send one, and sorts the response into HTML, image, video, or audio. Anything that doesn’t match any of those (a plain .txt file, for instance) just gets rendered as text on the page instead of the browser trying to parse it and displaying nothing. Images and media get pulled down recursively too. The fetcher walks the page for <img>, <video>, <audio>, and <source> tags and fetches each one alongside the page itself, the same way it already did for stylesheets.

Images go through stb_image, decoded from memory into an OpenGL texture. For video and audio tho, I built a small VideoPlayer wrapper around AVFoundation, so playback is currently macOS-only (sorry again, but I didn’t add support for Linux/Windows yet; just wait for next devlog, I promise). Fetched media gets cached to a folder on disk under a hashed filename, since AVFoundation wants an actual file to point at rather than a memory buffer.

resolve_url was improved. It now knows moon:// defaults to 8090 and star:// to 8490, so URLs don’t end up with a redundant :8090 attached onto every link.

@Rupnil sorry man, I ran into issues ;-;

StarWeb dev log #2

The client could fetch static files but couldn’t render them. Starweb finally has a browser called Starmap! The first version wasn’t much: a minimal parser splitting tags and attributes on whitespace, and a couple CSS properties. Only one page could be loaded at a time; it wasn’t much of a browser, but I wanted to at least render something.

The whitespace-splitting parser didn’t last long. It broke when an attribute value contained a space, which style="..." does constantly. So I rewrote it into a proper scanner that walks the tag character by character and actually respects quotes. That also made it easy to start picking up attributes I wasn’t tracking before: id, style (parsed into real properties instead of just stored as text), and type, value, and placeholder for form inputs. Nothing renders them yet, but the data’s there.

CSS grew from a couple properties to something you can work with: padding and margin per side, width, height, border-width, border-color, font-size, display, each with its own inheritance rule so one style doesn’t just blindly overwrite another. Headings finally scale too; h1 down to h6 each get their own font-size multiplier instead of rendering at the same size as everything else. Pages could finally pull in external styling too, via <link rel="stylesheet">: the fetcher walks the parsed page for stylesheet links, resolves them against the current URL, and fetches each one as a secondary request before the CSS gets parsed.

The layout change with the biggest visible impact was inline flow. Up to this point, every tag forced a new line, so a paragraph with a link in the middle of it rendered as three separate lines. Spans, links, buttons, and form controls now sit on the same line instead. <input> also stopped incorrectly behaving like a container element, since it shouldn’t be able to hold children.

Then I added tabs. Each tab now owns its own URL, page, style, and fetch state. The window title and each tab’s label now follow the page’s <title>, falling back to the host and path if the page didn’t set one. I also swapped the default OS window frame for custom traffic-light controls (because I’m on a Mac and that’s how Chrome does it; sorry if it looks wrong on Linux or Windows, I haven’t tested yet). And also, resizing works correctly since the first version.

By this point, the browser source had grown to 1700+ lines of code, holding everything: parsing, fetching, theme, and rendering. I split the browser into modules, leaving the main file responsible only for booting up and driving the UI loop.

Woah, that was a lot, but I managed to do it in 3 days? idk how to count it because my sleep schedule is so cooked that I often worked at night and went to sleep at 4 am.

StarWeb is my attempt at building an internet of my own. A custom protocol, a custom URL scheme, and a client and server that communicate using them.I put together STWP, my own protocol with request and response parsing, structured similarly to HTTP but built from scratch. The server runs multithreaded, serving files from a www folder, blocking path traversal attempts, and returning proper error codes when something’s missing or wrong. The client connects using its own URL format, moon://host:port/path, sends a request, and prints back whatever the server responds with. Essentially my own version of curl made for testing; I’m planning to make a browser later. Right now it only handles fetching static files.