Devlog: Making Flower’s Type Tracking “Enough” for Stdlib / String Work

The original motivation was simple and — supposedly — quick; I wanted to move on to v1.1.1, “stdlib basics,” especially string utilities. But Flower’s current compiler did not really know what types expressions had. It mostly had pointer_depth on declarations, which was enough to generate * in C types and sometimes guess whether field access should become . or ->.

That worked for very simple cases:

p: @Vector2 = new Vector2
p.x = 5

But it broke down for nested access:

outer.inner.value
outer.inner_ptr.value

The compiler could see the syntax chain, but it did not know whether inner was a value field or pointer field. Since Flower source only uses ., the C backend needs to decide whether each hop becomes . or ->.

That meant stdlib work was blocked. If I want something like:

if strings.equal(string_a, string_b):
    print("same\n")
end

then Flower needs to understand that string_a.length, string_a.data, etc. are real typed fields, not just random syntax to dump into C.

Originally I thought I could just simply create a Symbol Table to get that to work. Unfortunately, that plan soon failed as it introduced a mariad of issues relating to the import system. At the time, this frustrated me because I knew the import system sucked, but I didn’t want to fix it until a later version.

Eventually, I settled on a specific plan which did actually end up changing how imports are handled.. kinda.

The Plan

The design I landed on was to add a typecheck/type-tracking stage between parsing and codegen:

lexer -> parser -> typecheck -> codegen

The parser should stay mostly syntax-only. It builds AST nodes like “dot access,” so I didn’t really change it.

The typecheck pass now collects declarations into a TypeEnv:

struct TypeEnv {
    structs: StructInfo[1024],
    struct_count: int,

    vars: VarInfo[8192],
    var_count: int,

    funcs: FuncInfo[8192],
    func_count: int,

    error_count: int
}

Then it walks expressions and annotates dot access nodes with an access kind:

ACCESS_UNKNOWN: int = 0
ACCESS_DOT:     int = 1
ACCESS_ARROW:   int = 2

So codegen no longer has to fully guess. It can do:

if ast.data._dot_access.access_kind == ACCESS_ARROW:
    fprintf(out, "->%.*s", ...)
else if ast.data._dot_access.access_kind == ACCESS_DOT:
    fprintf(out, ".%.*s", ...)
else:
    // fallback
end

That fallback is important because Flower is self-hosted-ish right now. The compiler is written in Flower, and the compiler’s own source code is full of field access. If the new typechecker is too strict too early, it breaks the compiler while trying to compile the compiler. Very elegant. Very cursed. Just how I like it ;)

Import-Aware Type Tracking

At first, typechecking only saw the main file, so I added a module loading pass. Instead of codegen discovering and parsing imports late, the compiler now loads modules earlier:

load main module
load imports recursively
collect declarations from all modules
typecheck all modules
codegen all modules

That required a new module structure:

struct Module {
    path: @char,
    src: @char,
    tokens: @TokenStream,
    ast: @AST
}

struct ModuleSet {
    modules: Module[128],
    count: int
}

Unfortunately, it didn’t go as smoothly as I wanted. Near the end when I was sure everything would work, I ran into an Idempotency failure due to the new handling of system imports. I.. just manually passed it :p