You are browsing as a guest. Sign up (or log in) to start making projects!

Flower

  • 7 Devlogs
  • 29 Total hours

A compiled bootstrapped low-level programming language focusing on simplicity, versatility, and ability.

⭐ cskartikey marked your project as a Super Star! As a prize for your great work, look out for a bonus prize in the mail :)

0 20
Open comments for this post

2h 45m 9s logged

Devlog: Building Flower’s String Type Foundation

The core goal was to make this work in a meaningful, compiler-owned way:

name: string = "Ivy" as string
raw: @char = name as @char
other: string = raw as string

if name == other:
    print("same\n")
end

print(name.length)

That meant Flower needed more than a typedef in generated C. It needed actual type rules, actual lowering rules, and I needed enough hairs on my head to not be bald after the amount of scares that occurred.

What changed

The first step was giving string real compiler support without going too far too fast.

I added support for:

  • explicit casts between string and @char
  • string equality / inequality
  • string.length
  • runtime helpers in generated C for:
    • flower_string_from_cstr(...)
    • flower_string_eq(...)
    • flower_print_string(...)

That gave Flower a usable string foundation while still keeping literals conservative for self-hosting. Right now, string literals are not fully “native string values everywhere” yet; the compiler still uses explicit casts like "Ivy" as string as the safe bridge.

The reason for this “safe bridge” is because when I got too ambitious, the compiler would break since everywhere @char was used with a string literal it’d break due to no implicity being allowed.

I had to comment out the following new changes and keep the legacy support so it’d compile:

else if ast.kind == AST_STRING_LIT:
        // fprintf(out, "flowe_string_from_cst(")
        // fprintf(out, "%.*s", ast.data._string.str_length, src + ast.data._string.str_start)
        // fprintf(out, ")")
        // LEGACY CODE HERE

and

else if expr.kind == AST_STRING_LIT:
        // set_plain_type(out, TOKEN_STRING)
        // LEGACY CODE HERE
        return 1 

The part that broke

The most annoying issue was not string equality or .length. It was print.

Originally, non-string print(...) was basically lowered like this:

printf(expr);

That only works when expr is already a C format string. So something like:

print(name.length)

generated invalid C:

printf(name.length);

The fix was to make print(...) type-aware. Typecheck now records the operand type, and codegen lowers print differently for string, @char, char, floats/doubles, and integer-like values.

That sounds small, but it matters a lot: once string became real, print could no longer get away with pretending every printable value was already a C string.

An Unsolved Bootstrap Mystery

Somehow Flower’s bootstrap process was breaking, and it will probably break again. Despite the bootstrap requiring it to compile itself, somehow afterwords the binary could become corrupted.

l-2: ~/Documents/GitHub/FloC % make bootstrap
=== Building new version ===
Compiled ./bin/Flower_new.c → ./bin/Flower_new
=== Testing new compiler ===
Compiled ./bin/Flower_test.c → ./bin/Flower_test
Verified bootstrap build complete

l-2: ~/Documents/GitHub/FloC % make bootstrap
=== Building new version ===

Luckily, I always make sure to keep a backup Flower bin stored under /bin/Flower_backup so I used that to compile the new binary and then it worked.

Where this leaves v1.3

What still remains is the bigger ergonomic question: how fully native string literals should behave, and how to migrate the compiler’s own source to that world without setting bootstrap on fire again. I’m not sure yet how I want to approach this question, or what the goals for it should even be, but I guess I’ll have to find out soon enough!

That is the next fight. But at least now, it is a fight on solid ground.

Devlog: Building Flower’s String Type Foundation

The core goal was to make this work in a meaningful, compiler-owned way:

name: string = "Ivy" as string
raw: @char = name as @char
other: string = raw as string

if name == other:
    print("same\n")
end

print(name.length)

That meant Flower needed more than a typedef in generated C. It needed actual type rules, actual lowering rules, and I needed enough hairs on my head to not be bald after the amount of scares that occurred.

What changed

The first step was giving string real compiler support without going too far too fast.

I added support for:

  • explicit casts between string and @char
  • string equality / inequality
  • string.length
  • runtime helpers in generated C for:
    • flower_string_from_cstr(...)
    • flower_string_eq(...)
    • flower_print_string(...)

That gave Flower a usable string foundation while still keeping literals conservative for self-hosting. Right now, string literals are not fully “native string values everywhere” yet; the compiler still uses explicit casts like "Ivy" as string as the safe bridge.

The reason for this “safe bridge” is because when I got too ambitious, the compiler would break since everywhere @char was used with a string literal it’d break due to no implicity being allowed.

I had to comment out the following new changes and keep the legacy support so it’d compile:

else if ast.kind == AST_STRING_LIT:
        // fprintf(out, "flowe_string_from_cst(")
        // fprintf(out, "%.*s", ast.data._string.str_length, src + ast.data._string.str_start)
        // fprintf(out, ")")
        // LEGACY CODE HERE

and

else if expr.kind == AST_STRING_LIT:
        // set_plain_type(out, TOKEN_STRING)
        // LEGACY CODE HERE
        return 1 

The part that broke

The most annoying issue was not string equality or .length. It was print.

Originally, non-string print(...) was basically lowered like this:

printf(expr);

That only works when expr is already a C format string. So something like:

print(name.length)

generated invalid C:

printf(name.length);

The fix was to make print(...) type-aware. Typecheck now records the operand type, and codegen lowers print differently for string, @char, char, floats/doubles, and integer-like values.

That sounds small, but it matters a lot: once string became real, print could no longer get away with pretending every printable value was already a C string.

An Unsolved Bootstrap Mystery

Somehow Flower’s bootstrap process was breaking, and it will probably break again. Despite the bootstrap requiring it to compile itself, somehow afterwords the binary could become corrupted.

l-2: ~/Documents/GitHub/FloC % make bootstrap
=== Building new version ===
Compiled ./bin/Flower_new.c → ./bin/Flower_new
=== Testing new compiler ===
Compiled ./bin/Flower_test.c → ./bin/Flower_test
Verified bootstrap build complete

l-2: ~/Documents/GitHub/FloC % make bootstrap
=== Building new version ===

Luckily, I always make sure to keep a backup Flower bin stored under /bin/Flower_backup so I used that to compile the new binary and then it worked.

Where this leaves v1.3

What still remains is the bigger ergonomic question: how fully native string literals should behave, and how to migrate the compiler’s own source to that world without setting bootstrap on fire again. I’m not sure yet how I want to approach this question, or what the goals for it should even be, but I guess I’ll have to find out soon enough!

That is the next fight. But at least now, it is a fight on solid ground.

Replying to @IvyMycelia

0
14
Open comments for this post

3h 48m 29s logged

Devlog: Making Bool Real Without Breaking Bootstrap

The original goal, asper usual, was straightforward: before moving on to string, Flower needed bool to actually mean something semantically.

Up to this point, boolean-ish behavior mostly worked by accident. Comparisons, conditions, returns, and general expression checking were still loose enough that the compiler could get away with treating a lot of things as “close enough,” especially since the C backend ultimately lowers bool to int anyway.

That was not going to hold once v1.3 started leaning on richer typing.

So this was about drawing a clean line between Flower semantics and C lowering.

The Goal

In Flower:

  • bool should be its own real type
  • comparisons and logical operators should produce bool
  • arithmetic should not quietly accept bool
  • implicit conversion should stay narrow and predictable
  • explicit casting should remain available through as

In C:

  • bool can still lower to int

That distinction ended up being the easy part.

What Changed

  • The typechecker now treats bool as a proper language type instead of an int-shaped placeholder.
  • Condition checks were tightened so if, while, and not use condition-compatible rules.
  • Arithmetic paths now reject boolean operands, and return / initializer / argument checking now runs through more explicit conversion rules.

I also kept implicit widening one-directional:

int -> float -> double

while leaving narrowing conversions as explicit casts.

The actual implementation of booleans and their typing, enforcement, and semantics was a breeze. It’s what came after that got me.

What Actually Broke

As usual, bootstrap had opinions.

The stricter type work exposed older weaknesses that were easy to miss before (This is my coping):

  • function argument checks needed array-to-pointer decay
  • alias and module call lookup paths were too narrow
  • unresolved calls could prevent nested expressions from being fully type-resolved
  • that, in turn, broke some dot-access lowering and caused bad . vs -> emission in generated C

So part of this checkpoint turned into compiler stabilization work rather than just bool semantics due to my crappy ‘temporary’ logic, error handling — or lack thereof, and impeccable ability to miss very important details.

I’m an amazing programmer, if you couldn’t tell :p

Result

Flower now has a much firmer semantic base for bool, and the compiler is back to bootstrapping cleanly with those rules in place.

That matters because this was really the floor for the next part of v1.3: string.

And.. maybe I’ll fix the bootstrap process so it doesn’t blow up in my face again. Oh yea, forgot to mention that. The bootstrapper bootstrapped itself into this very broken parser state that refused to go away so I had to stash my changes, checkout the prior merge (detached), and then recompile manually and fix the parser.

Toodles!

Devlog: Making Bool Real Without Breaking Bootstrap

The original goal, asper usual, was straightforward: before moving on to string, Flower needed bool to actually mean something semantically.

Up to this point, boolean-ish behavior mostly worked by accident. Comparisons, conditions, returns, and general expression checking were still loose enough that the compiler could get away with treating a lot of things as “close enough,” especially since the C backend ultimately lowers bool to int anyway.

That was not going to hold once v1.3 started leaning on richer typing.

So this was about drawing a clean line between Flower semantics and C lowering.

The Goal

In Flower:

  • bool should be its own real type
  • comparisons and logical operators should produce bool
  • arithmetic should not quietly accept bool
  • implicit conversion should stay narrow and predictable
  • explicit casting should remain available through as

In C:

  • bool can still lower to int

That distinction ended up being the easy part.

What Changed

  • The typechecker now treats bool as a proper language type instead of an int-shaped placeholder.
  • Condition checks were tightened so if, while, and not use condition-compatible rules.
  • Arithmetic paths now reject boolean operands, and return / initializer / argument checking now runs through more explicit conversion rules.

I also kept implicit widening one-directional:

int -> float -> double

while leaving narrowing conversions as explicit casts.

The actual implementation of booleans and their typing, enforcement, and semantics was a breeze. It’s what came after that got me.

What Actually Broke

As usual, bootstrap had opinions.

The stricter type work exposed older weaknesses that were easy to miss before (This is my coping):

  • function argument checks needed array-to-pointer decay
  • alias and module call lookup paths were too narrow
  • unresolved calls could prevent nested expressions from being fully type-resolved
  • that, in turn, broke some dot-access lowering and caused bad . vs -> emission in generated C

So part of this checkpoint turned into compiler stabilization work rather than just bool semantics due to my crappy ‘temporary’ logic, error handling — or lack thereof, and impeccable ability to miss very important details.

I’m an amazing programmer, if you couldn’t tell :p

Result

Flower now has a much firmer semantic base for bool, and the compiler is back to bootstrapping cleanly with those rules in place.

That matters because this was really the floor for the next part of v1.3: string.

And.. maybe I’ll fix the bootstrap process so it doesn’t blow up in my face again. Oh yea, forgot to mention that. The bootstrapper bootstrapped itself into this very broken parser state that refused to go away so I had to stash my changes, checkout the prior merge (detached), and then recompile manually and fix the parser.

Toodles!

Replying to @IvyMycelia

0
9
Open comments for this post

1h 15m 7s logged

Devlog: Fixing the Typechecker After Bool Enforcement

Getting bool into Flower exposed a reallyyyy annoying problem: the typechecker had a couple of tiny logic mistakes (I’m so good at this) which caused the bootstrap to blow up in my face and I ahd no idea why.

At first the errors looked unrelated. The compiler started reporting huge numbers of mismatches across main.flo, lexer.flo, module.flo, typecheck.flo, and codegen.flo. I was so confused as to why out of nowhere 903 lines were reporting various typechecking errors when they shouldn’t be.

As it turns out, types_match() was the culprit. The function had been inverted in a few places: Mismatched base types and pointer depths were returning success, while normal matching primitive types were falling through to failure. That made the new compiler think ordinary initializers, assignments, comparisons, and returns were all invalid.

Once that was fixed, the remaining failures turned out to be a second real compiler bug: pointer arithmetic over arrays was decaying in the wrong direction. Expressions like:

mods.modules + j
env.structs + env.struct_count
arr + 1

should produce pointer types. Instead, the typechecker was reducing pointer depth when arrays decayed, which made perfectly normal indexed access and table walking look type-invalid.

So this dev session ended up doing three important things:

  • fixed types_match(...) so actual matches succeed and actual mismatches fail
  • preserved decimal compatibility for float / double matching
  • fixed array-to-pointer decay in pointer arithmetic so compiler internals and examples resolve correctly again

What made this especially annoying is that the symptoms looked much larger than the cause. A couple of bad returns in the typechecker created hundreds of downstream errors, and because Flower is self-hosting, that meant the compiler started rejecting its own source.

After correcting those core issues, both bootstrap and the example test suite started working again. That is the real value of issues like this: not just “fewer errors,” but restoring (MY) trust in the compiler and its ability to enforce Flower’s rules and report issues.

With that stable again, the path forward is much clearer: continue bool work from a solid base, then move on to the remaining numeric and string improvements without the whole compiler collapsing under one bad predicate.

Devlog: Fixing the Typechecker After Bool Enforcement

Getting bool into Flower exposed a reallyyyy annoying problem: the typechecker had a couple of tiny logic mistakes (I’m so good at this) which caused the bootstrap to blow up in my face and I ahd no idea why.

At first the errors looked unrelated. The compiler started reporting huge numbers of mismatches across main.flo, lexer.flo, module.flo, typecheck.flo, and codegen.flo. I was so confused as to why out of nowhere 903 lines were reporting various typechecking errors when they shouldn’t be.

As it turns out, types_match() was the culprit. The function had been inverted in a few places: Mismatched base types and pointer depths were returning success, while normal matching primitive types were falling through to failure. That made the new compiler think ordinary initializers, assignments, comparisons, and returns were all invalid.

Once that was fixed, the remaining failures turned out to be a second real compiler bug: pointer arithmetic over arrays was decaying in the wrong direction. Expressions like:

mods.modules + j
env.structs + env.struct_count
arr + 1

should produce pointer types. Instead, the typechecker was reducing pointer depth when arrays decayed, which made perfectly normal indexed access and table walking look type-invalid.

So this dev session ended up doing three important things:

  • fixed types_match(...) so actual matches succeed and actual mismatches fail
  • preserved decimal compatibility for float / double matching
  • fixed array-to-pointer decay in pointer arithmetic so compiler internals and examples resolve correctly again

What made this especially annoying is that the symptoms looked much larger than the cause. A couple of bad returns in the typechecker created hundreds of downstream errors, and because Flower is self-hosting, that meant the compiler started rejecting its own source.

After correcting those core issues, both bootstrap and the example test suite started working again. That is the real value of issues like this: not just “fewer errors,” but restoring (MY) trust in the compiler and its ability to enforce Flower’s rules and report issues.

With that stable again, the path forward is much clearer: continue bool work from a solid base, then move on to the remaining numeric and string improvements without the whole compiler collapsing under one bad predicate.

Replying to @IvyMycelia

0
50
Open comments for this post

4h 3m 12s logged

Devlog: Making Flower’s Bool Support Real Enough to Survive Bootstrap

The goal sounded small at first: add proper boolean support, while still lowering booleans to integers in C.

In practice, this turned into an overarching type-system problem (typical!).

Flower had reached a point where bool could not just be a naming trick over 1 and 0. The compiler itself needed to understand when an expression was semantically boolean, when it was integer, and when older bootstrap-era shortcuts were no longer valid.

A lot of those shortcuts had survived because the backend lowers bool to int anyway. But self-hosting exposed a major issue: source-level typing and C lowering are not the same thing.

Things like these started breaking:

val: int = parser_peek(ps).kind == TOKEN_TRUE

and helper predicates that returned comparison expressions from int-typed functions.

So the checkpoint became: keep C lowering simple, but make Flower’s typechecker treat booleans as a real source-level type.

The main changes were:

parser -> typecheck -> codegen
  • true / false now parse into AST_BOOL_LIT
  • bool literals are still emitted as 0 / 1 in C
  • logical and comparison expressions now resolve to bool
  • arithmetic rejects bool operands
  • if, while, and not now validate condition-compatible expressions
  • variable initializers, assignments, and returns now check types more strictly

One of the more annoying bugs turned out not to be “about bools” at all.

When code like this failed inside the compiler:

defined_structs[i].length
emitted_imports[i].path

the real problem was that for i in 0..count was never registering i in the type environment, so indexed access looked broken when really the loop variable itself had no known type. Once loop indices were typed as int, those field-access failures disappeared (yay!!).

The result is a much better place to build from:

  • bool is now semantically distinct in Flower
  • it still lowers cleanly to integer values in generated C
  • bootstrap works again
  • the compiler is stricter without collapsing under its own source

That makes the next step much easier: real string support, instead of trying to build strings on top of a type system that still half-thinks everything is an int.

Devlog: Making Flower’s Bool Support Real Enough to Survive Bootstrap

The goal sounded small at first: add proper boolean support, while still lowering booleans to integers in C.

In practice, this turned into an overarching type-system problem (typical!).

Flower had reached a point where bool could not just be a naming trick over 1 and 0. The compiler itself needed to understand when an expression was semantically boolean, when it was integer, and when older bootstrap-era shortcuts were no longer valid.

A lot of those shortcuts had survived because the backend lowers bool to int anyway. But self-hosting exposed a major issue: source-level typing and C lowering are not the same thing.

Things like these started breaking:

val: int = parser_peek(ps).kind == TOKEN_TRUE

and helper predicates that returned comparison expressions from int-typed functions.

So the checkpoint became: keep C lowering simple, but make Flower’s typechecker treat booleans as a real source-level type.

The main changes were:

parser -> typecheck -> codegen
  • true / false now parse into AST_BOOL_LIT
  • bool literals are still emitted as 0 / 1 in C
  • logical and comparison expressions now resolve to bool
  • arithmetic rejects bool operands
  • if, while, and not now validate condition-compatible expressions
  • variable initializers, assignments, and returns now check types more strictly

One of the more annoying bugs turned out not to be “about bools” at all.

When code like this failed inside the compiler:

defined_structs[i].length
emitted_imports[i].path

the real problem was that for i in 0..count was never registering i in the type environment, so indexed access looked broken when really the loop variable itself had no known type. Once loop indices were typed as int, those field-access failures disappeared (yay!!).

The result is a much better place to build from:

  • bool is now semantically distinct in Flower
  • it still lowers cleanly to integer values in generated C
  • bootstrap works again
  • the compiler is stricter without collapsing under its own source

That makes the next step much easier: real string support, instead of trying to build strings on top of a type system that still half-thinks everything is an int.

Replying to @IvyMycelia

0
27
Open comments for this post

5h 26m 53s logged

Devlog: Giving Flower Real Module Support

v1.2.x was the point where Flower’s impors had to stop being “you can import stuff.. and there’s a fake layer of security” to a proper (get it?) module system.

Before this work, imports and aliases mostly behaved like naming conveniences. You could write:

import "math.flo" as math
math.add(1, 2)

but the compiler was still too loose about what that really meant. Alias names were in emitted C symbol names, top-level declarations were not private, and prop existed more as syntax than as a real feature.

That was fine for early bootstrap work, but it was not strong enough for the next stage of the language.

So the main goal of v1.2.x became:

  • make imported aliases act like real namespaces
  • make prop define the public interface of a module
  • make top-level declarations private by default
  • lay groundwork for field visibility rules like hidden and readonly

Module Semantics Instead of Alias Tricks

One of the biggest changes was separating Flower semantics from C lowering.

Previously, imported aliases were drifting toward becoming part of emitted symbol identity. A local alias should just be a source-level name bound to a module, not the backend symbol.

So now each module has its own lowered symbol prefix, and alias lookup happens semantically during compilation. That means:

import "math.flo" as math
math.add(1, 2)

is resolved as “look up exported add in module math.flo,” not “invent a C symbol from the local alias text as math_add().”

That change also forced a cleanup of emitted names so they stopped leaking absolute filesystem paths. Generated symbols now use project-relative module prefixes instead of machine-specific paths.

prop Became Real

Another major part of this was making prop actually do something.

Instead of treating it as a thin wrapper around only functions, the compiler now recognizes prop as a real top-level export marker. That means the module’s public interface is defined by what it explicitly exports, while non-prop declarations remain internal.

This moved Flower closer to the model I wanted all along: explicit modules, explicit public APIs, no heavy object system (OOP), no fake privacy delegated to the C backend.

Dot Access, Pointers, and the Annoying Middle Part

This work also exposed a weakness in the compiler’s dot-access resolution.

Flower source always uses ., while the C backend must decide whether a given access becomes . or ->. That mostly worked already, but nested pointer-heavy field chains exposed cases where the type tracker was not propagating enough information, which caused bad fallback behavior during self-hosting.

So before field visibility could be trusted, I had to harden nested dot-access resolution. Not fun, but necessary. As it turns out, I was missing several AST kinds which caused the logic to fall through. Very tedious work, but it was well-worth it in the end!

Field Flags Groundwork

With module ownership and access resolution in better shape, I added the first field visibility modifiers:

  • hidden
  • readonly
  • frozen

Originally I debated having these as bitwise operations, but I decided it’d be best to stay within the current update’s scope, so sometime in the future I’ll refactor.

Right now, hidden and readonly are the meaningful additions:

  • hidden blocks external field access
  • readonly blocks external assignment

frozen is groundwork for later. It is parsed, stored, and carried through the compiler, but full assignment-once enforcement is still deferred. Unsure how I’ll go about it, but that’s an issue for future me :p

Devlog: Giving Flower Real Module Support

v1.2.x was the point where Flower’s impors had to stop being “you can import stuff.. and there’s a fake layer of security” to a proper (get it?) module system.

Before this work, imports and aliases mostly behaved like naming conveniences. You could write:

import "math.flo" as math
math.add(1, 2)

but the compiler was still too loose about what that really meant. Alias names were in emitted C symbol names, top-level declarations were not private, and prop existed more as syntax than as a real feature.

That was fine for early bootstrap work, but it was not strong enough for the next stage of the language.

So the main goal of v1.2.x became:

  • make imported aliases act like real namespaces
  • make prop define the public interface of a module
  • make top-level declarations private by default
  • lay groundwork for field visibility rules like hidden and readonly

Module Semantics Instead of Alias Tricks

One of the biggest changes was separating Flower semantics from C lowering.

Previously, imported aliases were drifting toward becoming part of emitted symbol identity. A local alias should just be a source-level name bound to a module, not the backend symbol.

So now each module has its own lowered symbol prefix, and alias lookup happens semantically during compilation. That means:

import "math.flo" as math
math.add(1, 2)

is resolved as “look up exported add in module math.flo,” not “invent a C symbol from the local alias text as math_add().”

That change also forced a cleanup of emitted names so they stopped leaking absolute filesystem paths. Generated symbols now use project-relative module prefixes instead of machine-specific paths.

prop Became Real

Another major part of this was making prop actually do something.

Instead of treating it as a thin wrapper around only functions, the compiler now recognizes prop as a real top-level export marker. That means the module’s public interface is defined by what it explicitly exports, while non-prop declarations remain internal.

This moved Flower closer to the model I wanted all along: explicit modules, explicit public APIs, no heavy object system (OOP), no fake privacy delegated to the C backend.

Dot Access, Pointers, and the Annoying Middle Part

This work also exposed a weakness in the compiler’s dot-access resolution.

Flower source always uses ., while the C backend must decide whether a given access becomes . or ->. That mostly worked already, but nested pointer-heavy field chains exposed cases where the type tracker was not propagating enough information, which caused bad fallback behavior during self-hosting.

So before field visibility could be trusted, I had to harden nested dot-access resolution. Not fun, but necessary. As it turns out, I was missing several AST kinds which caused the logic to fall through. Very tedious work, but it was well-worth it in the end!

Field Flags Groundwork

With module ownership and access resolution in better shape, I added the first field visibility modifiers:

  • hidden
  • readonly
  • frozen

Originally I debated having these as bitwise operations, but I decided it’d be best to stay within the current update’s scope, so sometime in the future I’ll refactor.

Right now, hidden and readonly are the meaningful additions:

  • hidden blocks external field access
  • readonly blocks external assignment

frozen is groundwork for later. It is parsed, stored, and carried through the compiler, but full assignment-once enforcement is still deferred. Unsure how I’ll go about it, but that’s an issue for future me :p

Replying to @IvyMycelia

0
11
Open comments for this post

3h 38m 25s logged

Devlog: Replacing Flower’s Old Function Syntax with func name(...): type

This feature started out as what I thought would be a straightforward syntax cleanup.

Flower’s old function syntax looked like this:

int add(a: int, b: int):
    return a + b
end

and exported functions looked like this:

prop int test():
    return 1
end

It worked, technically, but it had started to annoy me more. The biggest issue was that function declarations did not visually match the direction Flower was already heading in, and instead felt much more C-like than I wanted.

prop func create(...): Person
func main(): int

This style says “this is a function” up front, gives a clean place for modifiers like prop, and keeps the return type attached to the signature rather than awkwardly leading it.

The goal was simple:

func name(...): type

instead of:

type name(...):

Simple in theory. not so much when the compiler is written in the language being refactored.

The Problem

Flower’s parser assumes top-level function definitions start with a type. Old parser logic was built around that:

int main():

means:

  1. parse a type,
  2. an identifier,
  3. params,
  4. and a body

That assumption was scattered through a few different places:

  • normal function definitions
  • prop function declarations
  • forward declarations
  • examples and docs
  • the compiler’s own source

Now, in retrospect, this was actually one of the easier changes. It didn’t take a whole lot of planning nor brain power, and I was able to change it pretty quickly. Luckily for me, I’ve built codegen to rely purely on ast values rather than shaping.

The New Shape

The new syntax is:

func add(a: int, b: int): int
    return a + b
end

and exported functions now look like:

prop func test(): int
    return 1
end

This is nicer because:

  • func makes declarations immediately recognizable
  • prop composes more cleanly with func
  • return types now live where people expect them to
  • future syntax has more room to grow without feeling backwards

It also feels more in line with Flower’s broader goals: explicit, readable, and not complex.

The Refactor

Previously, parse_func_def looked for a return type first:

type name(...):

Now it expects:

func name(...): type

So the order changed to:

  1. expect func
  2. parse function name
  3. parse parameter list
  4. expect :
  5. parse return type
  6. parse body until end

That part was conceptually easy.

The more annoying part was updating all the places that assumed if it isn’t a var decl and it looks like a func, it’s a func.

That fallback had to go (Good riddance!).

Instead, top-level parsing needed to become more explicit:

  • prop func → exported function
  • func → normal function
  • forward func → forward declaration

The old one worked, but it relied on vague “anything else is probably a function” rules.

The Bootstrap Mistake

At one point, I made a bootstrap build so I could use a Flower_new binary to compile the refactored compiler. Unfortunately, I had made a mistake in parse_forward: advancing before peek checks.

Feel free to check out the bootstrapper to see why this was an issue ;)

This meant my “new compiler to compile the new compiler” was built from a broken parser.

The workaround was cursed, but it worked:

  1. check out the commit before the refactor
  2. fix the bug
  3. compile a new binary
  4. switch to the refactor branch
  5. use the binary to compile the new version

Since the compiled binary was ignored by Git, this was actually fairly simple.

Not the most elegant bootstrap story, but it got the compiler to, well, compile.

Devlog: Replacing Flower’s Old Function Syntax with func name(...): type

This feature started out as what I thought would be a straightforward syntax cleanup.

Flower’s old function syntax looked like this:

int add(a: int, b: int):
    return a + b
end

and exported functions looked like this:

prop int test():
    return 1
end

It worked, technically, but it had started to annoy me more. The biggest issue was that function declarations did not visually match the direction Flower was already heading in, and instead felt much more C-like than I wanted.

prop func create(...): Person
func main(): int

This style says “this is a function” up front, gives a clean place for modifiers like prop, and keeps the return type attached to the signature rather than awkwardly leading it.

The goal was simple:

func name(...): type

instead of:

type name(...):

Simple in theory. not so much when the compiler is written in the language being refactored.

The Problem

Flower’s parser assumes top-level function definitions start with a type. Old parser logic was built around that:

int main():

means:

  1. parse a type,
  2. an identifier,
  3. params,
  4. and a body

That assumption was scattered through a few different places:

  • normal function definitions
  • prop function declarations
  • forward declarations
  • examples and docs
  • the compiler’s own source

Now, in retrospect, this was actually one of the easier changes. It didn’t take a whole lot of planning nor brain power, and I was able to change it pretty quickly. Luckily for me, I’ve built codegen to rely purely on ast values rather than shaping.

The New Shape

The new syntax is:

func add(a: int, b: int): int
    return a + b
end

and exported functions now look like:

prop func test(): int
    return 1
end

This is nicer because:

  • func makes declarations immediately recognizable
  • prop composes more cleanly with func
  • return types now live where people expect them to
  • future syntax has more room to grow without feeling backwards

It also feels more in line with Flower’s broader goals: explicit, readable, and not complex.

The Refactor

Previously, parse_func_def looked for a return type first:

type name(...):

Now it expects:

func name(...): type

So the order changed to:

  1. expect func
  2. parse function name
  3. parse parameter list
  4. expect :
  5. parse return type
  6. parse body until end

That part was conceptually easy.

The more annoying part was updating all the places that assumed if it isn’t a var decl and it looks like a func, it’s a func.

That fallback had to go (Good riddance!).

Instead, top-level parsing needed to become more explicit:

  • prop func → exported function
  • func → normal function
  • forward func → forward declaration

The old one worked, but it relied on vague “anything else is probably a function” rules.

The Bootstrap Mistake

At one point, I made a bootstrap build so I could use a Flower_new binary to compile the refactored compiler. Unfortunately, I had made a mistake in parse_forward: advancing before peek checks.

Feel free to check out the bootstrapper to see why this was an issue ;)

This meant my “new compiler to compile the new compiler” was built from a broken parser.

The workaround was cursed, but it worked:

  1. check out the commit before the refactor
  2. fix the bug
  3. compile a new binary
  4. switch to the refactor branch
  5. use the binary to compile the new version

Since the compiled binary was ignored by Git, this was actually fairly simple.

Not the most elegant bootstrap story, but it got the compiler to, well, compile.

Replying to @IvyMycelia

2
407
Open comments for this post

8h 15m 31s logged

Devlog: Making Flower’s Type Tracking “Enough” for Stdlib / String Work

The original motivation was simple and — supposedly — quick; I wanted to move on to v1.1.1, “stdlib basics,” especially string utilities. But Flower’s current compiler did not really know what types expressions had. It mostly had pointer_depth on declarations, which was enough to generate * in C types and sometimes guess whether field access should become . or ->.

That worked for very simple cases:

p: @Vector2 = new Vector2
p.x = 5

But it broke down for nested access:

outer.inner.value
outer.inner_ptr.value

The compiler could see the syntax chain, but it did not know whether inner was a value field or pointer field. Since Flower source only uses ., the C backend needs to decide whether each hop becomes . or ->.

That meant stdlib work was blocked. If I want something like:

if strings.equal(string_a, string_b):
    print("same\n")
end

then Flower needs to understand that string_a.length, string_a.data, etc. are real typed fields, not just random syntax to dump into C.

Originally I thought I could just simply create a Symbol Table to get that to work. Unfortunately, that plan soon failed as it introduced a mariad of issues relating to the import system. At the time, this frustrated me because I knew the import system sucked, but I didn’t want to fix it until a later version.

Eventually, I settled on a specific plan which did actually end up changing how imports are handled.. kinda.

The Plan

The design I landed on was to add a typecheck/type-tracking stage between parsing and codegen:

lexer -> parser -> typecheck -> codegen

The parser should stay mostly syntax-only. It builds AST nodes like “dot access,” so I didn’t really change it.

The typecheck pass now collects declarations into a TypeEnv:

struct TypeEnv {
    structs: StructInfo[1024],
    struct_count: int,

    vars: VarInfo[8192],
    var_count: int,

    funcs: FuncInfo[8192],
    func_count: int,

    error_count: int
}

Then it walks expressions and annotates dot access nodes with an access kind:

ACCESS_UNKNOWN: int = 0
ACCESS_DOT:     int = 1
ACCESS_ARROW:   int = 2

So codegen no longer has to fully guess. It can do:

if ast.data._dot_access.access_kind == ACCESS_ARROW:
    fprintf(out, "->%.*s", ...)
else if ast.data._dot_access.access_kind == ACCESS_DOT:
    fprintf(out, ".%.*s", ...)
else:
    // fallback
end

That fallback is important because Flower is self-hosted-ish right now. The compiler is written in Flower, and the compiler’s own source code is full of field access. If the new typechecker is too strict too early, it breaks the compiler while trying to compile the compiler. Very elegant. Very cursed. Just how I like it ;)

Import-Aware Type Tracking

At first, typechecking only saw the main file, so I added a module loading pass. Instead of codegen discovering and parsing imports late, the compiler now loads modules earlier:

load main module
load imports recursively
collect declarations from all modules
typecheck all modules
codegen all modules

That required a new module structure:

struct Module {
    path: @char,
    src: @char,
    tokens: @TokenStream,
    ast: @AST
}

struct ModuleSet {
    modules: Module[128],
    count: int
}

Unfortunately, it didn’t go as smoothly as I wanted. Near the end when I was sure everything would work, I ran into an Idempotency failure due to the new handling of system imports. I.. just manually passed it :p

Devlog: Making Flower’s Type Tracking “Enough” for Stdlib / String Work

The original motivation was simple and — supposedly — quick; I wanted to move on to v1.1.1, “stdlib basics,” especially string utilities. But Flower’s current compiler did not really know what types expressions had. It mostly had pointer_depth on declarations, which was enough to generate * in C types and sometimes guess whether field access should become . or ->.

That worked for very simple cases:

p: @Vector2 = new Vector2
p.x = 5

But it broke down for nested access:

outer.inner.value
outer.inner_ptr.value

The compiler could see the syntax chain, but it did not know whether inner was a value field or pointer field. Since Flower source only uses ., the C backend needs to decide whether each hop becomes . or ->.

That meant stdlib work was blocked. If I want something like:

if strings.equal(string_a, string_b):
    print("same\n")
end

then Flower needs to understand that string_a.length, string_a.data, etc. are real typed fields, not just random syntax to dump into C.

Originally I thought I could just simply create a Symbol Table to get that to work. Unfortunately, that plan soon failed as it introduced a mariad of issues relating to the import system. At the time, this frustrated me because I knew the import system sucked, but I didn’t want to fix it until a later version.

Eventually, I settled on a specific plan which did actually end up changing how imports are handled.. kinda.

The Plan

The design I landed on was to add a typecheck/type-tracking stage between parsing and codegen:

lexer -> parser -> typecheck -> codegen

The parser should stay mostly syntax-only. It builds AST nodes like “dot access,” so I didn’t really change it.

The typecheck pass now collects declarations into a TypeEnv:

struct TypeEnv {
    structs: StructInfo[1024],
    struct_count: int,

    vars: VarInfo[8192],
    var_count: int,

    funcs: FuncInfo[8192],
    func_count: int,

    error_count: int
}

Then it walks expressions and annotates dot access nodes with an access kind:

ACCESS_UNKNOWN: int = 0
ACCESS_DOT:     int = 1
ACCESS_ARROW:   int = 2

So codegen no longer has to fully guess. It can do:

if ast.data._dot_access.access_kind == ACCESS_ARROW:
    fprintf(out, "->%.*s", ...)
else if ast.data._dot_access.access_kind == ACCESS_DOT:
    fprintf(out, ".%.*s", ...)
else:
    // fallback
end

That fallback is important because Flower is self-hosted-ish right now. The compiler is written in Flower, and the compiler’s own source code is full of field access. If the new typechecker is too strict too early, it breaks the compiler while trying to compile the compiler. Very elegant. Very cursed. Just how I like it ;)

Import-Aware Type Tracking

At first, typechecking only saw the main file, so I added a module loading pass. Instead of codegen discovering and parsing imports late, the compiler now loads modules earlier:

load main module
load imports recursively
collect declarations from all modules
typecheck all modules
codegen all modules

That required a new module structure:

struct Module {
    path: @char,
    src: @char,
    tokens: @TokenStream,
    ast: @AST
}

struct ModuleSet {
    modules: Module[128],
    count: int
}

Unfortunately, it didn’t go as smoothly as I wanted. Near the end when I was sure everything would work, I ran into an Idempotency failure due to the new handling of system imports. I.. just manually passed it :p

Replying to @IvyMycelia

1
211

Followers

Loading…