Ok i decided to make my own bios because the bigcomparations feed you trash, ok back to the code what idid Power-on/reset entry point - reset cpu vector(OxFFFF0) that jumps into real BIOS code, POST-CPUsanity checks nemory detection/sizing basicchipset/southbridge initialization dache and MTRR setup
ok so let me defend originality : it is a ring 0 os made for hardvare inspection while i saw others did os stuff to before me on stardance i already made an os before stardance in arm , technicality : assembly ,usability : it is meant to inspect pcs when something goes wrong hardvare level not daily driver, you decide storitelyng i pored my heart into those devlogs
Ok so refined devlogs polished sofaos and GitHub couldn’t ship because rhe whole kernel inflated from 52 to 838 sectors by some miracle, destroyed interrupt table, tensor hard coded stuff and other stuff it ruined my day
also ok so let me defend originality : it is a ring 0 os made for hardvare inspection while i saw others did os stuff to before me on stardance i already made an os before stardance in arm , technicality : assembly ,usability : it is meant to inspect pcs when something goes wrong hardvare level not daily driver, you decide storitelyng i pored my heart into those devlogs
So today i finished argmax which finds closest token to response, now i have everything but it still has some errors it prints 0 every time so meybie some register colision but the good thing is that it doesn’t crash.
Ok i have successfully made detokenizator which goes trough the token id book that many times as a token because token id =id book entries untill the token, and also i have made matrix–vector dot product calculator which goes through all token ids and translates them into a big amount of vectors which then in another step i will be doing comparison hbuf with all these vectors to find a token with lovest diff and that will be our output token, but big but it still isn’t tested i will be testing it when i will have the full command done for now i just want to live in an illusion that my code is error free
There was a problem with pointers in rnsm found it when i did double check and from there errors just piled up but at least its fixed now
Today i got help from my great teacher and was able to do more than usual, today i did hbuf setuper it basically just calls fcon 576 times to translate the values from 16 bit quantization to 32 then i have written a function RMSNorm which uses hbuf and weights you give it and calculates rescaled feuture vector
I have no idea how to fix it so i retreated ro the ol reliable method by first hardcoding it and after some time actually fixing it
Today i am a bit better so i did the harder part that is needed to be done, i have made tensor info gatherer, basically each layer has 9 tensor info, so for my model 9*30 tensor infos, also the string i am comparing in the function needs to be like [blk.][number of current layer in decimal(use bintrn)][specification string], it still doesn’t work but it will be something like infinite inc loop to memory or division basically smth that generates general protection, also because I was smart i saved my previous error where i used tensor info praser so i didn’t Neded to reerite it.
Ok so uuh those 5l of Kofola wasn’t a great ide today i got a fever or flu or whatever the thing that makes you feel hot but i made some progress even though it was hard cuz i had a headache and couldn’t do anything so what i did i made a f16 to 32 translation and you might say hmm but this doesn’t seem too vomplex first of all i had to split registers inro 3 sections do calculation on them and the glue them together to xmm0 but what you might not know is if you don’t allow xmm is 32 bit mode it will throw gp an the whole code and you cant know why it crashed because whole runtime in crash log looks absolutely normal for some reason also I don’t actually care about the time anymore i just want to get this working so i can get a laptop to not code on my phone
Ok bad News today i drank 5l of Kofola and now me attention me hyperactive me cant concentrate but did token and bos token enumetion there was a problem with division because of no xor rdx,rdx but ez fix if boestoken add 1 it adds tokwn
Made the metadata praser work i was using tensor format and nkt metadata Format also verifyed it works on pc
Tokenizer done now for vector context calculator . The tokenizer was somwhat refusing to work but one rewrite later it subdued to my power it is split into teo parts the text decoder and encoder decoder formats the words for encoder and encoder finds tokens for the words
Ok i gave lost the war against the matadata praser i will not fight it now i will fix it in second ship for now i have hard coded the data needed and also made the core of the tokenizer which was relatively easy compared to the praser
Ok all metadata entry data prased but when i delete the print at the bottom it for some reason triple faults going to report every ±9 hrs because i need to speedrun this because the stardance is ending soon now onto tensor data praser
Ok bigg devlog todat i decided to speedrun this so i can ship this project as fast as possible also changed the model to a smaller native f16 quantization and It works which is a proof that anyone can put their model into sofaos and run it rn it just loops troug metadata to find general architecture
Hello good news today have started on setting up the data from GGUF rn it is just getting version metadata entryes and tensor entry, I Also started a loop for the metadata entryes but just started this one was easy cuz it has prefixed everything 4 bytes gguf magic 4 version 8 metadata entryes and 8 tensor info entry which makes 24 bytes in total
I have successfully loaded whole LLM problem was that It was loading 1 gb by 512 bytes and that takes very long and thats why it seemed it hanged I fixed it by giving it a counter so you can see that it doesn’t hang
Ok sooo I made the code buuut it hangs when I do the loading idk what happening will have to debug
