Note to the reviewers/shipwright/fraud squad
so i accidentally added my home folder to the hackatime project and and when i contacted the fraud squad they told me to write a note so can you please deduct around 10 hours please also i attached a demo video of the benchmark script running for the ppl that dont want to download models and set my project up

removed multimodal and fixed the performance issue

so im writing this devlog the second time because stardance logged me out when i tried to post it and it didnt save it sadly so i have to write it again i guess but as the title said i removed the multimodal code basically because it was a pain to maintain and it broke with every update so i just wanted to focus on just closely matching llama.cpp on text speed as a solo developer project and i think i kinda achieved that because in the generation speed its only a few tokens of difference so i think the project is basically finished now and i maybe add a few small changes to it or ship it now but i dont know yet but i hope that someone will like my project even if it was a bit more boring then my other projects

Fixed some bugs and improved the performance in my free time

so i had some time on the weekend and the days i didnt go to school to fix

• added a benchmark shell script to compare the performance against ollama and llama.cpp for testing and for generating performance graphs
• now the vision tensors are run on the gpu too so the model wont get bottle necked by my laptops pci express bus speeds and uses the gpus memory instead
  so this or maybe another one will be probably the last devlog for this project because it has been super hard to develop this and kinda a pain to maintain and try to get it to the performance levels of modern interference libraries and i dont have much free time anymore because of school maybe i will continue development but idk

Working cuda and ci pipeline but sadly i cant work on this project anymore because school started

So working on this project required quite a beefy and heavy computer and i dont want to carry one so sadly i dont have time to continue on this project in this time so after adding a working cuda gpu backend and a ci pipeline so ppl dont have to compile it from the source code and have nvcc to even try i think this is the end of this project for the current time being and i hope that someone will find this
So this is what i added:

• added a cuda backend with a fatbin kernel so it will work on multiple compute levels
• a github actions ci pipeline to generate verified releases for the ppl who dont want to compile everything for themselves

Devlog 2: Finally working multimodal audio and video support

So after a lot of trial and error i finally added multimodal audio and video support because they were the only two things missing from a complete cpu engine because i wanted a fully working cpu reference before doing anything with cuda and stuff but i want to get cuda working asap because the smaller models that i have been testing that would run on my cpu with a good speed they kinda have bad multimodal performance
Thank you for reading my devlog

First devlog for this new project

So i decided to start this new project while waiting for my other project getting reviewed

Why did i do this:

I was kinda bored waiting for my other project to be reviewed by ppl and i wanted to try out zig so i thought this is a great toy project idea to play with in zig

What models are tested (others might work but i havent tested them):

• smollm2-360m
• gemma 4 and its variants with curently only image multimodal working but sound and video might be added in the future

Thank you for reading my devlog