Fixed some bugs and improved the performance in my free time
so i had some time on the weekend and the days i didnt go to school to fix
- added a benchmark shell script to compare the performance against ollama and llama.cpp for testing and for generating performance graphs
- now the vision tensors are run on the gpu too so the model wont get bottle necked by my laptops pci express bus speeds and uses the gpus memory instead
so this or maybe another one will be probably the last devlog for this project because it has been super hard to develop this and kinda a pain to maintain and try to get it to the performance levels of modern interference libraries and i dont have much free time anymore because of school maybe i will continue development but idk
Comments 0
No comments yet. Be the first!
Sign in to join the conversation.