mari is a desktop companion that has access to both your screen and microphone it can see what you are doing and comment on it when spoken too, it has long access memory and runs locally, the inspiration for this project was made by Just Rayen on youtube, the project he made is called riko, this is essentially a updated version with a different vrm and is local instead of run through a cloud provider
repackaged as a .app and added her to my personal site!!
Added this to a portable .app image now, the python is now a bundle and works out of the box with options to edit for a custom config
completly rewrote server.py to remove old ai code setup that was used to ensure the gpu passthrough would work
Today i removed old code that i used ai to assist with in the beginning the majority of it is now hand written along with using code from the riko project made by JustRayen, the main part made by ai is the gpu passthrough on my server as both vulkan and the other fix for amd cards wasnt working and i hadnt commited any of my new code on github to fix things like the memory bug and visual effects with the hair clipping into clothing
today i fixed the gpu passthrough for an amd gpu, it currently took a long time to respond when using vulkan it now responds a few seconds quicker after first prompt
Updated the try project page
i fixed the ability to interrupt it mid conversation to feel more realistic, she now speaks once every 25 minutes of not being interacted with, she can set timers, check battery and other similar stuff and finally she speaks before the prompt is finishing generating which feels more natural and reduces thinking time from 23s to 0.2s to first token.
today i decided to combine both whisper and kokoro to have a more accurate voice model that i can run locally, i also added a new design inside of vroid studio that will be added later on :)
Fixed idle animations to be less snappy and more human like
This is a demo of marina introducing itself to, its own tiktok account there will be more updates shortly!
today i typed up, the requirements file to have all of this running, tommorow i will be setting up the api link to my server. As well as a selectable menu to choose local or api hosted AI
the ui for the app
