Detection Working!!!

Shelf-fish Devlog #6

TL;DR

What have done?

• Created a basic jwt auth
• Used Gemini Api and Gemini 3.1 Flash Lite to create the product detection logic

What’s next?

• I will now make api routes for detection that would output the brand, item name and category of the item

Got lots of time?

Sooo the auth and detection logic is complete, we are now close to ending the backend, earlier after dropping the plan of using a classification model, I had decided to use an ocr to detect text from the input photo and then use that text as an input for openai api and ask it to identify the brand name, item name and category from the image, but since I am broke , I couldn’t use openai api as it costs money and then learned that gemini has free api plans that include multi-modal inputs like images and give out text, so I swited to that and it works flawlessly, the rate limits are also generous, at 500 requests per day and 15 per minute. I tested it with 6 text images and it gave 100% acc but the sample size is pretty small to say that it is 100% accurate, but it does work.

The detection logic, at this point, is just a function, i need to make api routes for that and add a detection log table, the log table would be useful at providing data about rate limits and is also nice to have if the project get bigger, I also have to make a list of categories out of which gemini would choose the category of the product cuz at this point, it outputs anything it likes as a category