The web app (React + HeroUI) — In progress

• A modern admin-style dashboard: sidebar nav, top bar, stat cards.
• Pages for managing medications, scanning, viewing the dashboard, and editing your profile.
• Scan history table where you can review past scans, edit the extracted drugs, and re-run the interaction check.
• Camera + upload - take a photo live in the browser or upload one.
• Made mobile-friendly (the sidebar collapses into a slide-out drawer on phones).

The hard part: reading the label (OCR)

This is where most of the recent effort went, and it’s been a real journey:

• Started with Tesseract - it choked on real photos (returned garbage like “ry”).
• Switched to PaddleOCR, then to its best models (PP-OCRv5 server detection + recognition, plus auto-rotation and image-unwarping). This reads real strips far better.
• Improved the matching: combine adjacent words so multi-word salts (“lactic acid”) stay together, skip packaging words (tablet, mg), and tolerate OCR typos with fuzzy matching.

Where it stands

The full chain works end to end: scan a medicine -> read the salts -> check interactions -> see ranked warnings, with accounts, history, and a polished responsive UI.