Hey everyone! I really enjoyed developing this personal assistant. While it has its limitations, its cool to have a group of programmers sharing ideas and cool concepts.
This first part will give a small peek into the errors i ran into during development.
Development Environment and Python 3.13
Using Python 3 as the foundation for the backend was the simplest part of the project, but certainly not the easiest. Because Python 3.13 is so new, several critical packages lacked prebuilt wheels. I had to manually check GitHub releases and pull pre-release wheels directly from CI artifacts. I pinned these specific versions to maintain stability. When that became unsustainable, I fell back to Python 3.12, utilizing virtual environment isolation for those specific packages.
FastAPI and SSE Streaming
Streaming LLM tokens over Server-Sent Events (SSE) is not well-documented, and it caused significant friction. The primary challenge was that the LLM returns a standard generator, while FastAPI’s StreamingResponse requires compatibility with async generators; they do not compose cleanly with synchronous LLM clients.
My solution was to wrap the DeepSeek client in an async adapter using asyncio.to_thread(). This ensured the blocking API calls did not stall the event loop, allowing me to yield each token as an SSE-formatted string.
Stability and Error Handling
Under heavy load, I found that DeepSeek could occasionally return 503 errors. To manage this, I implemented an exponential backoff strategy with a maximum of three retries to maintain stability without overloading the service. A major challenge was ensuring that errors were caught before the SSE stream began, preventing the user from receiving a "half-finished" response. Consequently, I only initialize the stream once the connection is confirmed as alive.
Voice Integration and Reliability
I chose to integrate ElevenLabs for the voice component to provide a more human-like touch. However, API-based voice services introduce their own reliability constraints. To ensure the assistant remains functional even when hitting the ElevenLabs free-tier limits, I implemented a fallback mechanism. If the API fails, the system reverts to basic browser-based Text-to-Speech (TTS)—a less polished experience, but necessary for overall system reliability.
Deployment and Connectivity
The backend is currently running locally on my PC and exposed via a Cloudflare Tunnel. I ran into a limitation where every restart of the tunnel generates a new URL. I have had to accept this for now, but I am currently looking for a permanent workaround to ensure a consistent connection point.
Now - if you have read this far, you probably want to know what i enjoyed about this project. Building up this backend architecture was certainly not an easy task. For some people this would have been a complete bore, and would've been a dreaded chore, but I enjoyed the challenge, and learning opportunity of dealing with Python 3.13 challenges. After I ship this project, I will be working more with python backend, potentially with another webapp.
Thanks for reading (:
