I focused on improving the entire OCR system, as the previous approach with EasyOCR was missing too much text and reducing the accuracy of the entire system. After spending quite a bit of time trying to improve it, I decided to replace it with PaddleOCR. Now more accurate text detections are being produced making the entire program more robust.
After, I expanded the regex based detectors to also account for URLs in addition to emails, etc. I also improved the phone number detection logic to reduce false positives and improve overall accuracy. Then, I adjusted padding around blurred regions so that stuff is covered more consistently and without blurring unnecessary content.
The last thing I did was optimizing PaddleOCR as it was much slower than EasyOCR. After a lot of tinkering I was able to significantly reduce its processing time. For some files it now performs at roughly the same speed as EasyOCR while being way more accurate.
Next, I probably want to add support for more file types and ultimately videos. And later also support for multiple languages.