tyse

The basic search engine is complete, I added css and made it look nicer. Still need to polish the backend and add more features.

The backend for the search engine is mostly done, I need to fix some issues with elasticsearch database and make it look good. The current index is being fetched and saved into elasticsearch through searxng as I do not have the resources to run the crawler/fetch the indexes myself, though I did add a prototype crawler.

Currently this is how it works:
User searches a query -> find the best results with the given query through elasticsearch, if it does not exist it fallbacks to searxng from port 8888 -> then display the result to user.

This is so that I could save the search results as continue to use it, I plan on adding some sort of timer or something to update the index for outdated searches.

As seen in the video, there are clearly some issues with overlapping searches.

I added a single threaded basic web crawler for my search engine. I have yet to implement a queue system to act as the Url Frontier, instead it is a single List of String(s) to represent Url’s. This was my first time making a web crawler as well as my first time working with JSoup.