Releases: Andreisgl/Longuinho_Searcher
Releases · Andreisgl/Longuinho_Searcher
First kinda stable release
Crawler is working, but still needs more exception handling and better status reporting for each individual page crawled. The 'multiprocessing' library was used to boost performance and crawling speed went from 1 page/1.5 seconds to an average of 1 page/0.25 seconds.
Crawler is mature enought to collect big ammounts of data.
The indexer is a WIP, but it can already make rudimentary searches and rank pages based on relevance. Speed is still subpar, but greatly improved with 'multiprocessing'.