rss-bot: Support unordered RSS feeds #2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rss-bot had 2 different feed_file variables: 1. The user provided file with the list of feed URLs. 2. The file for each feed URL, to store the feed entries' hashes. To clearly differentiate between them, the latter has been renamed to feed_hashes_file.
Previously, it was being set for every entry.
Renamed the OLDNESS_THRESHOLD constant that was being used for the same, to match the name of the newly added option.
We will be using entry time for sorting entries in the following commits.
By splitting the logic into two loops - one for processing all the entries in the feed, and another to post only the latest ones in chronological order. Instead of tracking new_hashes in memory while processing the feed file, we track unhashed_entries now, since we will not be hashing all the entries, only the ones that we post. Fixes #831.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes: #831
I've tested the basic functionality (unordered feeds + max batch size), but I haven't tested every case thoroughly (yet).
Self-review checklist
(variable names, code reuse, readability, etc.).
Communicate decisions, questions, and potential concerns.
Individual commits are ready for review (see commit discipline).
Completed manual review and testing of the following:
Recreated from zulip/python-zulip-api#895