Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Catch urlopen exceptions when retrieving list pages #67

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Catch urlopen exceptions when retrieving list pages #67

wants to merge 1 commit into from

Conversation

isameer
Copy link

@isameer isameer commented Apr 27, 2014

This is to prevent a non-existent URL from being fetched when iterating through results in multipage mode. For example, something like

for torrent in tpb.recent().multipage():
    do something with torrent

will throw an exception since the recent torrent listing has a maximum of 100 pages.

@umazalakain
Copy link
Collaborator

Thanks for bringing up this issue. The main problem with the PR though is that it's in the wrong place. Tha try ... except code should live in the items() function of the Paginated class.

There's also the issue of the try block being too large. A try block should always try to be the most specific possible and the exception should check for specific exceptions too. I think something along the lines of the following code would work:

# In Paginated.items, after the while True
try:
    items = super(Paginated, self).items()
except urllib2.HTTPError as error:
    if error.code == 404:
        raise StopIteration()
# the rest of the code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants