Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Navigation Timeout Exceeded with waitUntil: networkidle* #230

Closed
cascornelissen opened this issue Aug 1, 2018 · 13 comments
Closed

Navigation Timeout Exceeded with waitUntil: networkidle* #230

cascornelissen opened this issue Aug 1, 2018 · 13 comments
Labels

Comments

@cascornelissen
Copy link

cascornelissen commented Aug 1, 2018

I've been pulling my hair out on this for a couple of hours now and I'm hoping someone here knows what the problem might be in this case.

I'm running into the following error when executing react-snap on our Jenkins environment, which runs Debian 8 (it runs correctly on my local macOS High Sierra machine)...

🔥  / Error: Navigation Timeout Exceeded: 30000ms exceeded
    at Promise.then (/home/jenkins/jobs/workspace/Freshheads Website/build/node_modules/puppeteer/lib/NavigatorWatcher.js:73:21)
    at <anonymous>

I've tried playing with pretty much every setting available in react-snap but I couldn't get it to work so I started debugging. Turns out changing waitUntil on this line to either load or domcontentloaded resolves my issue.

The strangest thing is that networkidle* should work correctly in my mind: I've enabled logging for Puppeteer by passing DEBUG=puppeteer:session to the command that starts react-snap (output is available right here: log.txt) and it just hangs for about 20 - 25 seconds when the entire output is printed which should mean there is no network traffic going on, right?

My question is; would it be possible to add an option for the waitUntil variable to react-snap or doesn't that work/output the correct HTML?

@stereobooster
Copy link
Owner

Turns out changing waitUntil on this line to either load or domcontentloaded resolves my issue.

This doesn't solve issue in general it simply masks the problem, though it can work in your case. Issue with timeout is known #131. It was reported in puppeteer, but it still reported by users from time to time

@cascornelissen
Copy link
Author

@stereobooster, thanks for the quick reply! I noticed #131 but since the advice over there is to use 1.10 while the current version is 1.17.0 already I was expecting that to be solved.

I'll try downgrading to 1.10 tomorrow/friday, if that doesn't fix it I'll try and debug some more based on puppeteer/puppeteer#1908 (comment).

In the end the load value seems to work correctly for me. Would you accept a PR that adds a waitUntil option?

@stereobooster
Copy link
Owner

You can use this one https://github.com/peterbe/minimalcss/pull/199/files

@cascornelissen
Copy link
Author

@stereobooster, I've done some more testing using the tracker you've linked to in the previous comment.

I've modified it a little bit to make it easier to debug my case:

const createTracker = page => {
    const requests = new Set();
    const onStarted = request => {
        console.log('started', request._url);
        return requests.add(request);
    };
    const onFinished = request => {
        console.log('finished', request._url);
        return requests.delete(request);
    };
    const onFailed = request => {
        console.log('FAILED', request._url);
        return requests.delete(request);
    };
    page.on('request', onStarted);
    page.on('requestfinished', onFinished);
    page.on('requestfailed', onFailed);
    return {
        urls: () => Array.from(requests).map(r => r.url()),
        dispose: () => {
            console.log('requests', requests);

            page.removeListener('request', onStarted);
            page.removeListener('requestfinished', onFinished);
            page.removeListener('requestfailed', onFailed);
        }
    };
};

Which outputs the following: every request starts and finishes (in about 3s) and not a single one fails and I'm still running into the 30s timeout (the last 27s it just hangs).

started http://localhost:45678/
finished http://localhost:45678/
started http://localhost:45678/app.4be508b9885babb8ef75.css
started http://localhost:45678/homerun.5e04a4a0f278c718e2bb.css
started http://localhost:45678/lazysizes.3af867507f1aa122bd24.js
started http://localhost:45678/app.3af867507f1aa122bd24.js
started http://localhost:45678/homerun.3af867507f1aa122bd24.js
finished http://localhost:45678/app.4be508b9885babb8ef75.css
finished http://localhost:45678/lazysizes.3af867507f1aa122bd24.js
finished http://localhost:45678/homerun.3af867507f1aa122bd24.js
finished http://localhost:45678/homerun.5e04a4a0f278c718e2bb.css
finished http://localhost:45678/app.3af867507f1aa122bd24.js
started http://localhost:45678/vendors-home-view.02d944751c4d70716f2a.bundle.js
started http://localhost:45678/home-view.3a19da2afcb08790bdeb.bundle.js
started http://localhost:45678/fonts/neutrif-studio-regular.woff2
started http://localhost:45678/fonts/proxima-nova-extra-bold.woff2
finished http://localhost:45678/home-view.3a19da2afcb08790bdeb.bundle.js
finished http://localhost:45678/fonts/neutrif-studio-regular.woff2
finished http://localhost:45678/fonts/proxima-nova-extra-bold.woff2
finished http://localhost:45678/vendors-home-view.02d944751c4d70716f2a.bundle.js
started http://localhost:45678/fonts/proxima-nova-extra-bold.typeface.json
started http://localhost:45678/fonts/neutrif-studio-regular.typeface.json
finished http://localhost:45678/fonts/proxima-nova-extra-bold.typeface.json
started http://localhost:45678/images/cases/roadguard/blob.jpg
started http://localhost:45678/images/cases/interpolis/blob.jpg
started http://localhost:45678/images/cases/quiet/blob.jpg
started http://localhost:45678/images/cases/viia/blob.jpg
started http://localhost:45678/images/cases/ggz/blob.jpg
finished http://localhost:45678/fonts/neutrif-studio-regular.typeface.json
finished http://localhost:45678/images/cases/quiet/blob.jpg
finished http://localhost:45678/images/cases/ggz/blob.jpg
finished http://localhost:45678/images/cases/viia/blob.jpg
finished http://localhost:45678/images/cases/roadguard/blob.jpg
finished http://localhost:45678/images/cases/interpolis/blob.jpg

You have been evading my question about an option for the waitUntil value though, is that something you're not interested in?

@cascornelissen
Copy link
Author

Oh, and @stereobooster, FYI: locking react-snap at 1.10.0 does resolve my problem. I'd love to help you get this fixed/improved in an upcoming release. Let me know if there's anything else I can provide to resolve the issue.

The question about the waitUntil option still stands, though.

@stereobooster
Copy link
Owner

FYI: locking react-snap at 1.10.0 does resolve my problem.

Can you also say which version of puppeteer do you use (based on package.lock.json or yarn.lock)

@cascornelissen
Copy link
Author

└─┬ [email protected]
  ├─┬ [email protected]
  │ └── [email protected]
  └── [email protected]

@stereobooster
Copy link
Owner

This is the thing, in puppeteer v1.0 something is broken. Issue was reported, but closed. We need to reopen the issue and provide reproducible example

@cascornelissen
Copy link
Author

Do you have any issue numbers? I might be able to build a simple docker image showing the problem if the developers over at puppeteer are interested.

@stereobooster
Copy link
Owner

This is the issue puppeteer/puppeteer#1908 (comment)

@stereobooster
Copy link
Owner

I have another idea, if we will add tracker (to check that all requests are finished) we can omit false positive Timeout Errors, downside of this approach is that prerendering will take longer time.

@stereobooster
Copy link
Owner

@stereobooster
Copy link
Owner

Closed in favour of #240

You have been evading my question about an option for the waitUntil value though, is that something you're not interested in?

The problem here, this will mask issue but not solve it. By adding waitUntil I believe some people will start to use without realising, there is something broken down the line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants