Lazy-rendering in PhantomJS

twitter.js

// This example shows how to render pages that perform AJAX calls
// upon page load.
//
// Instead of waiting a fixed amount of time before doing the render,
// we are keeping track of every resource that is loaded.
//
// Once all resources are loaded, we wait a small amount of time
// (resourceWait) in case these resources load other resources.
//
// The page is rendered after a maximum amount of time (maxRenderTime)
// or if no new resources are loaded.

var resourceWait  = 300,
    maxRenderWait = 10000,
    url           = 'https://twitter.com/#!/nodejs';

var page          = require('webpage').create(),
    count         = 0,
    forcedRenderTimeout,
    renderTimeout;

page.viewportSize = { width: 1280, height : 1024 };

function doRender() {
    page.render('twitter.png');
    phantom.exit();
}

page.onResourceRequested = function (req) {
    count += 1;
    console.log('> ' + req.id + ' - ' + req.url);
    clearTimeout(renderTimeout);
};

page.onResourceReceived = function (res) {
    if (!res.stage || res.stage === 'end') {
        count -= 1;
        console.log(res.id + ' ' + res.status + ' - ' + res.url);
        if (count === 0) {
            renderTimeout = setTimeout(doRender, resourceWait);
        }
    }
};

page.open(url, function (status) {
    if (status !== "success") {
        console.log('Unable to load url');
        phantom.exit();
    } else {
        forcedRenderTimeout = setTimeout(function () {
            console.log(count);
            doRender();
        }, maxRenderWait);
    }
});

Thanks for this. It solved a lot of issues for some code I was writing.

How can I set proxy authentication? I tried this: phantomjs --proxy-auth=user:pass myTwitter.js but response is 401.
I have a script for screen capture and I and I was able to authenticate with the same way up.

ex: phantomjs --proxy-auth=user:pass --ignore-ssl-errors=true myScript.js http://urlforexample.com output.png timeout 1400(px) 700(px)

myScript.js

I was able to solve with page.settings.userName and page.settings.password, but this script twitter.js did not solve my problem, was animation in page, the data is loaded via ajax, in my case, phantomjs render the animation.

Thank you so much for share this!

Thanks, it helps!!!

Nice script. A small side note; why do you declare the variable forceRenderTimeout? This one is never used and setTimeout can be called without an assignment to a variable.

It worked great!
Thanks a lot!

This is actually doing nothing:

renderTimeout = setTimeout(doRender, resourceWait);

Increase the resourceWait and set to:

renderTimeout = setTimeout(function () {
                    //console.log(count);
                    doRender();
                }, resourceWait);

Excellent. Thank you!

It works with everything I tried except this:
https://myaccount.nytimes.com/auth/login

I do not get the rendering of the input for username, password and the submit button.
Any ideas why? Really thankful for your code.

Works like a charm .
Thank you .

Thank you! After 5 years, it's still working!

Thank you! Feb-2020 and still good

How can I get through the five seconds wait for cloudflare?
Tried this, but its still getting stuck at the wait for 5 seconds as cloudflare redirects you.

i tried windy.com, but it didn't work