WebPagetest Forums
Bot name ? - Printable Version

+- WebPagetest Forums (https://www.webpagetest.org/forums)
+-- Forum: WebPagetest (/forumdisplay.php?fid=7)
+--- Forum: Feature Suggestions (/forumdisplay.php?fid=9)
+--- Thread: Bot name ? (/showthread.php?tid=674)



Bot name ? - piratos - 04-17-2011 12:36 AM

Have you a bot name like w3c_validator or googlebot ?

Some sites must make two redirects to check some ability with javascript and write internal vars in php.

you detect this and follow, but you should do not - that was only for real users.

Bots like

$config['bots']=array('w3c_validator','googlebot','msnbot','slurp','gigabot','scrubby','robozil​a','nutch','ia_archiver','baiduspider','yahoo-mmcrawler','psbot','asterias','yahoo-blogs','SeitenReport');

are disabled automatically for redirects and so it was a good idea to set your name is this method.


RE: Bot name ? - piratos - 04-17-2011 03:46 AM

Ahhh i found it is ptst .


RE: Bot name ? - pmeenan - 04-18-2011 12:34 AM

Yep, sorry, didn't get a chance to reply earlier. I don't recommend special-treating WebPagetest traffic though (other than filtering from your back-end reports) because the whole point IS to measure the user experience and if you send regular users through 2 redirects you should do that for WebPagetest as well.

Thanks,

-Pat