WebPagetest Forums

Full Version: urlblast crawl mode
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi Patrick,

i have been running urlblast over our portfolio 24/7 and imported the page log into a mysql db.

Now i'm starting to mine the data i've noticed that some of the urls may have been tested 446 times, and some only 311 (where 446 is the highest count), i have noticed when watching the monitor that the urls appear to picked randomly despite me having put them in alphabetical order.

is this a feature or expected behaviour?

Cheers,


Stuart.
They do run randomly but it should be consistent. The behavior is supposed to be that the list is loaded and then the urls are picked off of the list in a random order until all of the urls have been tested and then it repeats. This was done so that when we have testing running across a large number of machines they aren't all hitting the same urls at the same time.

Should be an easy enough configuration change to have them optionally run in order.

I have seen issues where the random selecting seemed to not hit all of the urls but every time I went back to test it through a debugger or with logging it always checked out Undecided
Hi Patrick,

that makes sense.


Cheers,

Stuart.
Reference URL's