Current time: 01-19-2021, 03:28 PM Hello There, Guest! (LoginRegister)

Post Reply 
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
urlblast crawl mode
07-08-2009, 11:30 PM
Post: #2
RE: urlblast crawl mode
They do run randomly but it should be consistent. The behavior is supposed to be that the list is loaded and then the urls are picked off of the list in a random order until all of the urls have been tested and then it repeats. This was done so that when we have testing running across a large number of machines they aren't all hitting the same urls at the same time.

Should be an easy enough configuration change to have them optionally run in order.

I have seen issues where the random selecting seemed to not hit all of the urls but every time I went back to test it through a debugger or with logging it always checked out Undecided
Visit this user's website Find all posts by this user
Quote this message in a reply
Post Reply 

Messages In This Thread
urlblast crawl mode - hsiboy - 07-08-2009, 11:13 PM
RE: urlblast crawl mode - pmeenan - 07-08-2009 11:30 PM
RE: urlblast crawl mode - hsiboy - 07-12-2009, 07:24 AM

Forum Jump:

User(s) browsing this thread: 1 Guest(s)