Current time: 10-23-2019, 12:28 PM Hello There, Guest! (LoginRegister)

Post Reply 
 
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Different results using site & API
11-15-2010, 03:16 PM
Post: #1
Different results using site & API
Hi,

I've recently been experimenting setting up scripts to automate tests in the hosted platform and for a particular test got significantly different results between the automated script and manually going to webpagetest.org and running the test. I don't think it is an issue with the script as I only found this with one particular test. The tests are for the same URL, same location, only using first view.

Has anyone else experienced something like this?
Find all posts by this user
Quote this message in a reply
11-15-2010, 11:43 PM
Post: #2
RE: Different results using site & API
Were you running the automated script against WebPagetest or your own instance? Also, do you see it regularly or was it a one-time problem?

The API and the UI both use the exact same code path but it's possible one of the options was set differently in the UI (assuming both were run against WebPagetest for the same location, browser and connectivity).

Thanks,

-Pat
Visit this user's website Find all posts by this user
Quote this message in a reply
11-16-2010, 09:09 AM
Post: #3
RE: Different results using site & API
Hi Pat,

Thanks for the response.

Both tests are using the public Dulles_IE7 instance, but I get the difference no matter which location I use.

If you compare the two results below, as far as I can tell, the settings for each are identical, the only difference I can see is that the first was submitted using the API, second using the web interface.

http://www.webpagetest.org/result/101115...066344ee6/
http://www.webpagetest.org/result/101115...02e33d1e2/

Can you see something that I am missing?

Thanks,
Mark.
Find all posts by this user
Quote this message in a reply
11-16-2010, 09:14 AM
Post: #4
RE: Different results using site & API
The urls are different. The API is missing the &Nights=7. My guess is that you're not url-encoding the url before submitting :-)

Thanks,

-Pat
Visit this user's website Find all posts by this user
Quote this message in a reply
11-17-2010, 10:31 AM
Post: #5
RE: Different results using site & API
Hi Pat,

Thanks for following that up, I have replaced those ampersands now, looks to be working well.

I spent some time yesterday modifying the script we use to allow for the connection speed to be passed in as variable, expecting that if I passed in the same profile as Dulles_IE7 standard, the results would be the same.

Can you see any reason why the two tests down the bottom would return such different results? The first returns ~5secs, the second ~12 seconds. My expectation would be that they would be practically identical.

http://www.webpagetest.org/runtest.php?f...1&fvonly=1

http://www.webpagetest.org/runtest.php?f...1&fvonly=1

Thanks again,
Mark.
Find all posts by this user
Quote this message in a reply
11-17-2010, 11:21 AM
Post: #6
RE: Different results using site & API
Sorry, mistake in the documentation - it's bwDown and bwUp (instead of bwIn and bwOut). I just fixed the docs. It should be identical once it's working because the standard profiles are just pre-configured settings. Everything is effectively a custom profile.

Thanks (and sorry about that),

-Pat
Visit this user's website Find all posts by this user
Quote this message in a reply
11-17-2010, 11:23 AM
Post: #7
RE: Different results using site & API
No apologies necessary, appreciate the help.

You have a great tool here. Having lots of fun playing with it!

Thanks again,
Mark.
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread: 2 Guest(s)