Current time: 12-13-2017, 09:08 PM Hello There, Guest! (LoginRegister)

Post Reply 
 
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
why does info in "requests.csv" vary?
05-20-2011, 05:35 AM
Post: #1
why does info in "requests.csv" vary?
I recently submitted some tests to WPT through Perlscript (via API calls) for URL=www.cnn.com

First, please consider the "default" case (i.e. Dulles_IE7.DSL, runs=1, fvonly=0). Mostly the "requests.csv" file I download has all the info as expected - e.g. Default_IE7_1. Column C shows that both "Cleared Cache-Run_1" and "Cached-Run_1" tests were conducted - as expected.

However, on 2 occasions, I saw that only "Cached-Run_1" test was done - e.g. see Default_IE7_2 - even though I didn't change any of the parameters.

Question #1: Would you know why this arbitrary behavior occurs?

Now consider a slightly different case from the "default" - the only difference being that instead of IE7, the browser was Chrome (i.e. Dulles_Chrome.DSL). In this case, the "requests.csv" files I receive seem to consistently:

a) Conduct only one of the 2 tests: "Cleared Cache-Run_1" and "Cached-Run_1" (I can't tell which - most probably the former), and

b) Leave the first 3 columns, and a few others, empty

To see what I mean, here are 2 example "requests.csv" files: Default_Chrome_1 and Default_Chrome_2.

Question #2: Would appreciate very much if you could shed some light on why this happens...

Thanks,
Anshul
Find all posts by this user
Quote this message in a reply
05-20-2011, 05:45 AM
Post: #2
RE: why does info in "requests.csv" vary?
#1 - Do you see it regularly? I had some system issues this morning (and one of the two servers is actually not responding right now so I need to go physically kick it). It isn't supposed to run the repeat view test if the first view test fails.

#2 - I'll look closer at the CSV export for Chrome. Looking at the actual tests it looks like all of the data is there for first and repeat view, it's just not making it to the requests.csv. A lot of the fields aren't populated because we basically implemented just enough to get the waterfalls working and haven't had time to go back and fill in some of the other fields yet.
Visit this user's website Find all posts by this user
Quote this message in a reply
05-20-2011, 06:09 AM
Post: #3
RE: why does info in "requests.csv" vary?
First of all: Wow, amazingly fast response Pat - Thank you!

#1 - Actually it has happened only twice so far: at 9am and 9:30am (CST) today. Server problems would probably explain it. Will discard those files as anomalies.

Regarding "It isn't supposed to run the repeat view test if the first view test fails" - so if I see only one test result (as in the Chrome case), is it perhaps an indication that the first view test is repeatedly failing?

#2 - I understand why lot of the fields may not be populated. But the first 3 cols were populated for the IE7 case, so shouldn't they be filled for all browsers? (Unless of course different code is being used for different browsers...)

Would appreciate it if you can fix it such that all the relevant data makes it to the requests.csv files. Would such a fix apply to all the locations around the USA?

Thanks again,
Anshul
Find all posts by this user
Quote this message in a reply
05-21-2011, 04:28 AM
Post: #4
RE: why does info in "requests.csv" vary?
#1 - Chrome is running both, if you go to the actual results page you can see that the data is available for both tests, it's just not making it through in the CSV stage.

#2 - Different code is being used for the Chrome support. It's a new codebase that will support multiple browser and eventually we'll migrate the IE testing to use it as well (the IE code is VERY IE-specific).

The fixes will automatically get deployed globally.
Visit this user's website Find all posts by this user
Quote this message in a reply
05-21-2011, 07:23 AM
Post: #5
RE: why does info in "requests.csv" vary?
#1 - By "actual results page", do you mean this GUI results page? I think the small table at the top of this page is a snapshot summary of the First View and Repeat View runs?

In this post, you said that "the CSV is actually used to draw the UI" so it's strange that the CSV does not have all the data. Perhaps there's another source from which both CSV and GUI draw...

#2 - When might the fixes be deployed? (time frame of days/weeks/months?)

BTW, except for Dulles_Chrome which explicitly states Chrome 11, other locations like Chicago_Chrome, LosAngeles_Chrome, NewYork_Chrome, and SanFrancisco_Chrome do not state the version number (info extracted from this XML file). How to determine the browser version we're dealing with?

Thanks,
Anshul
Find all posts by this user
Quote this message in a reply
05-21-2011, 07:28 AM
Post: #6
RE: why does info in "requests.csv" vary?
#1 Yes, but click on the waterfall for either to go to a page-specific run. The CSV data is actually stored in the filesystem as tab-delimited files with one file for each run (and separate files for first and repeat view). The CSV code just merges the files together into a single file and changes it from tab to comma delimited.

#2 somewhere between days and week (wekk, to get al of the data in there and make enough data columns available to be useful). Full data parity would be months (if ever) but a lot of the fields aren't useful.

All of the Chrome locations are running whatever the current production version is. Chrome is super aggressive about automatically updating itself. I'll see if I can add the full version number to the data.
Visit this user's website Find all posts by this user
Quote this message in a reply
05-27-2011, 03:48 AM
Post: #7
RE: why does info in "requests.csv" vary?
(05-21-2011 07:28 AM)pmeenan Wrote:  #2 somewhere between days and week (wekk, to get al of the data in there and make enough data columns available to be useful). Full data parity would be months (if ever) but a lot of the fields aren't useful.

Could I request that columns BL and BM (Connect Start and Connect End) in particular be included in the CSV file?

Thanks,
Anshul
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread: 1 Guest(s)