Correct location/browser/connectivity settings for mobile test triggered via API?

[zacklerner]

I've been using a fork of Eduardo Bouças' SpeedTracker tool to collect historical performance data on sites our team has in development, but recently I've run into trouble when attempt to run tests with mobile settings. I'm using the webpagetest command line tool in an attempt to determine the correct Location:Browser.Connectivity settings, and have posted some of my attempts below.

Here's an example of a test that runs successfully, but gives me an error on the WPT results page:

Code:

$ webpagetest test barrelny.com --key xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx --location Dulles:Dulles_iPhone5c.LTE
{
  "statusCode": 200,
  "statusText": "Ok",
  "data": {
    "testId": "161201_HP_65CJ",
    "ownerKey": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
    "jsonUrl": "http://www.webpagetest.org/jsonResult.php?test=161201_HP_65CJ",
    "xmlUrl": "http://www.webpagetest.org/xmlResult/161201_HP_65CJ/",
    "userUrl": "http://www.webpagetest.org/result/161201_HP_65CJ/",
    "summaryCSV": "http://www.webpagetest.org/result/161201_HP_65CJ/page_data.csv",
    "detailCSV": "http://www.webpagetest.org/result/161201_HP_65CJ/requests.csv"
  }
}

Here's the error on the results page:

Quote:First View: Test Error: Invalid Browser Selected: Dulles_iPhone5c

It's clear to me I'm using the incorrect ID for the browser here, but I've been unable to determine what the settings should be. I was able to find "Dulles_iPhone5c" in the getLocations.php list, but setting that value as the location returns the following:

Code:

$ webpagetest test barrelny.com --key xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx --location Dulles_iPhone5c
{
  "statusCode": 400,
  "statusText": "Invalid Location, please try submitting your test request again."
}

Are there specific settings that would allow us to run an LTE or custom throttled test on an iOS browser? Are theses browsers/locations not available to the API? Would we need to set up a private instance attached to a mobile device?

Thanks for taking a look!

[vincent]

The json result from getLocations.php still lists the browser names.

getLocations.php?f=json

[zacklerner]

Thanks Vincent. The json data will definitely be useful, but it looks like many of the locations in those results aren't recognized by the API as valid:

Code:

$ webpagetest test barrelny.com --key xxxxxxxxxxxxxxxxxxxxxxxx --location Dulles_Nexus7
{
  "statusCode": 400,
  "statusText": "Invalid Location, please try submitting your test request again."
}

$ webpagetest test barrelny.com --key xxxxxxxxxxxxxxxxxxxxxxxx --location Dulles_IE8
{
  "statusCode": 400,
  "statusText": "Invalid Location, please try submitting your test request again."
}

$ webpagetest test barrelny.com --key xxxxxxxxxxxxxxxxxxxxxxxx --location Dulles_GalaxyS5
{
  "statusCode": 400,
  "statusText": "Invalid Location, please try submitting your test request again."
}

$ webpagetest test barrelny.com --key xxxxxxxxxxxxxxxxxxxxxxxx --location Dulles_iPhone6
{
  "statusCode": 400,
  "statusText": "Invalid Location, please try submitting your test request again."
}

$ webpagetest test barrelny.com --key xxxxxxxxxxxxxxxxxxxxxxxx --location Dulles_iPhone5c
{
  "statusCode": 400,
  "statusText": "Invalid Location, please try submitting your test request again."
}

I dug a little and was able to find an inactive thread referencing the same issue: http://www.webpagetest.org/forums/showth...?tid=13664

Does anyone have any additional info on the disconnect between what the getLocations page displays and what the API recognizes as a valid location?

[vincent]

Not sure how the speedtracker tool works, but going by your example, if you want iphone5c and lte, and you want to use this location:

"Dulles_iPhone5c": {
"Label": "Apple iPhone 5c (iOS 9)",
"location": "Dulles_iPhone5c",
"Browsers": "iPhone 5c iOS 9,iPhone 5c iOS 9",
"status": "OK",
"relayServer": null,
"relayLocation": null,
"labelShort": "Dulles, VA",
"group": "Mobile Devices - Dulles, VA",
"PendingTests": {
"p1": 0,
"p2": 0,
"p3": 0,
"p4": 0,
"p5": 0,
"p6": 0,
"p7": 0,
"p8": 0,
"p9": 0,
"Total": 0,
"HighPriority": 0,
"LowPriority": 0,
"Testing": 0,
"Idle": 1
}
}


location:browser.speed

should be:

Dulles_iPhone5c:iPhone 5c iOS 9.LTE

[zacklerner]

That's what I would've thought myself, Vincent. I've actually been testing this outside of SpeedTracker, using the OSX command line tool and Marcel Duran's API console. The setting you mentioned for location:browser.speed is still throwing an error:

Code:

webpagetest test barrelny.com --key xxxxxxxxxxxxxxx --location Dulles_iPhone5c:iPhone 5c iOS 9.LTE
{
  "statusCode": 400,
  "statusText": "Invalid Location, please try submitting your test request again."
}

Have you been able to successfully use that location by any chance?

Instead of using this technique I've started using the "mobile" parameter to have Chrome emulate a mobile browser. Not entirely sure what I'm losing by using this method, but it does seem to be giving us fairly complete test results (the mobile testing agents seem to have some trouble recording for the film strip view).

[vincent]

Code:

http://www.webpagetest.org/runtest.php?url=http://www.google.com&f=json&fvonly=1&private=1&runs=9&video=1&location=Dulles_iPhone5c:iPhone 5c iOS 9.LTE&k=[key]

This worked for me.

Results: https://www.webpagetest.org/results.php?...1646301593

[zacklerner]

Just tried the same thing, substituting my key, and got this:

Code:

{"statusCode":400,"statusText":"Invalid Location, please try submitting your test request again."}

Did you just paste the URL in the address bar of your browser? If so, which browser do you happen to be using? If not, how did you trigger the test?

[vincent]

I think for the results I posted, I just pasted it via browser (Chrome), but I've also got a powershell script calling it for a daily test.

[zacklerner]

Thanks Vincent! Both myself and a colleague tried it without any luck. Going to let it rest for the time being since the mobile emulation through the Dulles Chrome testing agent has been working well for us.

[midimid]

I get the exact same error with the exact same settings. Is there a new or different way of doing this that we are not understanding? Would definitely prefer the device over the chrome emulation.