WebPagetest Forums
Benchmarks - Printable Version

+- WebPagetest Forums (https://www.webpagetest.org/forums)
+-- Forum: WebPagetest (/forumdisplay.php?fid=7)
+--- Forum: Private Instances (/forumdisplay.php?fid=12)
+--- Thread: Benchmarks (/showthread.php?tid=12838)

Benchmarks - ymanzanilla - 03-01-2014 01:08 AM

Greetings everyone,

I'm currently running a private instance, and exploring benchmarks. Going into /benchmarks/ brings up the docTime graphs of the benchmarks that I have running. when I click on a data point two options are available:
  • Scatter Plot
  • Comparison Distribution

Scatter Plot works great and it offers a ton of valuable information. Comparison distribution is a different story though. It leads me to /benchmarks/delta.php. There is supposed to be a graph on this page, which does not show. Digging through the code I end up in /benchmarks/data.inc.php, on function LoadDelta. In the forEach loop found in this function, there are two lines of code that read like this:

PHP Code:
foreach ($median_data[$config][$loc][$cached] as $url => &$data) {
                if (
array_key_exists($url$median_data[$ref][$refLoc][$cached])) { 

This piece of code is failing because it tries to find the url from config::loc::cached in ref::refloc::cached but the function loadMedianData never creates such structure. LoadMedianData creates an associative array containing the parsed results of the benchmark. Let me provide an example:

Lets say we run a benchmark with two configurations: 'SiteA' and 'SiteB'. Site A is running from location 'Office_IE9.Cable' and it has 3 urls: www.mysitea.com, www.mysitea.com/my_video_page, and www.mysitea.com/some_page. Site B is running from 'Office_wptdriver:Chrome.3G' and it has also 3 urls: www.mysiteb.com, www.mysiteb.com/my_profile_page, and www.mysiteb.com/some_lost_page .

After succesful completion of the benchmark, function LoadMedianData retuns an array that looks something like this:

PHP Code:
$median_data['SiteA']['Office_IE9.Cable'][0]['http://www.misitea.com/'] = associative_results_array()
$median_data['SiteA']['Office_IE9.Cable'][0]['http://www.misitea.com/my_video_page'] = associative_results_array()
$median_data['SiteA']['Office_IE9.Cable'][0]['http://www.misitea.com/some_page'] = associative_results_array()
$median_data['SiteB']['Office_wptdriver:Chrome.3G'][0]['http://www.misiteb.com/'] = associative_results_array()
$median_data['SiteB']['Office_wptdriver:Chrome.3G'][0]['http://www.misiteb.com/my_profile_page'] = associative_results_array()
$median_data['SiteB']['Office_wptdriver:Chrome.3G'][0]['http://www.misiteb.com/some_lost_page'] = associative_results_array() 

Following this example, when execution arrives to:

PHP Code:
foreach ($median_data[$config][$loc][$cached] as $url => &$data) {
                if (
array_key_exists($url$median_data[$ref][$refLoc][$cached])) { 

It tries to run something like this:

PHP Code:
foreach ($median_data['SiteA']['Office_IE9.Cable'][0] as $url => &$data) {
                if (
array_key_exists('http://www.misitea.com/'$median_data['SiteB']['Office_wptdriver:Chrome.3G'][0])) { 

This statement fails all the time, and the function always retuns NULL, thus no graph :-(

My questions are: What is the purpose of delta.php? I mean, if I know what is supposed to do then I can 'fix' it. I'm I missing something in the setup of my benchmarks? No mean to insult, but is the code possibly faulty?

Thank you in advance for your answers.


RE: Benchmarks - pmeenan - 03-05-2014 03:09 AM

Sorry, you're wandering into stuff that hasn't been cleaned up for general consumption yet. The way the benchmarks are set up currently, it basically expects the same set of URLs across various configurations and lets you compare the configurations against each other (which is why you're having issues with delta.php).

Think "mod_pagespeed accelerated vs Not Accelerated" or "Chrome Stable vs Chrome Canary". I haven't had a chance to focus on also having it support general performance trending and things more like "Us vs Competitor".