Possible changes to grading

[pmeenan]

Given W3TC, Supercache and a lot of other options your typical wordpress user has a lot of tools at their disposal, maybe with some questions but even just getting the basics set up can be a huge win :-) Given that NOTHING can load until the first byte comes back it's absolutely critical - the key is figuring out where to set the thresholds and still be reasonable. At the end of the day, it doesn't matter if it's database or back-end API calls, if it's slow it's slow.

[jklein]

Totally agree on first byte time - it's so critical that if it's not under your control you should go change that to make sure that it is :-). It would definitely be nice to have a visual indication if you are having TTFB issues.

[pmeenan]

The CDN change is now live and I'm working on the first byte logic (at which point the JS/CSS will be demoted). Right now I'm thinking that the grading for the first byte will look something like this:

Base time = 3 * rtt to server (allow for DNS + socket connect + http request)
If TTFB <= base + 100ms then it get's an A
and then deduct a grade for every 100ms beyond that

SSL would also be accounted for and the extra round trips would be factored in for a https request.

That way the connection latency will not be a factor and it will purely be a measurement of back-end processing time (<= 100ms = really good). It will also penalize for redirects.

The end result for the default DSL connection profile would be:

TTFB <= 250ms = A
250ms - 350ms = B
350ms - 450ms = C
450ms - 550ms = D
> 550ms = F

The actual times will be slightly more generous because these times don't include the additional latency to get to the actual server which will be included in the actual grading (using the socket connect time).

[pmeenan]

ok, the changes are live (and retroactive to existing tests). Feedback appreciated.

[Kye]

Boo I have another B to fix now >.<

Which is annoying as my CMS records the time it takes to generate a page, and that records them at 70ms. Now I need to find out where the other 140ms are coming from.

Least theres no more big red F staring me in the face!

[pmeenan]

@Kye, do you have a test result? I just want to make sure the math on my side is correct since it's pretty new.

[Kye]

http://www.webpagetest.org/result/110628_5K_Y5JR/ -B

http://www.webpagetest.org/result/110628_JQ_Y6BH/ -D

http://www.webpagetest.org/result/110628_2D_Y6HZ/ -B

Trouble with FBT is it varies a log with each and every request.

[LeptienM]

(06-24-2011 02:34 AM)LeptienM Wrote:  Two things I would like to add Re: JS/CSS:

First, combining these two, when they are up there in the rendering path, is a huge, and sometimes simple, gain. Our former site would probably very well serve as "Worst practice example", like the "CNN in JS/CSS concatenation". We had, when this portal moved under my responsibility, something like 30 CSS'es and 15 JS'es in the HEAD. And the impact IS huge. So I still would see it as main grade.

Second, we have different behaviour on at least IE7 and IE8, regarding JS in the body of the basepage included via Script-Tag. I wrote something about my observence in my blog. The point is: I get different grades on this one if I test the page with IE7 compared to when tested with IE8. And that is somehow confusing. (IE8 engine seems to look ahead and pull the download of JS external Scripts in the body ahead, which results in a worse grade)

I am a little bit hesitating, taking it of the main grades (As important as I think this still is), because a valid algorithm for this beast to evaluate from the top of our heads doesn't seem too obvious yet.

And one more thing to add:
See Slides 26-31 and 59 on this deck:
https://docs.google.com/present/view?id=...52c9g976c4

So I still think, a grade on Concatenation of JS/CSS in the HEAD would be relevant. :-)

[pmeenan]

Can you make the doc public for reading (or share the key points on the slides)?

I'm not arguing that it's not important, just that it's not universally true that combining resources will be faster. At the end of the day I hope that people pay more attention to the actual waterfalls than the grades themselves.

[LeptienM]

(09-17-2011 12:48 AM)pmeenan Wrote:  Can you make the doc public for reading (or share the key points on the slides)?

I'm not arguing that it's not important, just that it's not universally true that combining resources will be faster. At the end of the day I hope that people pay more attention to the actual waterfalls than the grades themselves.

Whoops... This Link should work. http://bit.ly/hKXBKB
Got it from Souders Twitter Stream.

Basically this guy has tested an artificial test page:
"A single 21 KB HTML document
26 image files totaling 1,789 KB
Five Javascript files totaling 554 KB
Eight style sheets totaling 43 KB
Total page weight of 2,407 KB
In other words, results not typical"
and then has measured the impact of 14 of the most common best practices, each on its own. Each practice was then put on a coordinate system with engineering effort / performance gain being the axis. And Concatenating JS and CSS pretty much got the biggest bang for your bucks. In this scenario. He didn't write, though, what kind of network he tested this on.

Best wishes,
Markus