Current time: 12-14-2017, 10:22 AM Hello There, Guest! (LoginRegister)

Poll: Should the dulles testers be moved to EC2?
Yeah, that looks awesome, go for it!
Hard to say until I get more information and see how big the impact will be.
Please don't, I rely on trending the results over time and any change will destroy my use.
[Show Results]
 
Post Reply 
 
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Move Dulles testing to EC2?
10-29-2010, 03:57 AM
Post: #1
Move Dulles testing to EC2?
I'm looking at possibly moving the Dulles test systems to use EC2 instead of running on hardware in my basement Big Grin

I'll do a lot more testing and provide the details of the analysis before committing but I wanted to get a feel for how people would feel about making the move.

Pros:
- The test environment will be consistent for a long period of time (and there's no risk of it changing just because I move)
- It looks like EC2 provides more consistent results when running multiple tests
- It also looks the the times with and without video capture are identical
- (essentially) infinite capacity. More test systems can be brought up as needed
- The public instance will be using test systems that are available for anyone to replicate for private instances

Cons:
- Test results from EC2 are not identical to what is currently running in Dulles so comparison of times to previous tests will not necessarily be valid (this will be a problem if I ever change out the hardware as well)
- The EC2 instances are VM's without hardware acceleration for video so things like IE9's hardware acceleration will not be active (don't think this impacts most "pages" and is more relevant for highly active apps but hard to say)
- The costs (to me) will be higher but probably cheaper than if I put dedicated servers in a co-lo


Initial reactions?
Visit this user's website Find all posts by this user
Quote this message in a reply
10-29-2010, 08:03 AM
Post: #2
RE: Move Dulles testing to EC2?
Go for spot instances... in the case of test servers the state really doesn't matter much...

cdnplanet.com: We help you select the right CDN
Visit this user's website Find all posts by this user
Quote this message in a reply
10-29-2010, 08:08 AM
Post: #3
RE: Move Dulles testing to EC2?
Yep, that's what I've been testing with - the testers are completely stateless so it's a great fit and the pricing is pretty good (5-12 cents/hr)
Visit this user's website Find all posts by this user
Quote this message in a reply
10-29-2010, 08:13 AM
Post: #4
RE: Move Dulles testing to EC2?
btw, the results from my initial testing is available here: http://blog.patrickmeenan.com/2010/10/pe...y-and.html
Visit this user's website Find all posts by this user
Quote this message in a reply
11-03-2010, 04:44 AM
Post: #5
RE: Move Dulles testing to EC2?
This definitely seems like it would be a good move, the only major downside that I see is the increased cost to you personally. I know that you have sponsors for other testing locations, but have you ever considered putting a donation button or something similar on the site? I'm sure there are plenty of people who would contribute money to help defray your hosting costs. Demand seems to be growing for WebPagetest, and the last thing we want is for it to become prohibitively expensive for you to run.

-Jonathan
Visit this user's website Find all posts by this user
Quote this message in a reply
11-03-2010, 05:15 AM
Post: #6
RE: Move Dulles testing to EC2?
Thanks. The ads are doing a good job at offsetting the costs (with some headroom) so I don't think it will be a big deal to cut over. One thing I'm looking at doing is adding support for spinning up instances as needed so some of the less-used configurations stay powered down when they are not being used (and more instances get spun up to handle big bulk requests).

The one I'm having troubles justifying so far would be to move to S3 for archiving the test results. I need to do some modeling to see how expensive it would be but it would be nice to have the raw results archived to S3 where I'll be a little more sure about the reliability (right now it's all on Dreamhost's SAN and hopefully backed up :-) ). Last I checked it was around 700GB of data though. My intent is still to keep all test results for ever but you might see something where authenticated user's test data gets archived to S3 as well for more reliable storage or something like that (or everything but the screen shots gets archived since it's nicely compressible).

Thanks,

-Pat
Visit this user's website Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread: 1 Guest(s)