Current time: 12-18-2017, 10:48 PM Hello There, Guest! (LoginRegister)

Post Reply 
 
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
WebPagetest on EC2
11-25-2010, 10:19 AM
Post: #21
RE: WebPagetest on EC2
(11-25-2010 10:01 AM)georgeh Wrote:  I've tried to fire up the EC2 images mentioned, but I can't seem to get anything out of them.

Q1. What, exactly, are they intented to do?

Q2. How are they meant to be controlled?
- is there an Administrator password?
- is there a web-interface on some port?

Do you have a web server configured with the WebPagetest code on it? The EC2 instances are just test agents for running tests and they need to connect to a web server for getting work and posting results.

You configure it using the user data string when you start up the instance (the web server and location are critical, otherwise it won't do anything).

We're planning on creating an EBS-backed image of the web server as well at some point but for right now all we have set up are images of the testers (so you can spawn the test agents up as needed and wherever needed).

PM me if you still need the administrator password to RDP in and poke around.

Thanks,

-Pat
Visit this user's website Find all posts by this user
Quote this message in a reply
11-25-2010, 11:10 AM
Post: #22
RE: WebPagetest on EC2
(11-25-2010 10:19 AM)pmeenan Wrote:  Do you have a web server configured with the WebPagetest code on it? The EC2 instances are just test agents for running tests and they need to connect to a web server for getting work and posting results.

OK, I think I'm getting confused about the architecture here. Correct me if I'm wrong, but there seem to be 3 kinds of servers here:
a) The Windows+IE 'agents' which run the actual tests
b) A WebPageTest 'web server' which you can POST URL's to for queuing and testing using (a)
c) A WebPageTestMonitor server to schedual URL's to be posted to (b)

In which case, I would ask, how does (b) launch a test on (a) and collect the results?
I could not detect a webserver on (a). What ports are used (for the benefit of firewalling)?
Find all posts by this user
Quote this message in a reply
11-25-2010, 11:31 AM
Post: #23
RE: WebPagetest on EC2
Pretty close. Are you talking about a WPT Monitor configuation (automated testing) or just a straight-up WebPagetest instance?

There are 2 parts to the system:

a) The windows+IE 'agents' which run the actual tests (and these are what the EC2 images are)
b) The WebPagetest 'web server' which houses the WebPagetest UI, results, test queues, etc (and has an automation API that can be used by WPT Monitor)

If you're running a monitor configuration then that can be run on the web server as well and it just posts the tests and gathers the results through WebPagetest's REST API.

For launching tests the 'agents' (a) poll the web server looking for work. If the web server has a job for them to do it responds with the job and then when the agent is done it posts the results back. All communications are originated from the agents to the web server (I even have some running behind NATs).

The only connection needed for communications is to port 80 on the web server. If you're concerned about rogue agents connecting to the agent queues you can assign secret keys to each location so only testers configured with the correct key will be able to communicate with the server.

Hopefully that makes sense (I'll have to create an actual diagram and put it on the wiki at some point).

Thanks,

-Pat
Visit this user's website Find all posts by this user
Quote this message in a reply
11-25-2010, 12:07 PM
Post: #24
RE: WebPagetest on EC2
(11-25-2010 11:31 AM)pmeenan Wrote:  Pretty close. Are you talking about a WPT Monitor configuation (automated testing) or just a straight-up WebPagetest instance?
...
Ultimately, it will be a automated testing setup, for ongoing monitoring.

(11-25-2010 11:31 AM)pmeenan Wrote:  ...
The only connection needed for communications is to port 80 on the web server. If you're concerned about rogue agents connecting to the agent queues you can assign secret keys to each location so only testers configured with the correct key will be able to communicate with the server.
...
Ahh. So that's what the 'wpt_key' is for. I was wondering how it fitted into the picture.

It's making more sense now.

I think I was putting the cart before the horse by starting with the EC2 agents.

Thanks for clarifying the setup.
Find all posts by this user
Quote this message in a reply
11-26-2010, 02:36 PM
Post: #25
RE: WebPagetest on EC2
Can I make the agent connect to an alternate port by using:

wpt_server=www.example.com:8080

How do I know when my 'agent' is available on my WebPageTest web server?

How do I use the wpt_key on the WebPageTest web server?
Find all posts by this user
Quote this message in a reply
11-27-2010, 12:36 AM
Post: #26
RE: WebPagetest on EC2
You should be able to connect to an alternate port (just like you showed) but I haven't had a chance to test it yet.

As far as status goes, there will be an API in the next release that will let you see the status of every machine for every location (you can see what it looks like for WebPagetest here: http://www.webpagetest.org/getTesters.php ). Until then the only way to tell when a location has last checked for work is by looking in the work/times directory on the server and there will be a file for each location (with the timestamp of when it last connected).

For the location key, on the web server you go into locations.ini and for any location you want to protect you put a "key" entry in the same part that specifies the localDir that looks like:

key=myKey-GoesH3re

(just replace the key with whatever you want to use - it's a full string match on the key so it needs to match exactly on both sides and can't have any whitespace).
Visit this user's website Find all posts by this user
Quote this message in a reply
12-30-2010, 12:05 PM (This post was last modified: 12-30-2010 12:05 PM by mjlp.)
Post: #27
RE: WebPagetest on EC2
Hi Pat,

We're having some trouble with the EC2 box not connecting to our local box. We some issues over Xmas, gave the EC2 box a reboot, deleted the backlog of tests and all came good. However, for some reason today, it seems to have given up again and we're seeing a number of tests building up.

Just checked the work/times and the last connection was 50 minutes ago. I've tried giving the box another reboot but no luck this time. Is there anything we could try to get it working again?

Thanks,
Mark.

I should have also added - the EC2 box is responding to pings.
Find all posts by this user
Quote this message in a reply
12-30-2010, 12:30 PM
Post: #28
RE: WebPagetest on EC2
PM me if you want the Admin password to RDP in and poke around but my success with EC2 has been hit and miss. I've had a couple of VM's that ran for months with no issue but more often than not they seem to go out to lunch inside of 24 hours (and even become unpingable). Usually a reboot recovers them but sometimes I have to nuke the instance and bring up a new one.

I'm going to be adding logic into WebPagetest itself for it to automatically manage EC2 instances which should help mitigate the problem but I'd love to hear if anyone has seen similar issues and knows what might be going on.
Visit this user's website Find all posts by this user
Quote this message in a reply
12-30-2010, 02:09 PM
Post: #29
RE: WebPagetest on EC2
As best I can tell, the agent wasn't able to reconnect as Win2003 was stuck waiting for someone to tell it why it shutdown unexpectedly. Once I logged in and got past this message, everything started up and tests starting running.

I'm not sure what is causing the machine to say it shutdown unexpectedly when doing a reboot from the EC2 portal, but switching off the shutdown error message seems to get around this problem.

Hope this helps.
Find all posts by this user
Quote this message in a reply
12-31-2010, 07:38 AM
Post: #30
RE: WebPagetest on EC2
The box stopped responding again this morning - a manual reboot from the EC2 portal brought it back to life.

I put some monitoring on yesterday and the timing seems too coincidental not to be related to the 750min reboot. Has anyone seen anything similar?

I have no idea what could have caused this to start happening after weeks of incident free operation or how this could be related to EC2.

Will keep investigating.
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread: 1 Guest(s)