Current time: 10-23-2018, 03:33 PM Hello There, Guest! (LoginRegister)

Post Reply 
 
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Multi-server install (High Availability)
06-05-2018, 05:24 AM
Post: #1
Multi-server install (High Availability)
Hello,

Not sure if anyone tried this or not before. I am looking for the fastest path to support high availability for a self-hosted WPT on AWS.

Ideally, it will be great to be able to spawn two WPT servers in different AZs and front them with ELB.

It seems this is not supported easily since the servers have their own test results store and not using a shared one. If a client asks for a particular test result, it might have a miss in case the request goes to the wrong backend server. Clients will be API consumers and web browsers. Session stickiness help in the latter but not the former.

I can see that there is a serverID setting but this seems to influence only the test ID value generation.

Thanks!
Find all posts by this user
Quote this message in a reply
06-05-2018, 07:16 AM
Post: #2
RE: Multi-server install (High Availability)
The test ID generation ensures the ID's are unique across servers so they can use the same backing store for results. It only works for the archived store (and they archive immediately) so any server can display results from any other server but the process of actually running the tests still needs to go through specific servers.
Visit this user's website Find all posts by this user
Quote this message in a reply
06-05-2018, 07:17 AM
Post: #3
RE: Multi-server install (High Availability)
btw, I have thought about how to make it truly HA with SQS or pub/sub instead of beanstalkd for the queues and result processing but none of it got plumbed and it will probably be a reasonable amount of work.
Visit this user's website Find all posts by this user
Quote this message in a reply
06-06-2018, 01:56 AM (This post was last modified: 06-06-2018 02:04 AM by wpt2018.)
Post: #4
RE: Multi-server install (High Availability)
(06-05-2018 07:16 AM)pmeenan Wrote:  The test ID generation ensures the ID's are unique across servers so they can use the same backing store for results. It only works for the archived store (and they archive immediately) so any server can display results from any other server but the process of actually running the tests still needs to go through specific servers.
Thanks Pat for the quick response.

For immediate S3 archiving over HTTPS and also test results can be retrieved from any server, would you please confirm below settings to achieve that? And is there any way to rely on IAM role attached to the EC2 to upload to S3 instead of injecting the access & secret keys?

archive_s3_server=s3.amazonaws.com
archive_s3_key=<access key>
archive_s3_secret=<secret>
archive_s3_bucket=archive-bucket
archive_s3_url=https://s3.amazonaws.com/

;Number of days to keep tests locally before archiving
archive_days=0

;Run archive script hourly automatically as agents poll for work
cron_archive=0

(06-05-2018 07:17 AM)pmeenan Wrote:  btw, I have thought about how to make it truly HA with SQS or pub/sub instead of beanstalkd for the queues and result processing but none of it got plumbed and it will probably be a reasonable amount of work.
This will be great addition to the WPT, thanks for your efforts!
Find all posts by this user
Quote this message in a reply
06-06-2018, 02:56 AM
Post: #5
RE: Multi-server install (High Availability)
As long as "lasyArchive" isn't set in settings.ini the tests will be uploaded to S3 as soon as they complete, independently of the archive setting: https://github.com/WPO-Foundation/webpag...s.php#L104

The archive setting really just controls how long the tests are kept on the local drive as well. In principle, the test should be readable from any server within a few seconds of it completing.
Visit this user's website Find all posts by this user
Quote this message in a reply
06-06-2018, 06:20 AM
Post: #6
RE: Multi-server install (High Availability)
(06-06-2018 02:56 AM)pmeenan Wrote:  As long as "lasyArchive" isn't set in settings.ini the tests will be uploaded to S3 as soon as they complete, independently of the archive setting: https://github.com/WPO-Foundation/webpag...s.php#L104

The archive setting really just controls how long the tests are kept on the local drive as well. In principle, the test should be readable from any server within a few seconds of it completing.

Would you please confirm below settings are the only required ones for S3 archiving? For some reason, we don't see the test results in the S3 bucket even though the AWS key have the proper permissions and confirmed that by running aws s3 cp command and setting the keys on the WPT server.

archive_s3_server=s3.amazonaws.com
archive_s3_key=my-key
archive_s3_secret=my-secret
archive_s3_bucket=my-archive-bucket
archive_days=0
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread: 1 Guest(s)