Current time: 08-20-2018, 04:02 AM Hello There, Guest! (LoginRegister)

Post Reply 
 
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Random failures from Taiwan WPT instance
07-26-2018, 06:47 PM
Post: #1
Random failures from Taiwan WPT instance
Hello,

I've been testing using various locations, and I've noticed that when I use "Taiwan - GCE" instance (any browser - Chrome, Firefox, Opera), sometimes it results in 503 response.

It is happening for any resource on Dailymotion.com domain (the example is a static JSON file to make things less complicated).

See here:
- (HTTP request) https://www.webpagetest.org/result/18072...6c7df8f30/
- (HTTPS request) https://www.webpagetest.org/result/18072...0da260523/

My question is, how can I be sure the issue is not on WPT Taiwan instance, but on DM server side?
I asked my infra team and they say they don't see any 503s in the graphs/logs.

Can those 503s come from WPT itself? If so, what could be the reasons?

Does WPT do Man-in-the-Middle and other things like that?

Thanks in advance for any suggestions.
Find all posts by this user
Quote this message in a reply
07-26-2018, 11:56 PM
Post: #2
RE: Random failures from Taiwan WPT instance
No, WPT doesn't proxy the outbound requests. A 503 would be generated by a reverse-proxy somewhere in the path, not at the server itself. If you are using a CDN then it is likely coming from the CDN edge, possibly because it can't reach the origin.

It's also possible that some security layers somewhere in front of the server are filtering/blocking the traffic intermittently intentionally (particularly since it is coming from Google Cloud).

You can capture a tcpdump which will show the 503's coming back from the edge (195.8.215.136 in the case of the test you sent). From the looks of it that's not a CDN edge but may still be a load balancer or security device somewhere upstream of the server logs.
Visit this user's website Find all posts by this user
Quote this message in a reply
07-26-2018, 11:58 PM
Post: #3
RE: Random failures from Taiwan WPT instance
btw, the tcpdump option is in the "advanced settings" tab of the advanced settings (capture packet dump). Sharing the actual tcpdump with the ops team will tend to help move the conversation along to "yes, it's really happening"
Visit this user's website Find all posts by this user
Quote this message in a reply
07-27-2018, 07:59 PM (This post was last modified: 07-27-2018 08:04 PM by jakub.)
Post: #4
RE: Random failures from Taiwan WPT instance
Hi Patrick, thanks for you input and the pcap suggestion!

Some things I forgot to mention / did not emphasise enough:

- the issues happen both for static and dynamic resources, I just used a static JSON URL for simplicity
- the IP from the WPT logs is 195.8.215.136 which is the expected IP for the domain, resolved in the same way from everywhere (it's not geodistributed / not a CDN)
- the issue happens both for http and https requests with equal chance
- so far I only reproduced on WPT Taiwan instance, not others (that's a bit weird honestly)

My reasoning is: since the IP matches, and issue happens for HTTPS request, and there's no MitM etc. (hence the integrity is guaranteed, no third-party proxies messing with the output), the problem indeed must be on the server side (maybe some anti-bot solution as you mentioned).

This is an interesting run where the initial http request is fine (redirects to https) but https gets 503 response:
https://www.webpagetest.org/result/18072...1_request2
pcap is here:
https://www.webpagetest.org/getgzip.php?...file=2.cap

I checked the PCAP for the https request and it looks like it contains the proper cert etc.

I will check with the devops again. Thanks!
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread: 1 Guest(s)