WebPagetest Forums
OSError: [Errno 12] Cannot allocate memory - Printable Version

+- WebPagetest Forums (https://www.webpagetest.org/forums)
+-- Forum: WebPagetest (/forumdisplay.php?fid=7)
+--- Forum: Private Instances (/forumdisplay.php?fid=12)
+--- Thread: OSError: [Errno 12] Cannot allocate memory (/showthread.php?tid=15969)



OSError: [Errno 12] Cannot allocate memory - jedmichnowicz - 12-20-2019 03:01 AM

Our test agents have started to fail after successfully running for a few hours. The error is: OSError: [Errno 12] Cannot allocate memory

We use docker images on c4.large instances (4GB RAM) but have also seen this failure using on-prem instances with 8GB of ram. Rolling back to an image from early December did not fix the issue, so I suspect this could be due to a browser change.

To get agents working again, we can restart the container. Adding swap also seems to fix the issue, but we have not run tests long enough to see if this is a permanent fix.

Anyone else see this issue and have a permanent fix?

We run via docker with these options:
docker run -d --shm-size=1g -e "SERVER_URL=http://<fqdn>/work/" -e "LOCATION=${region}_Linux" --cap-add="NET_ADMIN" --restart unless-stopped --name <custom-repo>/webpagetest/agent:latest


RE: OSError: [Errno 12] Cannot allocate memory - doronin - 01-16-2020 11:21 PM

Any news here? Our 2 agents exhaust 8GB of memory within less than 8 hours.


RE: OSError: [Errno 12] Cannot allocate memory - jedmichnowicz - 01-17-2020 09:19 AM

We are working around the issue by doing an hourly container restart. If you do a "ps" on the host, you will see a lot of defunct chrome processes.

From our user-data script:
"# Restart once/hour for zombie procs\n",
"sudo echo '#!/bin/bash\n",
"sleep $(( ${RANDOM:0:1}*60 ))\n",
"docker restart wpt' > /etc/cron.hourly/wpt-restart\n",
"sudo chmod 755 /etc/cron.hourly/wpt-restart\n"


RE: OSError: [Errno 12] Cannot allocate memory - doronin - 01-20-2020 09:53 PM

Thank you, jedmichnowicz. So do we restart them with some frequency. Weird, why just two of us are here. I believe it's a common issue.
The only reason for no answers, I guess, is that wpt docker is not maintained here.