Current time: 07-18-2019, 02:26 AM Hello There, Guest! (LoginRegister)

Post Reply 
 
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Issues with disk space due to too many inodes
08-30-2014, 08:38 AM
Post: #1
Issues with disk space due to too many inodes
We have been seeing an issue in our private instance due to too many "result" data files in webpagetest server. We fired off a big load on our private instance, and after a while, the server started complaining about disk space issue. On investigating, we seem to have ran out of inodes (rather than actual disk space) and we suspect it is because of a large number of "result" data files created for each measurement. For what it's worth, we are using ext4fs.

Is there a way to tune the file system to deal with this in a fashion other than purging/archiving old data? Some file system tuning option maybe?
Find all posts by this user
Quote this message in a reply
02-03-2015, 01:00 AM
Post: #2
RE: Issues with disk space due to too many inodes
I'm facing the same issue with private instance running from the AWS AMI. Haven't found a solution yet...
Find all posts by this user
Quote this message in a reply
02-03-2015, 08:23 AM
Post: #3
RE: Issues with disk space due to too many inodes
Ooh .. Linux - my favorite subject..

The best option - create a ZFS partition for your result data files.
See http://zfsonlinux.org/

I would personally keep everything else on the ext4 partition.

If you want to use an ext4 partition, you want to create a partition with a high inode count (few bytes per inode) like the 'news' type.
Create a new partition with something like:
Code:
mkfs.ext4 -T news /dev/yourpartition

A better solution:
Use ZFS - it allocates inodes on the fly.

Quote:In ZFS, you will run out of inodes at the same time as you run out of space.
http://www.mail-archive.com/zfs-discuss@...43751.html
https://blogs.oracle.com/roch/entry/need_inodes

------------
All websites deserve to load fast.
Find all posts by this user
Quote this message in a reply
02-03-2015, 08:00 PM
Post: #4
RE: Issues with disk space due to too many inodes
Thanks. This was an issue with the 'out-of-the-box' AWS machine image (AMI), so changing the FS might be a good suggestion for an AMI update.
Find all posts by this user
Quote this message in a reply
02-03-2015, 09:35 PM
Post: #5
RE: Issues with disk space due to too many inodes
(02-03-2015 08:00 PM)kevinrdixon Wrote:  Thanks. This was an issue with the 'out-of-the-box' AWS machine image (AMI), so changing the FS might be a good suggestion for an AMI update.

Ah .. I shot a message to pmeenan about this.
Maybe he can chime in.

------------
All websites deserve to load fast.
Find all posts by this user
Quote this message in a reply
02-04-2015, 12:12 AM
Post: #6
RE: Issues with disk space due to too many inodes
One thing on my todo list that will help significantly would be to archive tests in-place into zip files after the test is complete and all writes have finished. That would replace the result directory with a result zip file and then all of the code would extract the files they need at runtime directly from the zip.

That would reduce inode usage at least an order of magnitude and also reduce wasted disk space from block allocation.

It shouldn't be too hard to pull off. Most of the file reads are already centralized so it would just be a matter of hooking into them. Reading individual files out of a zip should also be plenty fast since you can random seek directly to them.

In the short term the best answer is to configure the server to archive to a different path on the same server (/var/wpt-archive or something like that) which will zip up the tests and get the same inode savings but keep the results on the same server.
Visit this user's website Find all posts by this user
Quote this message in a reply
02-04-2015, 12:21 AM
Post: #7
RE: Issues with disk space due to too many inodes
A better solution would be to replace the files with a (file) database that can be merged on the server.
One solution is sqlite (if you want SQL db) or unqlite (if you want a Document store engine - better suited for WPT imho)

But I imagine that would take quite a bit of rewriting as well ..

------------
All websites deserve to load fast.
Find all posts by this user
Quote this message in a reply
02-04-2015, 12:26 AM
Post: #8
RE: Issues with disk space due to too many inodes
I don't know if I'd say "better". Having all of the results be stand-alone and completely stateless and not having a database dependency makes it a LOT easier to manage.
Visit this user's website Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread: 2 Guest(s)