Sunday, April 17, 2011
The Cost of Small Files in the Cloud
I've been working with a friend on a little project that generates a mound of small files. Specifically, we'd like to serve 12 million files of less than 4 KB with an average size of 500 bytes. It's only 6 GB.
I thought Amazon S3 might be a reasonable way to do it, so I did the math. I'm not expecting the project to get a lot of usage, so I initially only cared about the cost of getting the files out there. Amazon S3 charges $0.01 per 1000 PUTs, $0.14/GB/month for storage, and $0.10/GB for input bandwidth. For one month that's 12M/1000 * 0.01 + .14 * 6 + .1 * 6 = $121.44, of which $120 comes from the cost of PUTs alone. That quickly dissuaded me from considering S3.
Then I remembered that Rackspace has a cloud file storage product, too. Rackspace's Cloud Files pricing is based only on bytes stored and incoming and outgoing bandwidth. My 6 GB would cost $0.15/GB/month to store and $0.08/GB to upload, for a total of $1.38. Now we're talking.
I was a little dubious about the pricing, so I contacted Rackspace to double check my numbers. The rep told me I was doing the math right and claimed that you only pay for reported file size, not size rounded up to the nearest block and not HTTP header overhead.
The cost winner if you want to store a bunch of small files in the Cloud is clearly Rackspace.
I thought Amazon S3 might be a reasonable way to do it, so I did the math. I'm not expecting the project to get a lot of usage, so I initially only cared about the cost of getting the files out there. Amazon S3 charges $0.01 per 1000 PUTs, $0.14/GB/month for storage, and $0.10/GB for input bandwidth. For one month that's 12M/1000 * 0.01 + .14 * 6 + .1 * 6 = $121.44, of which $120 comes from the cost of PUTs alone. That quickly dissuaded me from considering S3.
Then I remembered that Rackspace has a cloud file storage product, too. Rackspace's Cloud Files pricing is based only on bytes stored and incoming and outgoing bandwidth. My 6 GB would cost $0.15/GB/month to store and $0.08/GB to upload, for a total of $1.38. Now we're talking.
I was a little dubious about the pricing, so I contacted Rackspace to double check my numbers. The rep told me I was doing the math right and claimed that you only pay for reported file size, not size rounded up to the nearest block and not HTTP header overhead.
The cost winner if you want to store a bunch of small files in the Cloud is clearly Rackspace.
Labels: cloud files, cost, rackspace, s3
posted by Leif Wickland @ 1:02 AM 
