memcached is an in-memory caching layer that is used by many websites to improve performance. Brad Fitzpatrick originally wrote it to help LiveJournal scale.
Facebook uses and contributes to memcached. As of April, 2008, Facebook was using
805 memcached servers. From the same source, Wikipedia was using 79, Flickr 14, and Fotolog 40.
Ted Dziuba of
uncov fame may
publicly mock you if you write a blog post about using memcached to scale your site.
memcached has a 1MB max object size. If you need to store bigger objects, split them up, store the supplemental object count in the first object and have a naming policy for the supplemental keys. (e.g. “foo” says it has 5 parts → get “foo-2”, “foo-3”, “foo-4” and “foo-5” too.)
set a chunk_size that's right for your usage