Tag Archive for 'Search'

Lessons Learned Scaling the Audiogalaxy Search Engine

Photo by flickr user vadaniaTackling a big, new programming challenge frequently means that you don’t actually understand the problem until after you solve it. Because of that, one of the most important things to do after nailing down a solution is to think about better solutions. Post-mortems seem to be more popular from a “how could we have managed this project better” angle, but the technical aspect should not be overlooked.

Sure, constant criticism of everything you do may not make you a happier person, but even if you don’t actually implement your new ideas, reevaluating your designs will make you a stronger architect. Just try to keep the habit from spilling over too much into your personal life. :)

Continue reading ‘Lessons Learned Scaling the Audiogalaxy Search Engine’

Design details of Audiogalaxy.com’s high performance MySQL search engine

As I mentioned before, search was one of most interesting problems I worked on at Audiogalaxy. It was one of the core functions of the site, and somewhere between 50 to 70 million searches were performed every day. At peak times, the search engine needed to handle 1500-2000 searches every second against a MySQL database with about 200 million rows. It was frequently hard to design for more than 10 or 100x our current traffic (and our growth was so dramatic that there wasn’t really ever time to spend more than a few weeks on the problem), so it wasn’t until the 4th major iteration on the cluster design that I really considered the scale problems to be solved.

Continue reading ‘Design details of Audiogalaxy.com’s high performance MySQL search engine’