Evolution of MySpace
April 26, 2007
Here’s a summary of the evolution of the MySpace architecture as described here:
- Initial system: ColdFusion on 2 web servers + 1 SQL Server, added more web servers as load increased
- >500K users: Add more DBs. 1 master for all writes, many slaves for reading (initially 2 slaves).
- >1M: DB I/O maxed out. Vertical partition of DB, i.e. separate DBs for different functions. Switch to SAN.
- >3M: Single “logical” distributed database. Giant tables are split into chunks running on different machines.
- >9M: Replace ColdFusion with ASP.NET/C#.
- >10M: SAN I/O maxed out. Switch to SAN from 3PARdata.
- >17M: Added caching servers in front of DBs.
- >26M: Switch to 64bit computing w/ SQL 2005 and Windows 2003. Machines are loaded with 64G of memory.
It seems like a new web site should start with 64bit servers, a compiled language (Java/C#), a distributed DB design, and caches everywhere. This basic design approach should scale for quite a while by simply throwing more hardware at the problem. Maybe.