Evolution of MySpace

April 26, 2007

Here’s a summary of the evolution of the MySpace architecture as described here:

  1. Initial system: ColdFusion on 2 web servers + 1 SQL Server, added more web servers as load increased
  2. >500K users: Add more DBs. 1 master for all writes, many slaves for reading (initially 2 slaves).
  3. >1M: DB I/O maxed out. Vertical partition of DB, i.e. separate DBs for different functions. Switch to SAN.
  4. >3M: Single “logical” distributed database. Giant tables are split into chunks running on different machines.
  5. >9M: Replace ColdFusion with ASP.NET/C#.
  6. >10M: SAN I/O maxed out. Switch to SAN from 3PARdata.
  7. >17M: Added caching servers in front of DBs.
  8. >26M: Switch to 64bit computing w/ SQL 2005 and Windows 2003. Machines are loaded with 64G of memory.

It seems like a new web site should start with 64bit servers, a compiled language (Java/C#), a distributed DB design, and caches everywhere. This basic design approach should scale for quite a while by simply throwing more hardware at the problem. Maybe.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: