Website scaling – no magic formula

Lately I’ve spoken to numerous entrepreneurs all keen to ensure that their new website can rapidly scale to cater for the massive amount of traffic that their brilliant idea will bring in.

Thought I would write up a bit of advice from the trenches.

Scaling is easy and incredibly difficult at the same time.

Easy in that there is plenty of examples scattered around the web how various people solved their problem. Just use technology X, Y and Z => problem solved.

The hard bit; your problem is most likely similar to, but not exactly the same as someone else. Small differences in architecture and feature set may seem inconsequential when serving 1K of users, but these small differences add up to massively different and complex problems when serving out to thousands/millions of users.

My feeling are a startup should focus on solving the user problem with an insanely great experience. Speed is part of that, but probably not the most important factor with early adopters.

Some thoughts.

1. Hadoop, Hive & similar are fantastic, but really overkill for a startup
2. NoSQL’s are great and fast and all that – but carry the risk of loosing transactional data.
3. MySQL & Postgres, seem no longer cool, but work reliably and can take you far
4. PHP, Rails, Python, & even DotNet all scale (scaling is in the architecture, not language – anyone who tells you otherwise is waving a big red flag that they are not all that clued up on web tech). Of course all can be used to build something that falls over with 5 users.
5. Caching (memcache & similar) is difficult to get right – it is not simple, particularly for logged in users with different access levels, and conducting transactions.
6. Amazon EC2, Google app engine & Heroku – are great to get started – you only pay for the services you use – at some point it will make sense to move to your own infrastructure, or go your own way if your really strapped for cash.
7. Hosting videos and images on Amazon s3 & Cloudfront helps keep infrastructure complexity down, but do cost. One startup I am working with has a couple of thousand users with a bill of around $300 per month for video & image hosting
8. Paypal credit card payments has not been a turnoff – I’ve had two services use it & seen no increase in conversions when replaced with a “named” solution. Web payments can be setup in less than 30 minutes.
9. Validate any advice & thoughts by asking questions and searching for existing answers on www.quora.com. Most things have been discussed a number of times before.
10. Be aware it’s usually where you do not expect to find problems that you come unstuck. For example, your users heavily using parts of your application  in unexpected ways, backing up transaction logs, updating data across multiple servers taking longer than you thought it would.
11. Make sure you have a plan – know your thresholds of tolerance, and have upgrade paths ready to go – you might just get those hundres of thousands/millions of users so be prepared.

So my view is, in the early days, keep things simple and tolerate as little complexity as possible.

Guide as what to measure and set thresholds for.

Budget for around a couple of days to a week to setup a monitoring system. You want to know resource usage trends, as well as if anything out of the ordinary is happening.

Your web site / application

  1. How fast home page loads
  2. How long various user tasks take (ie: logging in, doing a common tasks)
Each web server
  1. CPU usage & trends
  2. Memory usage & trends
  3. Threads usage & trends
Each database server
  1. As for each web server plus..
  2. Query response times – look in the slow query log file religiously
  3. Connection pooling
  4. Disk space usage & trends

Leave a Reply

Your email address will not be published. Required fields are marked *