Servers – more than a case, CPU, RAM, and a HD

3 Mar

I never understand people who try to build a large web-presence on the back of a $7.99 hosting plan. The same people that will spend 10x that on lattes in a month.

What I do find amusing is when people claim that hosting fees are a negligent part of a web-business. Ugh. Hosting is one of the nastiest expenses I have come across. I started off from a virtual host back in 1997, graduated to a Cobalt RaQ in 2000, upped to my first colo server in 2002, and last year setup our own rack. Things to consider:

  • Servers – We colocated, so we built our own servers. We purchased a dozen servers with dual dual-core Opteron 265s, 4 gigs of ram, and 2×73 GB HDs. We upgraded one of the servers to have 16 gigs of ram and 8×73 gb HDs (to be used as a database server).
  • Router – Needed to direct our traffic. Later on we can connect directly to carriers without having to go through our current host.
  • Firewall – Getting DDoSed is old hat. We get a DoS attack roughly once every 6-8 weeks. The reasons vary, from a jilted webmaster to an angry competitor. While we never had the pleasure, one of my friends once had a competitor write a worm that DDoSed his site. It wasn’t pretty – he peaked at roughly 1500+ mbit/s
  • Power – Power ain’t cheap. The most common circuit is 20 amps. We suck enough juice that we need two circuits. When setting up our servers, we put them all in an infinite loop to see the max amount of power each server would suck.
  • APC – Servers go down. Act of God, damn script kiddies, or just Apache going bonkers, they do go down. You need a way of resetting them without having to call in your host. You can get an APC that either takes up 1u or one that hangs on the side (taking up no server space).
  • Cost of Rack – You are basically ‘renting space’ from a host.
  • Bandwidth – This ain’t ev1 here folks. You need to be concerned about two things – throughput (amount of data transferred per second) and carrier (who you are connected to). The more carriers, the faster your site will load, but the more expensive it is.
  • Console / KVM – A console/KVM allows one to administer a server as if you are there yourself. Using our console, we can see our server boot up through the BIOS and so forth (basically the server has a serial output that connects to the console. You then connect to the console, letting you control the server as if your keyboard was connected directly to it). Unless you want to send your server admin every time there is a problem, a console is a requirement.
  • Phone line (optional) Our APC and console units have a telephone line connected to them. In case of emergency, we can actually dial into our setup and manipulate directly.
  • Spares – HDs, RAM, etc. – Computer hardware dies. And nothing beats up on computer hardware like the web. RAM will die. A hard disk will die. You better be ready.
  • Server Admin – Sort of an obvious one, but something many miss. Your job is to run a business, not to worry about how the servers are doing.
  • Functions – In our current setup, we have 10 web servers, one database server, and one ‘monitoring’ server. The monitoring server does mail, it records information on all servers, it does DNS lookups, and so forth. Having one server that did everything (web, mail, SQL, DNS) was nice when you begin, but you need to specialize as you scale. As our sites grow, we will add other servers (for stuff such as memcached, etc)
  • Backups – We backup every day to a separate server. We also backup once a week to an off-site server. Once a month DVD burn. This is your livelihood – screw ups happen all the damn time. Make sure you protect yourself.
  • Insurance (depends) – If a fire breaks out, who is gonna cover the cost of hardware? Some hosts provide for it, some don’t. Ask.

Fun ain’t it?

1 Response to Servers – more than a case, CPU, RAM, and a HD


Legal Threat for tracking Uptime - Tech Soapbox

March 27th, 2007 at 10:50 am

[...] Just another case of over-reaction (and I know full well both what DDOSes feel like and what running a server infrastructure entails). [...]