Dont do your own IT at all
These were the amazing words I heard today from Larry Halff, founder of Ma.golia.com. I was submitting a blog article today and decided to check out the little link sharing icons our website puts below each article to submit it to link sharing sites. 
I tried to go to ma.gnolia.com, one of my favorites. To my surprise, it was gone, though their cute flower logo was still visible. Instead there was a web site with a painful story of file loss and implosion of their site, with a very interesting video from the founder of ma.gnolia.com listing his mistakes and learnings.
Citizen Garden Episode 11: Whither Ma.gnolia? from Larry Halff on Vimeo.
In short, their MySQL database got its files corrupted. Their half-terabyte database was backed up, but the corruption was backed up as well. Eventually, the data became unreadable, and ma.gnolia.com came to and end.
The founder, Larry Halff, showed tremendous and admirable humility in listing his mistakes, but he also told a cautionary tale that we here at ENKI both lived ourselves at startups as well as observed with some of our customers prior to their joining us:
- MySQL is a dangerously inadequate database for production work. Sure, YouTube uses it, and so do hundreds of thousands of LAMP sites. But it's very susceptible to corruption, and its administration facilities are inadequate to prevent it. In the hands of inexperienced or amateur DBAs, it is ripe for disaster, yet it is presented as an easy entry-level database. We've had customers suffer long downtimes due to corruption from crashes in MySQL. We also have customers with nearly insoluble performance problems. And, even though our cloud computing environment restarts software on failed servers automatically, MySQL does not always successfully survive a restart, or if it does, it requires extensive database table repair (especially with myISAM tables). If uptime is important, we feel you should look elsewhere, though it can be made to work with sufficient expertise.
- Startups shouldn't do their own IT. I bring this up often because of my own experiences as a software person trying to do my own IT, or watching the stream of missteps in the early years of startups I've worked at including NetSuite. In the video, Larry says, " The real lesson Learned is if you’re a startup, don’t do your own IT at all.” I'm hoping it's a lot more convincing when it doesn't come from my mouth, now that I've started a company to address this need.
- Infrastructure is what makes the difference between a web site and a web business. Larry points out that "in the process of developing Magnolia, infrastructure always took a back seat." Then he and the co-host go on to joke about how Cloud Computing (and Amazon) would have solved their problems. However, the actual failure they had would have happened in Amazon AWS just as well as their homebrew hardware, because the root cause was a lack of IT experience at ma.gnolia.com, not having physical hardware versus the Cloud's virtual hardware. What Larry needed was IT expertise expressed as IT practice and procedures, yet he didn't have any (and shouldn't have had to learn it himself since he was the creativity guy!)
TrackBack URI for this entry
