A recent Developer Zone article, Why Startups Should Not Choose NoSql, argues that startups should “stick to a SQL solution” instead of choosing technologies such as Cassandra, MongoDB, HBase, Redis, etc. The author writes that this is not only because building good NoSQL data and query models is very difficult; but also that agility is key. As a startup learns what it has to change one or two years later, the challenge of extending these models is simply daunting.
The article goes on to explain how the tools typically found in relational databases make such evolution relatively easy: if you need a new entity type, create a new table. If you need a new relationship, write a new JOIN.
While it might be true, as the article points out, that many startups don’t require the web scale benefits that NoSQL offers, what if you are one of the lucky ones with an instantly viral service? After all, are you going to sink your blood, sweat and tears into a business and assume that you will not be wildly successful? Not believing in and being prepared for that kind of success, is a self-fulfilling disaster in the making.
In that context, the NoSQL solutions certainly seem more reasonable. Simplified horizontal scaling for user/query capacity and distributed/sharded repository management then become important, but that still doesn’t address the ease of use and agility shortcomings the article presents.
Some of the leading commercial examples have solved some of these problems, but they either have other limitations or are extremely complex and expensive to set up.
A hybrid system in which you model using a relational database, and then push the data out to a sharded system to serve queries, is another option. It’s not all that difficult to evolve into, and you get the benefit of good tooling and modeling support early on. Then you can treat the NoSQL system as a “downstream” system that you just feed and query. The issue here is that now you’ve done twice the work, implementing a relational database on day one and building a great application for your users, but then you have to throw much of it away when you move to NoSQL. Anyone who has worked in a startup knows that throwing away something that ‘kinda works’ and starting from scratch is tough to do….until it’s too late.
So is there a “Goldilocks” or “just right” solution for startups? Not surprisingly, we believe Attivio’s Active Intelligence Engine (AIE) is the perfect fit. AIE supports a vast range of SQL to help developers get up and running quickly. Need to prototype a dashboard in a SQL tool like QlikView, Tableau or Tibco Spotfire? No problem. Some business guy says he needs to see the relationship between cats, dogs and mice with five minutes notice? We can help you there too. Our patented JOIN operator lets you use any relationships between records of any type — on the fly.
So far so good, but that sounds like a relational database, doesn’t it? We would be remiss in mentioning that full-text search is part of AIE, but not available in most RDBs. So you’re already way ahead.
Now what happens if, as we mentioned, you succeed and demand for your service soars? This where we really start to look good. Underneath Attivio’s support for SQL is an index structure that supports massive, linear scalability. We recently helped a customer index 300 million documents; its 1.7 TB of data, sharded across three servers. The entire set is replicated to three additional machines to provide fault tolerance and additional query capacity. Our support for non-collocated JOINs means that you can STILL query the entire data set. You don’t have to manage the content on each shard, either. AIE takes care of all of that for you.
AIE fully supports the startup technology model, as described by the article. As the startup evolves, pivots and ultimately connects with their audience, they will find the ingestion, modeling, SQL, full-text search and scalability that they need — all supported by a commercial company with a superb track record of delivery and successful implementations.