Guest Column | August 28, 2014

Will Open Source Hadoop Last?

By Gil Allouche, Vice President of Marketing, Qubole

Hadoop Big Data

When it comes to Big Data implementation, there’s no bigger name around than Hadoop, to the point that many people think that Hadoop is Big Data. For a variety of reasons the Hadoop framework has become the go to system for numerous Big Data options.

When using Big Data, the process isn’t as simple as just downloading Hadoop and then beginning. A quick visit to the Hadoop website will reveal the complex nature of Hadoop and its programs. There are numerous different modules and projects too, like Hadoop YARN, Hive, Pig, Spark as a Service, etc. It takes a lot to create all these systems, and it also takes a lot for companies to properly implement them.

With all the work that goes into them and their importance in the Big Data field, one of the most important questions to be asked is how viable the current open-source situation of Hadoop is? Can it last?

A Look At Open Source vs. Proprietary

With open source Hadoop, companies that provide the Hadoop service don’t make any money from the Hadoop software itself, they only gain monetarily from the services they provide to help companies use Hadoop to its fullest. On the proprietary side, companies build software off the Hadoop framework that is sold for profit, along with additional services.

Many people in the Big Data industry don’t think the open-source model can last. Eventually, they feel, it’s going to be too much to try and manage and improve, and the operating costs will take their toll on the Hadoop community.

Other people, however, disagree with the notion that open source isn’t a viable option for the future. It has worked so far, why wouldn’t it continue? Especially with all the projected growth that Big Data is supposed to have in the upcoming months and years. If companies keep using it, then why change it? If you can generate enough money from services, and not software, then why worry about making proprietary software?

It’s a complex problem to be sure, without a definite answer.

Concerns For Businesses

One of the biggest concerns for companies looking to implement Hadoop and its ecosystem is finding a provider that will last. That means it’s got to be a company that’s going to withstand the financial storms that come. And if an open-source company can’t do that, then it’s going to be hard for people to go with them for their services.

Really, the deciding factor between open-source and proprietary is monetary gain. There’s no doubt that maintaining the software open-source is an enormous benefit to companies across the globe that want to implement Hadoop. They have access to tremendous Big Data tools that can drastically alter the way they do business, all without charge.

The question though, is whether open source can survive without the same monetary gain through the software that proprietary brings in.

Which Focuses On The Customer?

Along with that, another important question to examine is which solution — open source or proprietary — is more customer based. On first glance it seems that open source is the answer. After all it’s free to download the software. and companies are only paying for the services they need. On the surface it sounds fantastic, but if it’s not a viable solution, then is it really benefitting the customer?

On the other hand, it seems like proprietary solutions are all about the money. However, if it’s the only way companies have to stay in business and offer their customers the services they need, then that’s saying a lot. Maybe it’s actually more customer-based than open source. It’s difficult to answer.

Currently, there’s no simple answer to the open source vs. proprietary debate because both are faring particularly well at the moment. Maybe, there’s room for both of them in the Big Data arena, and customers will continue to have two choices for how they use Hadoop.

Whatever the outcome is, it’s important that companies looking to implement Big Data and Hadoop understand the implications of which one they choose and they make the appropriate choice for their business.

Gil Allouche is the Vice President of Marketing at Qubole. Gil began his marketing career as a product strategist at SAP while earning his MBA at Babson College and is a former software engineer.