EMC’s Pivotal to Embrace Open Source, and Hortonworks Too

Pivotal, the big data and software division of storage and IT giant EMC, will today announce plans to release open source versions of its big data and analytics software.

Creating what it is calling the Pivotal Big Data Suite, the company will release its products as open source software, allowing anyone to download and use them for free, to modify them as needed and to contribute code for use in future revisions.

The move also includes a strategic partnership with Hortonworks, which supplies and supports Hadoop, the open source software used for organizing large pools of corporate data for later analysis. Pivotal has its own flavor of Hadoop, but the two have been partners since July.

In what is likely to be a big deal for Hortonworks, it will essentially take over support of Pivotal’s version of Hadoop known as Pivotal HD. Perhaps more important, Pivotal will be recommending Hortonworks’ version of Hadoop to most customers, except in a few special cases. Further, the two companies will team up on the development of HAWQ, an analytics engine, and GemFire, a product used for deploying software in large-scale environments, both of which are included in the open source offering.

That doesn’t mean that everything Pivotal makes will be free. It will still offer commercial versions of its software with added features and capabilities not available in the open source versions, as well as support for companies that buy them.

So why go open source at all? Many customers require it, says Sundeep Madra, a VP at Pivotal. “They don’t want to be locked in to any one vendor. … If they decide to change, someone else will be able to come in and pick up where the other left off,” he said. Customers can also more easily build custom add-ons for open source software, he said.

Finally, Pivotal is launching an industry group devoted to promoting Hadoop and related big data software called the Open Data Platform. Its members will include GE, which is an investor in Pivotal, and Hortonworks, but also Verizon, and IBM and Infosys. A few other names are likely to be included in the announcement later today.

This news is likely to provide a jolt to the world of Hadoop, which has been the target of aggressive investment by venture capital firms in companies like Hortonworks, which went public last year, and Cloudera, which is partially owned by Intel.

The basic premise of Hadoop is that it helps companies sort and manage huge troves of corporate data — Pivotal likes to call them Data Lakes — which they can then analyze in search of useful insights.

This article originally appeared on Recode.net.