EC Wise, Inc.
Optimizing ETL Transformations

This is an interesting time to be a player in the data integration space. According to Tom Spitzer, VP of Engineering at EC Wise, “The past few years have witnessed the maturation of NoSQL databases as popular data stores, which evidences the explosion of types of data that organizations need to capture, store and analyze.” For many years, the bulk of data business dealt with was structured data produced by ERP, CRM and related systems, and the integration driver was data warehousing for business intelligence purposes.

Fortunately, the tooling is evolving together with the challenge. For instance, EC Wise uses the integration toolset in Talend Studio, which includes con¬nectors for NoSQL databases like Cassandra, Neo4J, HBase, MongoDB, CouchDB, Couchbase, and Riak. These connectors allow EC Wise integration specialists to work with these databases, as well as with SQL databases, queues and applications through the common abstractions of data maps and flows. In addition to support for NoSQL databases, vendors are adding support for stream processing and Metadata management which were historically very specialized and costly niche capabilities.

At the high level, EC Wise undertakes two main classes of data integration projects. In one class, processes assemble and often aggregate data for analytic purposes, whether in a data warehouse to facilitate Business Intelligence reporting or in a data lake to enable various types of analytic initiatives. In the other, data from multiple sources needs to be brought together in a common form for further processing. Applications that EC Wise developed exemplifying this class include preparation of customer statements for presentment via paper or e-bill, health insurance enrollment, and marketing systems.

At the end of the day, a data integration project is a type of software development project. That becomes obvious when you realize that the end result of using a graphical editor to model data transformations and processes is a set of java packages that the Talend engine executes or a set of .NET classes that Microsoft SQL Server Integration Server executes. As a company that develops complex software systems for its clients, EC Wise can apply software engineering techniques and practices to these projects.

Our team judiciously applies a build vs buy prism to all of their engagements to determine the optimal mix of prebuilt products and components vs new development

Such techniques include logging strategies, test driven development and secure programming. In addition, the EC Wise team’s deep expertise in a variety of RDBMS and NoSQL databases enables it to design performance optimizing ETL transformations.

There are several key leverage points that EC Wise uses to gain operational efficiencies, claims Spitzer. One of them is choosing strong technology partners. Spitzer and his team judiciously apply a build vs. buy prism to all of their engagements to determine the optimal mix of prebuilt products and components vs. new development. Another key leverage point is intelligent offshoring. The company developed a tech center in China as a wholly owned subsidiary. This provided much greater control over the way the outsource partner works than one would get in a typical contract based outsourcing arrangement.

EC Wise’s uniqueness lies in collaborating with its customers to provide strategic solutions. “We can build on integration solutions with machine learning capabilities to deliver industry specific decision hubs that help our clients understand and motivate behavior,” says Spitzer.

EC Wise’s transparent engagement model enhances collaboration and promotes partnership with its customers, making it one of the differentiators most appreciated by client CIOs. “In the final analysis, we understand that the value is not in the integration itself, but in the delivery of new insights or new capabilities that it enables, and we continually look for opportunities to increase the value of the information to our clients,” ends Spitzer.