Case study: Data integration / mashup

The IT department in a mid-sized company tackles a data integration task: Recently, sales operations were moved to Salesforce. However, historic sales and customer records still reside in a local database. Retrieval of unified records therefore is not possible. Knowledge workers need to perform two lookups, one with a web browser to get Salesforce info, another using a custom legacy applications, which provides access to the old database. Clearly, this needs to change and a way needs to be found to give workers a simple, unified view, regardless of where the actual data resides. A classic data integration task.

The options they consider:

1. A data integration tool/framework with GUI designer.
2. All custom code, using a framework such as Spring.
3. RESTx.

In the end, they decide to go with RESTx, because:

• RESTx has a much smaller learning curve compared to other frameworks and tools, thus there is less organizational commitment (training of resources), and it is even suitable for less skilled developers. This allowed the IT department to let a junior developer implement the data integration code, keeping their senior developers free for other tasks.
• Seamless integration of other languages, allowing them to use packages and libraries written for those languages. This was important, because they were able to use a robust Java Salesforce client library, while accessing their proprietary application with some quickly written Python code. Access to Salesforce and access to their custom database were encapsulated by two components.
• Flexibility to express custom integration logic in easily written custom code: As easy or easier than figuring out how to construct integration pipelines with limited filter resources in a graphical editor such as the ones provided by some data integration tools. Using a few lines of Python code within an integration component, the two specific Salesforce and custom DB resources were accessed. The two lists were combined and the sorted result was provided as output. This is also demonstrated in our simple data integration example.
• Ability to quickly express the various resources that need to be integrated as individual resources. This makes them reusable in different contexts and with different parameters. Configurations do not need to be hardwired either in code or configuration files that are only accessible to administrators. No custom logic needs to be implemented to make various configurations easily available, since this is one of the core features provided by RESTx through RESTful resources. Two specific resources (configurations) for these Salesforce and custom DB components were created for the data integration task at hand. But since access to these data assets was now so conveniently encapsulated, it was possible for IT as well as some power users to quickly provide new configurations, creating specific resources, accessible through an ordinary URI and suitable for other situations.
• The resources around the different data sources can be linked together easily, since the data format from those various sources is unified into a simple hierarchical document or record structure. Data format conversion is automatically taken care of, once the access component/resources for the data source is configured. This made the data integration task particularly easy: Combining the two lists in the data integration component became natural and simple, since data from both diverse resources was presented to the developer in a single, unified format.