Custom Application Integration and Migration
Integrating or migrating custom applications that were written by in-house developers is typically one of the easiest kinds of integrations. You have all the advantages of access to the developers and tons of really good documentation. No, wait – a consultant built it and had his contract cut the day the system went into production, isn’t on the best of terms with your employer, and documentation just wasn’t a priority. Now what do we do?
Legacy Systems: Keep or Kill?
Accounting is accounting. Payroll is payroll. Many business processes are standardized according to legal requirements, and the IRS won’t give you a pass just because you’re on last year’s release of the tax tables or electronic forms.
So why would anyone use a custom solution? Custom applications exist for many reasons. Sometimes, a custom application adds value to your company that just can’t be found in an off the shelf application. Perhaps it’s a core function of your unique business model that gives your company a strategic advantage. Perhaps you just can’t afford to let go of what works because the transition costs are too high. Perhaps everyone’s happy with the current solution and there is no compelling need to change. Or perhaps it’s time to dump it.
In any event, if the information is to have any value, it’s got to be migrated or integrated with your other applications. Data migration is just another form of data integration, except for its short shelf life. And just to make things more interesting, data migration usually comes with short deadlines because users want immediate value from the new system.
Custom applications are built by people operating under deadlines. Most development projects miss those deadlines, so something has to give. Documentation is the first deliverable to go by the wayside, as nobody gets rewarded for a well-documented system that doesn’t work, but people get rewarded, the consultants get their contracts
terminated, and last-minute bug fixing always takes priority over documentation. Perhaps there was a specification in the beginning, but by the time the project is over there have been so many diversions from the original plan that that specification is obsolete. Perhaps users have been clever in corrupting the original meaning of the schema to get out what they need. In any event, you can hope that the database schema at least makes sense from the names of things, and the ranges of values can be determined by running SQL queries.
Relative Simplicity of Source Schema
Compared to vendor solutions that have to be infinitely customizable, a home-built custom solution doesn’t have to solve all the problems for all the customers to which a vendor might want to cater. The degree of customization is hard-wired into the database tables and column names instead of using generic custom fields that might later be defined with metadata, such as the case with vendor package solutions. The meaning and intent of the database designer is transparent and is directly encoded with meaningful table and column names. You won’t have as much need for a platform-specific data dictionary, but you very likely won’t have one entirely, just table and column names that will hopefully be meaningful. One problem with legacy custom schemas is that they don’t always have timestamps, which make incremental integration very much harder. But overall, home-built databases tend to be easier to integrate due to their purpose built and self-documenting nature.
Make Friends Fast
It is important to identify not only the stakeholders in the systems being integrated, but if possible the original developers. Be nice to these people. They may have grown tired of hearing about the many faults in the systems that they depend upon or that they developed. They may have moved on to other projects and don’t want to revisit that dark period in their lives when they had to work 80 hours a week for months on end to meet unrealistic deadlines. Perhaps the user of the system is the same person who developed it ten years ago, and you aren’t particularly impressed with his coding style. You don’t want to be on the wrong side of that guy if you’re going to need him as a fountain of information, given that the documentation is ten years obsolete or completely missing.
Update Missing or Erroneous Documentation
If you have documentation, verify its accuracy. If the documentation says the range of values for a field includes a fixed set of values, test whether that is true. If you discover other values, update the documentation, including the meanings of those extraneous values. Test all the assumptions about the data. Is the order ship date really always later than the order request date? What does it mean if it’s earlier? What if “Maybe” or a null value is one of the actual choices for a “True” or “False” field? What do you do if a date is out of range, such as an order date of January 5, 2071? And what does the business expect you to do with that knowledge – reject the records, patch up the data, or provide exception reports?
Custom-built systems offer advantages and disadvantages over package solutions. The physical implementation is usually a product of business needs rather than platform flexibility needs, so the learning curve should be easier. Your political skills in finding sources of documentation or information will be useful.