Do I have to predict the data type?
A traditional challenge faced by organizations that want to use Intelligent Document Processing (IDP) software is the need to configure the system so that it does what you want it to do. Within that realm of challenge is having to anticipate the range of data types within documents.
Configuring Your IDP System
The most common answer to this dilemma is to configure a system based on sample data so that you increase the likelihood of success if one field on your document is handwritten vs. typed or if a another field is commonly numeric. However, the reality is that this approach requires serious attention to detail and a lot of drudgery in the form of sample analysis. At the end of the day, you might find that there is no reliable data type with which to configure your system. So you fall back to a configuration that uses many types of redundant processes to account for the unpredictability of your data.
Somewhere in between hours 48 and 163 configuring the system you scream out “why can’t the software determine the data type on its own????!!!!” and then slump over to continue your work.
Truly Intelligent Automation
What I’m about to tell you may either have you straighten-up in your chair with enthusiasm or smack your forehead in exasperation: truly intelligent document automation software has been able to determine data type for years. By applying runtime machine learning, the abilities are getting even better.
Take, for instance, the need to configure a system to process a form that can have handwritten data or text. Or even both. The traditional approach would be to design for one or the other and handle the exceptions with staff. Systems such as Parascript software can easily detect that data is handwritten or text and utilize the correct deep learning algorithm to interpret it. There are even algorithms that can either type equally as well.
Parascript software has been utilizing a function called “auto-detect” for years that looks at the data and identifies whether it looks like text or handwriting and then employs the appropriate interpretation algorithm. The latest interpretation techniques can parse a page that combines both in a single pass.
Imagine having a page of typed form data along with handwritten comments or other longer-form information and converting it into structured data without having to inform the software. That is the state of today’s automation.
Imagine having a page of typed form data along with handwritten comments or other longer-form information and converting it into structured data without having to inform the software. That is the state of today’s automation.
It gets even more interesting when you apply runtime machine learning on other data types. We can discover other features of data such as format, typical length, and even common words and terms to build a series of algorithmic hypotheses to further reduce the need to configure your automation tasks. This approach is also different in the way it handles changes.
Gone are the issues associated with configuring a system for one set of use cases only to find that the data has changed – the form layout might have changed, or the account number moves from a 10-digit number to a 20 character alpha-numeric entry. The software detects these changes, identifies them as a material change and not just an error, and can adapt.
The software detects changes, identifies them as a material change, and not just an error, and adapts.
Ultimately the promise of applying machine learning isn’t simply about automating document tasks like document classification or data review and entry, but to make the software learn more like a human so that the typical expense and drudgery required to get to a highly automated state is nearly eliminated. We all want the easy button, and we’re getting closer.
