What Are the Options for Document Automation?
Project Goals & Investment Level
There are many different options within the document automation world that cover a range of scenarios from implementation needs such as cloud vs. on-premise, from basic forms processing requirements to complex needs that call for sophisticated natural language processing for contract analysis and integration capabilities. Some may require an SDK vs. a full-featured automation platform complete with user interfaces and workflows.
Given the wide range of options, the most-important questions to answer are: (1) what is the objective of the project? and (2) how much is it necessary to invest? The first question answers the scope of what you need. For instance, if you need to get access to the text within documents to aid with an enterprise search project and the deployment will be in the cloud and integrated into another system that has user management and workflows, then you might select an OCR cloud service such as that offered by Microsoft, Google or Amazon. If you need the same functions, but prefer to integrate it into your own software, then an OCR SDK would most likely support your needs.
In-house, Custom or Out-of-the-Box
If you have similar on-premise deployment and integration needs, but require the ability to locate and extract data from highly variable documents – such as invoices, purchase orders, explanation of benefits, or bills of lading – then you need to answer the second question regarding how much it’s necessary and realistic to invest. Depending on your willingness to invest, you could take an OCR SDK and develop on top of that to meet your requirements. If you don’t want to spend the extra time and money or don’t have the expertise to build this type of software, then a ready-made commercial intelligent capture solution is a better fit.
Generally speaking, the more your requirements point towards the need to locate and extract specific data, the more expensive a custom development project is if you go down the OCR SDK or Web Service path and the more economical it becomes to purchase a more comprehensive set of capabilities. For a more detailed look at building a bespoke solution, check out our eBook, Building Bespoke Document Automation: Options & Challenges.
Intelligent Document Processing Key Components
What Are the Key Components to Consider in Selecting Your Intelligent Document Processing Solution?
If you have the need for more than just transcribing scanned documents into text, then you should be considering solution that fall within the intelligent document processing domain. This software generally includes the following support that is almost universally fundamental to document automation projects: (1) image perfection that overcomes the various quality problems inherent with scanning documents; (2) document classification and separation that is required prior to performing data extraction functions; (3) data location and extraction including support for forms and variable document types; (4) automated data validation using rules and other techniques to ensure uniformity and accuracy.
Depending upon the technical aspects of your project, you might only need an SDK that can provide the above leaving workflows, manual exception handing, and final review to other applications. This would be a common scenario if you wish to add document automation to an existing software solution such as ERP or CRM systems. However, if you need to manage the full document automation process outside of your other systems, then you also should be on the lookout for software that adds the following: (5) user management such as the ability to direct specific documents and verification tasks to specific users; (6) a user interface to allow presentment of documents and tasks to users to verify and/or correct; and (7) workflow management to shepherd the process from capture of documents through manual verification and then output.
Other elements that are commonly desired include: (8) reporting/analytics that allow you to understand the performance of the system in terms of both accuracy and throughput; and (9) application programming interfaces that include REST Services to support integration needs.
How AI Is Used in Intelligent Document Processing and What to Examine
How is Artificial Intelligence Used in Intelligent Document Processing?
This question is probably the most-confusing and misunderstood, primarily due to the challenges of understanding what Artificial Intelligence (AI) is and how it is applied to intelligent document processing. AI encapsulates the full breadth of activities and techniques that can be used to automate a process. These can range from simple rules-based approaches to sophisticated self-learning systems within the machine learning realm. It is likely you are most interested in the latter.
The objective for using any machine learning is to automate a process with high precision without incurring the typical costs associated with manually analyzing data, constructing rules, measuring the results, optimizing the configuration and then maintaining the performance of the system over time. Instead, we expect machine learning to take over these complex, expensive tasks.
Because machine learning is such a hot topic, everyone likes to claim they use it. Another real problem is the belief that machine learning is superior to traditional AI. The reality is that, just like any other tool, machine learning has its own strengths and weaknesses, and even the specific type of machine learning used determines its performance. The most common use of machine learning currently is in document classification where systems are trained on sample data (usually text within documents) that automate the process of identifying attributes. This offers the most reliable way to identify documents.
Document Classification
Automated document classification leveraging machine learning is superior to the traditional method of manually identifying keywords unique to each document type and then encoding them into a set of rules that have to be constantly maintained. If document classification is a requirement in your project, a system that can support both a rules-based and machine learning approach is ideal.
Document Location
More recently, machine learning has been used to locate data. The benefit is similar to that of document classification: remove the need to manually create rules that locate the required data.
The use of machine learning in this area is still relatively new and immature. Some vendors claim the use of “learning” when the process is actually an automated rules-based approach.
Other solutions do actually use machine learning, but the process is very customized requiring a lot of custom development based on training the system. The most important aspect of any claim of machine learning is whether or not it can achieve the intended goal: high performance without the typical high cost of manual configuration and maintenance.