Combining Rule-Based and Model-Based Approaches for Improved Document Processing

Information is power. For most companies, plenty of valuable business information is trapped in documents. Given the variety of document types, sizes, and formats that companies often manage, efficiently processing documents to gain insights can be challenging.

Here at UiPath, we understand this challenge. Owing to our newest document understanding framework, our customers can easily automate data extraction and processing for a wide range of documents despite their type, format, or volume. This helps you approach document processing with flexibility, using whatever process works best for your unique needs.

For a higher-level look at the benefits of document understanding, check out our white paper Increase Operational Efficiency and Mitigate Risks with Document Understanding.

In this article, we’ll:

• Review common document types and classifications

• Examine rule-based and model-based data extraction methods

• Look at the common challenges companies face when applying each of these standard approaches to document processing

• Review the benefits companies can get if both document processing approaches are combined as a multi-approach data extraction method

Let’s get started.

Document landscape

Depending on their structure and format, documents can be classified into three types.

1. Many documents, such as tax forms, stay fixed in format — these are called structured documents.

2. Others, such as contracts, have no standard structure—these are referred to as unstructured documents.

3. Finally, documents that have differing qualities, such as varying layouts or designs, but include similar types of information are called semi-structured documents. Receipts, invoices and purchase orders are common examples of documents in this category.

Based on the classification of documents, there are two common types of data extraction methodologies. Rule-based data extraction targets structured documents, while model-based data extraction is used to process semi-structured and unstructured documents.

Benefits and limits of rule-based data extraction methods

Rule-based data extraction relies on a set of rules to extract data from a document. For example, you can create document templates and apply rules based on specific data position. Alternatively, without having to create templates, you can simply apply rules based on how frequently some data sets are used in a document (occurrence patterns) or how those data variables usually look in a sequence of characters (regular expression or regex).

The former is helpful when dealing with forms that can be templatized, and the latter is used if it is possible and easy to create such rules. We find that rule-based methods are easy to set up and understand, and they work very efficiently in document processing. However, they are limited to structured documents and only in a few simple cases to semi-structured documents.

So, while rule-based data extraction techniques are beneficial in many contexts, they have obvious application limitations. Since template-based extraction is closely tied to a fixed document layout, any changes in the layout can break the rules and require rule reconfiguration.

Similarly, regex-based techniques can be challenging to implement, troubleshoot, and cumbersome as situations become more complex. However, there is an alternative approach to rule-based extraction solutions—a model-based approach.

Benefits and limits of model-based data extraction methods

Model-based data extraction methodologies are based on machine learning (ML). These methods are powerful owing to their ability to learn from a diverse set of documents. We use these methods by employing sophisticated techniques such as natural language processing (NLP) and statistical learning.

The UiPath Validation Station arms users with a human-in-the-loop capability so models can learn on-the-fly and adapt themselves to changes in the data. Artificial intelligence (AI)-powered technology is typically used for data extraction from semi-structured and unstructured documents. We have, for example, created ML models for use in our document understanding framework to address scenarios such as receipts and invoices processing.

Read more: Using AI to Automate Invoices and Receipt Processing

The challenge of using model-based extraction techniques is the time and expertise they can take to create and implement ML models. In many scenarios, though, model-based techniques are superior in their ability to learn and adapt to different document structures and inclusions.

Embracing multi-approach data extraction

There is no silver bullet to address all document processing needs. Both rule-based and model-based approaches for data extraction are potent tools but limited in their abilities to optimally process the range of documents companies manage.

Some structured documents may need much more than just rule-based methodologies as some data cannot be extracted with the help of rules or templates. Likewise, solely model-based methods do not work for all unstructured and semi-structured documents.

We want users to be able to easily combine different approaches to extract information from a single document. So, we’ve designed our document understanding framework to give you the power to overcome limitations imposed by any individual approach. We highly recommend using multi-approach data extraction when you are dealing with complicated documents and want to achieve the highest levels of accuracy during the data extraction process.

Fast and accurate multi-approach data extraction

Using our flexible framework, you can mix and match document processing approaches by simply dropping multiple data extraction techniques directly in your workflow in UiPath Studio.

You can easily configure extractors for data processing, set up preference order for extraction execution, and set a value as a threshold for certain extractor results to be accepted as valid. This way, neither variable document structure nor complicated rules for data extraction will pose a challenge anymore. At the same time, within end-to-end automation, you get faster and much more accurate document processing with the latest AI technology.

Interested?

Having efficient and accurate document extraction and processing capabilities is crucial. Through our emphasis on multi-approach data extraction, we want to make document processing and analyzing as easy as possible for UiPath customers.

Currently, extended Document Understanding capabilities and functionality are available as Software-as-a-Service (SaaS) in a beta version for users involved in earlier pilots. You can expect these features and other advanced Document Understanding tools to be available soon. Meanwhile, we encourage you to sign up for the UiPath enterprise trial to get access to the UiPath Document Understanding solution.