Intelligent document processing: To build or to buy?

Business runs on documents and communications. They underlie almost every process you can think of—anywhere a message (like an email or chat) or document is read or sent. It’s no surprise, then, that the market for intelligent document processing (IDP) is growing 28.9% each year and is expected to hit $17.8 billion by 2032.

IDP typically blends numerous AI technologies, including natural language processing (NLP) and image recognition, to help enterprises rapidly process documents and communications at scale. As business leaders ponder how to use the latest AI tools to drive real value in their organizations, IDP is a great use case to start with for its fast returns and major impact on efficiency. Indeed, Forrester analysis for UiPath shows that technology decision makers see ‘transformative value’ in document automation use cases like knowledge management support (48%), data extraction and routing (43%), and document-based Q&A (40%).

However, this raises the question: to build or to buy? It’s important to assess all options and consider which approach to IDP can deliver the highest return on investment, the best performance, and the fastest time to value. For most businesses, the right option is to buy an existing IDP solution. I’ll explain why:

Build your own (BYO) IDP

With the rise of publicly available large language models (LLMs), and the growing availability of supporting APIs, enterprises have never had more tools to help them build custom IDP systems. However, lots of tools in your toolbox don’t make the actual task any easier.

In a BYO IDP system, each component (from language understanding to data extraction and automation) will need to be built from scratch or, more likely, sourced from multiple third party providers. For example, to supply the necessary NLP component to understand documents and communications, a business might build its IDP system around an external generative AI (GenAI) model from a cloud provider or hyperscaler.

A BYO IDP system provides a business with end-to-end ownership and greater customization potential compared to vendor solutions. They have the flexibility to adapt their system to changing business needs, without having to work with another organization. However, in most cases, these benefits are outweighed by the key challenges of the BYO approach:

Cost

It’s a common misconception that BYO is cheaper than paying for IDP as a service. In most cases, this isn’t true in the short or long term. Developing and then maintaining your own IDP system demands significant time and expensive specialist talent. You need software developers to create the platform and user interface (UI), data scientists for data preparation, pre- and post-processing, and many other experts for tasks like monitoring performance (which you’ll need to create your own reporting dashboards for), as well as auditing and logging. Consider that even third party GenAI models require AI specialists and engineers to fine-tune the chosen model to exact business requirements.

Maintaining your own platform also necessitates continuous updates and resources. Any BYO system that requires data annotation will need training materials for users, and these will have to be updated to reflect your user interface.

Risk

Depending on rare and expensive technical talent to keep a system operational is risky. Due to cost and talent shortages, these teams tend to be small. They may face limitations in the number of use cases and business units they can realistically support. Talent attrition can also make a system non-performant or non-viable in the long term. There’s the ever-present risk that project funding could be pulled as well.

These challenges are compounded when AI skills and model fine-tuning are required. In Forrester’s September 2023 Artificial Intelligence Pulse Survey, 30% of AI decision makers saw a lack of technical skills as among the greatest barriers to adopting GenAI at their organization—the most frequent response.

Complexity

When you build an IDP system, you are solely responsible for complex AI model and platform governance. Indeed, systems built for complex use cases may require hundreds of AI models that have to be managed. For instance, a large bank might need several hundred models fine-tuned for various use cases to achieve the necessary level of accuracy. Even if an AI system can process documents and communications ‘out-of-the-box,' a significant amount of prompt engineering or context gathering is still needed for acceptable performance. Scaling this to hundreds of use cases would be incredibly difficult, as you would have to annotate data, benchmark, deploy, and maintain hundreds of prompts.

There are many hidden costs involved in an IDP system you’ve built yourself. Every component of the system is an important decision, and each technology requires specialist skills and increases technical debt (along with increased risk). Inevitably, BYO is a heavy lift and means slower time to value. Higher lifetime cost is likely due to talent, governance, and maintenance requirements. It’s no surprise that 69% of technology decision makers consider document extraction and routing use cases highly difficult to deploy, according to Forrester analysis for UiPath.

The advantages of buying your IDP system

The alternative to building your own system is purchasing IDP as a service from a third party vendor. There are two main approaches to this:

• Purchasing IDP as a point solution and integrating it with the rest of your enterprise tech stack.

• Purchasing IDP as part of a larger solution or platform. This provides access to additional capabilities, such as automation, as needed.

Purchasing IDP as a service gives businesses less control over platform development compared to a custom-made system. However, platform providers will work with their customers to ensure the system evolves to meet their needs. There are also many other advantages to consider:

Time to value

Implementing an existing IDP platform is usually faster than developing a new one. Established platforms are tried and tested over many years of use in large enterprises across diverse use cases. Training and enablement resources are already created and, in many cases, professional services support is provided to help users hit the ground running and start generating value from their implementations.

Foundational GenAI systems require costly and time-consuming fine-tuning and prompt engineering before they are ready for business use. In contrast, the AI underlying IDP as a service is typically designed around rapid, low-code training experiences. As IDP capabilities, UiPath Document Understanding™ and Communications Mining™ leverage active learning, where ordinary business users and AI models actively collaborate to complete the training process faster.

Reduced risk

Opting for IDP as a service substantially lowers various forms of risk. You are no longer reliant on expensive internal talent to keep your IDP system operational. The vendor owns system maintenance as well as platform and model governance. IDP solution providers are also expected to deliver the highest levels of enterprise-grade data security.

Cost of ownership should also be considered. Hasty development, imperfect code, or the use of soon-to-be-outdated technologies in a BYO IDP system all raise the risk of technical debt. Over time, this necessitates costly corrections and system upgrades.

Purchasing IDP as a service greatly reduces risk and technical debt for the buyer. Vendors prioritize technological upgrades to stay competitive, adopting the latest capabilities and continually iterating and improving their service. They deal with the necessary rework, testing, updating, and replacing obsolete components.

Don’t attempt to build the models yourself. Attention-grabbing headlines make it look simpler than it is. Find a vendor that was doing AI before the LLM hype. Competence in adjacent AI processes is needed such as unsupervised learning, data preparation, and basic ML calculations.

Simon Knowles, Head of Technology, Vabble

Scalability

A custom system requires custom-made connectors and APIs to integrate with the relevant enterprise systems. Depending on the size and complexity of the business, this may require hundreds or even thousands of combined development hours. Established IDP platforms will have pre-made connectors for the most popular enterprise systems, enabling fast integrations and time to value. With a cloud-based IDP system, you can also expect access to the latest AI upgrades and capabilities without eating the cost to build or integrate those capabilities yourself.

For all these reasons, IDP as a service should be the preferred option for enterprises seeking fast time to value, accuracy, and reliability in their IDP system. Custom builds invite considerable technical debt, along with heightened risk and liability in the long term. IDP as a service places the burden of responsibility on a specialized platform honed by years of competition and iteration.

An enterprise-ready platform to scale AI and IDP

UiPath IDP capabilities are the ideal choice for enterprises looking to implement best-in-class AI and IDP safely and successfully. IDP is embedded in our end-to-end automation framework, enabling you to automate document and communications processing. You can also leverage prebuilt capabilities, customize them, and bring in your own or third party components as required.

The UiPath Business Automation Platform™ provides an industry-leading solutioned approach to IDP. We provide extensive platform capabilities and controls over GenAI models, enabling them to be rapidly scaled across the enterprise:

Enterprise-ready

UiPath first party models are governed under the strongest enterprise controls. The UiPath Platform delivers robust rules-based access control (RBAC), model versioning, and extensive performance guardrails for our first-party Document Understanding and Communications Mining models. Human in the loop is also built into our platform experience, ensuring generated outputs are properly validated. 

Furthermore, UiPath manages all third party models through the UiPath AI Trust Layer, delivering governance, trust, and security for GenAI. This means zero data retention and external model training with your business data.

A flexible, open model strategy

AI models are advancing quickly, and the best model now may not be the best model for your use case in six months’ time. We are constantly investing to create the best specialized LLMs for core business processes and are constantly improving them. Our release of CommPath and DocPath for IDP are testament to that.

Yet our AI strategy remains open. We provide the tooling needed to integrate your favored third party or proprietary LLMs, and manage them under the UiPath AI Trust Layer. With UiPath, you have the flexibility to use a combination of the best models in class for every task across your use cases.

Optimization for improved accuracy and reliability

Our IDP capabilities are optimized for advanced data extraction with techniques like retrieval-augmented generation (RAG) and curated system prompting. Context grounding means IDP models that are safer, more performant, and accurate. Our Validation Station interface shows evidence of where the extracted information was found in the document, meaning generated outputs can be easily verified with evidence.

Project-based experience

Our innovations are guided by real, on-the-ground IDP deployments in major enterprises. Our users simply define their document type and can leverage multiple models. They can also evaluate performance, monitor, and manage model versions—all key capabilities for deploying AI, maintaining, and then scaling it across the business.

One customer experiencing the benefits of UiPath IDP capabilities is top-20 mutual insurance company Encova Insurance. To automate claims invoicing, Encova had initially developed its own IDP solution, built around traditional optical character recognition (OCR) and a third party AI model for language understanding. However, once they adopted the UiPath Platform for IDP, they noticed an immediate improvement in accuracy:

When it comes to document understanding, with traditional optical character recognition we were getting 40% through without issues and 30% would be partially completed. With this new [UiPath] process, the success rate is 99%.

Jeffrey Martin, Solution Architect, Encova

UiPath IDP capabilities delivered rapid time to value and greatly increased efficiency in Encova’s finance, sales, underwriting, operations, and customer support departments. In its policy intake program, for example, manual data entry time was reduced by 98%. A 95% reduction in annual processing time was similarly achieved in commercial lines endorsements.

UiPath has also recently been recognized as a Leader in the IDC MarketScape: Worldwide Unstructured Intelligent Document Processing Software 2024 Vendor Assessment. According to the report:

“UiPath views GenAI and LLMs as a critical technology tool and it is increasingly integrating them throughout all phases of its IDP, communications mining, and automation portfolios...Further, UiPath continues to prioritize and deploy enterprise-grade security, privacy, access, context grounding, and control features to ensure that GenAI models and services can be safely and securely utilized in support of mission-critical use cases.”

The report also comments: “UiPath's multi-modal AI focus provides it with the strategic vantage point to continually drive new methods and approaches to maximize the value of traditional (i.e., predictive AI) and GenAI for unstructured IDP.”