Democratizing Automation: UiPath and University of Michigan Join Efforts

Photo credit: Marcin Szczepanski/Lead Multimedia Storyteller, University of Michigan College of Engineering

This summer, we learned about the University of Michigan (UM) Computer Science and Engineering department’s efforts to advance the intersection of programming systems and artificial intelligence (AI).

We’ve long held the continued democratization of technology as a strategic pillar. Today, technically oriented business users and power users can design, build, and deploy automations entirely on their own. Still, we have work to do to make automation available to everyone.

Through his research and academic teachings, Xinyu Wang, Assistant Professor at UM, has a goal of building fundamental intelligent programming techniques that are useful in practice. This aligns well with our vision of semantic automation.

UiPath founder and Co-CEO Daniel Dines, unveiled semantic automation in 2021. He noted that robots already do a lot for workers, but robots still lack the capacity to understand the way humans actually do work everyday.

Dive deeper:

Dr. Wang's work has impressed the UiPath team. His recent publications highlight his commitment to democratizing the entire automation space:

• WebRobot: Web Robotic Process Automation using Interactive Programming-by-Demonstration, ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), 2022

• SemanticOn: Specifying Content-Based Semantic Conditions for Web Automation Programs, ACM Symposium on User Interface Software and Technology (UIST), 2022

Watch the SemanticOn video demo.

UiPath is committing to support Dr. Wang to advance his research.

In his own words, Dr. Wang outlines his expanded research objectives and goals:

My research aims to democratize automation such that more and more people around the world can automate their tedious tasks.

A specific line of research I’m currently pursuing aims to democratize web automation: a particular kind of automation that involves interactions between two applications, namely a web browser and a spreadsheet. A great example is web scraping which pulls data from websites to a spreadsheet. Another example is data entry, i.e., filling web forms using local data. They involve a lot of manual work, but this work can be easily automated by software robots. The key problem, though, is that none of these users can write automation programs.

My approach is based on program synthesis that can automatically generate programs from high-level specifications. In other words, users without any programming background can “write” programs as long as they know how to manually perform their tasks. For example, in the context of web automation, my approach generates web automation programs by observing user interactions with the web browser and the spreadsheet. At a very high level, this can be viewed as an AI-based programming assistant.

I believe my research is highly aligned with the UiPath vision of semantic automation, both aiming to democratize automation to a much broader audience through the use of AI. In fact, UiPath and I have been working on overlapping problems and developing complementary techniques, and it makes a lot of sense to join forces going forward.

I’m extremely excited about this great opportunity to achieve broader societal impacts. It’s also a wonderful testbed for the research work that I started doing almost a decade ago. And it's an intellectually challenging problem to motivate new advances in the area of AI-based programming systems. I really look forward to collaborating with UiPath in this space.

What's next?

You can find all the publications Dr. Wang has contributed to on his UM page. We also invite you to join us this September at UiPath FORWARD 5 in Las Vegas. You’ll hear how UiPath customers are democratizing automation across their organizations and stories of Automation for Good from around the world. UiPath leaders will share the future of automation and more on what’s possible with semantic automation.

This blog post was co-authored by Dr. Xinyu Wang, Assistant Professor at University of Michigan.