Academics Explain Process Mining: Josep Carmona on Conformance Checking

Process Mining can traditionally be divided into Discovery, seeing what the as-is-process looks like, Conformance, comparing this as-is-process to a desired to-be world, and Enhancement, identifying the necessary areas of improvement and implementing changes in the future process. Successful data-driven transformation depends on a well-balanced interplay of all of the three components (Discovery, Conformance, Enhancement). In particular Conformance Checking is an unavoidable linking step to turn insights from x-ray-like Discovery into process improvements and to identify undesired bottlenecks and deviations. Conformance Checking forms a science and discipline of its own within Process Mining research. Josep Carmona, Associate Professor at the University of Catalunya and active partner of the Celonis Academic Alliance, researches core questions at the essence of what Process Mining is. Furthermore he is a member of the Process and Data Science UPC group and ALBCOM research group. A particular focus of his work is on Conformance Checking working on the interface of Process Mining and Modeling. We asked him to lift some of the secrets around Conformance Checking for us and explain the role of Conformance Checking within the Process Mining universe to us!

1. How did you first get in touch with Process Mining and what fascinated you about it?

Back in 2008 I was working on a topic closely related to process discovery (region-based synthesis of Petri nets from automata), and my group interacted closely with Wil van der Aalst, who pointed out the challenges of Process Mining. We found that the research we were carrying out could be applied in the Process Mining field, and besides discovery, we started to look to other problems like estimating precision, decompositional techniques for discovery/conformance, and many more. What fascinated me was the fundamental problems that were tackled by obtaining process models from real process executions, and the multiple real-life applications of these topics.

2. You do a lot of research on the interface of Process Mining and Modeling. Can you explain the difference between the two disciplines and how they complement each other?

Process Mining and Modeling are complementary disciplines. While in Process Mining, the human factor is relegated, the human part in modeling is still very important. Conformance Checking lies at the intersection of the two: a process model can be manually crafted, and Conformance Checking results can be automatically derived once the process model and the event log are in place.

3. Alright, so we already understand that Conformance Checking is a way to combine insights obtained through Process Mining with models. But what exactly is Conformance Checking?

I love to answer this question! Conformance Checking is a discipline to relate modeled and observed behavior. The former may come from manually crafted or automatically discovered process models. The latter from the footprints left by process executions in an organization’s information system known as event logs. With Conformance Checking, one may know if the real process is functioning as expected or modeled, and take remedies in case it is not. Conformance Checking further allows to project real data on process models, so that they become rich and insightful sources of live information!

4. So Conformance Checking works a bit like a sanity check for my process by comparing as-is and to-be worlds. How exactly does it do that?

Conformance Checking solves a fundamental problem: aligning a real process execution to a process model. Algorithmically, this can be done from different perspectives and guarantees, ranging from practical but incomplete techniques like token-replay, down to complete but complex techniques like alignments. On top of this problem several applications arise like computing the level of observed behavior that is present in the process model (fitness) or the level of modeled behavior that is actually observed in the event log (precision). As in classical data science, fitness and precision give useful quantitative insights on the relation of modeled and observed behavior.

5. You mentioned token-replay and alignments. Can you explain in easy words what different types of Conformance algorithms exist and what they mean?

Different methods are used to obtain a process model as the key component for Conformance Checking. Both techniques - token replay and alignments - aim to find a model run that mimics an observed process execution. However this model can embed infinite behavior, i.e. it can allow for an endless number of different process executions, which makes the task quite difficult. If the trace is a possible model run, then you would like to confirm that the model can actually replay it without problems. Token-replay is a heuristic technique, i.e. it does not always guarantee that the obtained model run will be the closest to the observed trace. This is due to the fact that token-replay takes local decisions, which prevent in general to see the “optimal path”, i.e., the most similar path from start to the end of the process model. Optimal alignments, in contrast, perform an exhaustive search of the model’s behavior, and hence guarantee returning the model run that is closest to the observed process execution. A nice analogy that tells the difference between token-replay and alignments is searching for a particular place (e.g., a restaurant) in a city: in token-replay, you decide the direction to take just by looking at what you see. With alignments, you take your mobile phone and look at Google Maps, which will tell the optimal route (but pays the price of connecting to a GPS, downloading the city map, etc …).

6. Once we have a model and can compare it to the as-is process what are all the things that Conformance Checking can do? Can Conformance also be used to analyze other factors such as e.g. whether certain persons conducted a process step as required or the time between two process steps is as required?

Once the process executions are aligned to a process model, trends on the execution can be extracted as you suggest. For instance, given a choice between two payment methods A (credit card) and B (cash), one can see that in reality persons that do A are always performing a subsequent required step which is verify the credit card number, as required in the process model. Conformance Checking will reveal when this situation is violated. Similar performance analysis between process steps can be performed.

7. So it seems like Conformance Checking is already quite a well-established field in academia. How well is Conformance Checking currently adapted within industry? Are there any factors that can facilitate or hinder the integration of Conformance Checking into the commercial use of Process Mining?

Process Mining tools are starting to introduce some form of Conformance Checking, but I believe we will see more mature techniques in the years to come. If infinite resources were available, then optimal alignment techniques could be incorporated by any commercial tool... However, since this is not the case (quantum computers are still not available in Amazon ;-) ), reliability of the feedback provided by Conformance Checking is still not 100% accurate. The research community working on algorithms for Conformance Checking is pushed to solve a real problem that may impact considerably the industry in the near future.

8. You said that Conformance Checking will have a greater industry adoption in the future? What do you think the Conformance Checking of the future and innovation for this field will look like?

Complexity is a clear problem that is in the focus of research nowadays. However this is not the only challenge. Multi-perspective Conformance Checking techniques, online techniques, stochastic techniques, sampling techniques are examples of innovative variants of the fundamental problem that have received attention recently, to name some examples.

9. Which changes will we see in the way Conformance Checking is practiced (also within industry) in the next years?

Any of the examples of innovation (Multi-perspective Conformance Checking techniques, online techniques, stochastic techniques, sampling techniques) are likely to make a big change in Conformance Checking practices. For instance, multi-perspective techniques allow a 360º analysis of deviations, not only control-flow: in multi-perspective alignment checking, a process execution can be labeled as deviant if data is not satisfying the rules expressed in the data-enhanced process model. In online Conformance Checking, one can work with live (i.e., non-finished) process executions instead of post-mortem analysis. This picture (obtained from Andrea Burattin) explains the opportunities of online Conformance Checking very well:

10. Can you give us a recent example of some of the research that you did within the field?

In the last years we have been proposing different approximate/suboptimal techniques to compute alignments. Most of them are in the thesis of Farbod Taymouri. Recently, together with Lluís Padrò we got interested in applying relaxation labelling, a technique original from computer vision, for the computation of alignments. The last advances we have made show it is a very efficient technique that can guarantee the derivation of alignments, sometimes close to the optimal ones. 

11. If there is one key message you would like to share with academic and industrial process miners, what would it be?

For research process miners: go back to the answer of question 9, pick one of the challenges, and work in a very impactful problem. You will have fun!

For industrial process miners: simply think about all the possibilities Conformance Checking may provide in any process improvement initiative. For Process Mining software vendors: Keep close ties to innovation and research that is happening in academia (like Celonis does with its Academic Alliance programme)!

Thank you very much for the interview!

If you want to learn more about Conformance Checking check out Josep Carmona et al.’s book on Conformance Checking - Relating Processes and Models.

Josep Carmona

Associate Professor