Every business is built on thousands and millions of individual processes or workflows. Every customer engagement, invoice sent and product released is part of a measurable process that can be measured, analyzed and improved.
Increasingly enterprise leaders have recognized process analysis as a source of competitive advantage and have adapted data mining techniques to the realm of business process management. According to Deloitte, 63% of enterprise use or plan to adopt process mining software in the near future.
What is process mining?
Process mining is a technique used to discover, analyze and improve business processes using data mining methods. Process mining is seen as an objective and data-driven way to evaluate the real state of processes and workflows in an organization.
Process mining can be seen as the intersection of business process management and data mining. In this way, process mining tools interact with business software systems to extract process information, typically in the form of event logs or business objects, for the purpose of process analysis and business process improvement.
Process mining gives visibility to end-to-end processes, thus providing companies with valuable data that can be used to improve business operations and reach better company performance. In this way, it is also talked about as an x-ray to the company workflows and processes.
Why mine business processes?
To understand the value of process mining it's good to start with the origins of the solution. Process analysis is not a new concept. In the past analysis in business processes was conducted based on manual surveys or interviews of key personnel. The result of the manual analysis may then have been summarized in business intelligence initiatives, as part of an audit findings or seen as part of a process map visualization.
While process analysis has become a proven and established part of effective business process management, process mining resolved some of the key limitations of manual analysis:
- they rely on subjective views of the interviewees or survey analysts,
- they need significant time, cost and team resources needed for conducting analysis,
- you only get a one-time snapshot of processes - instead of continuous results you can track and improve.
Process mining is an automated solution that solves some of the key pain-points of manual business process analysis. It gives a data-driven and objective view into business processes and how they can be improved.
What are typical processes to mine?
Virtually any business processes can be mined from the front office sales to product develepment and the back office shared services. Typically process mining thrives in areas where there is a high volume of repetitive workflows, for example in financial operations and monthly processes.
- Accounting processes - including invoicing, accounts receivable and order-to-cash.
- Procurement processes - including purchase-to-pay and accounts payable.
- Logistics and supply chain - including order management, warehousing and fulfillment.
- Customer service - including customer support and customer on-boarding.
- Human services - including recruitment, employee on-boarding and payroll.
How does process mining work?
Process mining gets the data from, you guessed it right, processes logged in enterprise databases and systems. In this context, a process is a chain of individual events with a clear starting and ending point.
That being said, process mining is not a single-step project and the following stages show how it works in a simplified way.
Step 1. Extract the data
In its core process mining collects and analyses process data based on event logs. This can be seen as a type of digital footprint to each task, transaction and process recorded in your data.
Event logs record, gather and store the events from different information systems such as ERP and CRM that the company has in use. As a minimum requirement for mining a single performed activity or process, event logs need to have a distinct case ID, timestamp, and know what the performed activity is.
Data from event logs are then cleaned and prepared by analysts for further analysis in case there is incorrect, inaccurate, or missing data.
Step 2. Reconstruct the data
Once the necessary data is collected in a nice neat way and freed from false data of any kind, it is ready for the actual “mining”. This is when the process model is reconstructed presenting a graphical view of the process.
Typically process mining software represents event logs in a detailed map of processes, showing variations, pathways and outliers in process flows. In many cases there is an ideal or "happy path" for key processes - and process mining helps visualize the various deviations taken from this path across the hundreds or thousands of times the process has been executed.
Step 3. Use the data for various needs
In this step, freshly represented processes can be analyzed for various purposes. Depending on the goal, you can identify inefficiencies and deviations from the normal "happy path" processes, find previously unknown processes, and analyze potential risks and bottlenecks.
One common use-case for process mining is to identify the root-cause of process variations. In other words, exploring the reasons why a specific process may result in many variations, errors or bottlenecks.
Step 4. Monitor performance continuously
A final step in process mining is the continous monitoring of performance. Even well running processes and workflows can run into unexpected errors, and continous monitoring helps process leaders fix issues before they become major bottlenecks, hold teams accountable for performance, and measure the success of process improvement initatives.
Process mining techniques
Much of the theoretical background of process mining has been defined in the Process Mining Manifesto - which defines three key types or techniques of process mining.
Process discovery: The most adopted of process mining, process discovery, uses event log data to create a process model without outside influence.
Process conformance: Conformance checking confirms that intended process models are reflected in practice. This type of process mining compares a process to an "should-be" process model based on its event log data, identifying potential deviations.
Process enhancement: In this case additional information is used to improve an existing process model, for example, location data, costs, or timing. By enhancing process models, the aim is to allow for advanced analysis in order to improve processes.
Process mining vs. task mining
Task and process mining often go hand in hand and get used together for knowledge work discovery. While both are pretty powerful, they bring better results and higher work coverage when used simultaneously. To fully understand what each of them does, let’s discuss the difference between processes and tasks.
As discussed, process mining brings visibility to end-to-end processes. The process is a logical chain of individual events with a clear starting and ending point. Processes can be divided into subtasks that make a complete set of related work. Purchase-to-Pay, for example, is a large process that runs through multiple teams.
On the other hand, tasks are smaller components of work that occur between different processes and subprocesses. Copy-pasting the data, uploading and downloading the files are typical examples of manual tasks.
Benefits of process mining
We have so far looked at the whats and hows of process mining, which helped to understand why process mining is necessary for the automation journey. If that wasn’t convincing enough, here are more concrete answers for the question “why process mining”. Typically this starts with costs, time and quality but benefits can go well beyond:
- Reduce costs. Automatic process discovery finds the most manual, repetitive, and generally inefficient processes, which are usually of the highest cost for different units.
- Reduce lead time. For most businesses time is money - and a key benefit of process mining is the reduction of lead or throughput time in key processes.
- Improve quality. Process mining can help uncover faults, delays or errors that reduce the quality of goods or service.
- Improve customer satisfaction. The quality of customer service depends directly on the efficiency of the client-facing processes.
- Enable process automation. Coupled with automation tools like RPA, process mining helps to optimize previously inefficient processes.
- Make data-backed decisions. Process mining tools foster data-driven decision-making, closing the data gaps within raw data and analyzing recreated processes.
- Increase transparency. Most of the decisions in big companies are based on experienced senior-level people and their expertise and gut feeling. With process mining, it’s possible to see how the processes are running de-facto.
Process mining use case examples
So far we’ve discussed whats, hows, and now whys of process mining. We can now look at a few more concrete examples of how process mining could bring value to different units/roles within an organization.
- Process discovery. This is the most typical and more generic use case for process mining. Through an analysis of process data, companies can easily find more apparent repetitive and inefficient processes and work on improving those.
- Process improvement. Data insights from information systems allow a better and more effective redesign of the processes, instead of having to conduct countless interviews, surveys, and analyze hundreds of documents.
- Process optimization. The end-state of effective process mining is that processes are continously monitored and optimized. This is possible through the visibility that process mining provides on a continuous basis.
- Root-cause analysis. A fourth use case is simply quantifying and identifying the root cause of key process challenges.
- Conformance checking. Finally, process mining allows you to monitor and continously track processes through conformance checking.
Each of the above use case examples can be applied to a team, business unit or the entire organization. Some common use-cases could include:
- Finance - improving working capital, increasing productivity of the finance team, or ensuring compliance within financial operations.
- Procurement - simplifying approval processes, reducing maverick buying, or identifying errors in supplier performance.
- Customer service - reducing the lead time of customer response, identifying root-causes of issues, improving customer satistaction.
- Logistics - ensuring reliability of supply, optimizing inventory and improving productivity within logistics and transportation.
Building the case for process mining
Once you've established a clear benefit and use case for process mining you can prepare a business case for implementing it in your organization. How this happens depends much on your resources and scope - and this may vary greatly between small, mid-sized and large enterprise businesses.
Build or buy?
As with most enterprise software implementation decisions - one key starting point for a business case for process mining is the evaluation whether it is worth building the solution yourself or buying a ready-made solution from a vendor.
In both cases it is helpful to have evaluated both options together with the enterprise IT organization - as there are pros and cons of both options. There are open source libraries of process mining algorithms that can be adapted to self-built solutions, but in many cases the costs of building a custom solution may outweigh the speed and efficiency of partnering up with a software vendor. Even in the case of buying a process mining solution, you may need to consider whether a process mining team or center of excellence requires some additional staffing.
Team roles and responsibilities
Process mining is a large initiative that can bring enormous benefits if and when used properly. That being said, it also requires quite a level of effort from organizations adopting it. A typical process mining project requires at least the following roles to be involved in it:
Process owner. The person who is responsible for the strategic side of the project. Process owner sets the goals, defines the objectives, and leads the process, providing the team with any help and support.
Database expert. The primary person who derives the data from multiple information systems, checks the quality of the data and hands it over for further analysis. It’s usually someone from the IT or analytics team or both, and he/she can truly be called a “data guru”.
Process mining experts. It’s usually a small team of specialists, where one carries out the more technical part of actual analysis and the other one evaluates the process and finds automation opportunities based on the data insights in line with the objectives initially set by the process owner.
However, in larger organizations the team might comprise more people, thus making their roles even more narrow and specific.
Choosing the right process mining software
There are over 35 different process mining solutions available in the market catering for different types of organizations and clients. No matter what your specific needs, it’s good to keep in mind some key areas to consider:
- Ease of use - process mining software doesn’t need to be difficult to use. Seek screenshots or a product demonstration to see if it looks easy to use or navigate.
- Configurability - some process mining solutions only cater for specific use-cases or require significant effort to configure or integrate with different source systems.
- Analytics and reporting - if you’re looking for enterprise grade process mining, you’re likely to need advanced and trustworthy analytics and reporting capabilities.
- End-to-end visibility (E2E) - as more work is digitalized, you need full E2E visibility of the full process journey across different tools and systems.
- Level of support - whether you are a first time process analyst or an experienced process architect you’ll likely have different needs for support and on-boarding.
- Total cost of ownership - process mining solutions come in a variety of costs and service levels. It’s smart to consider the total cost of ownership including the implementation cost as well as the cost of running on-going process analytics.
💡 Looking for a software solution? For more information see our deep dive on process mining software.
When to request an RFP?
When you're evaluating different software vendors you may consider writing an RFP for process intelligence software.
A request for proposal is a document that you create when you’re seeking out and evaluating multiple bids for one or more solutions or procured items. Typically they are used when you need a solution with more technical expertise or a specialized approach for unique business needs.
RFPs are a common practice in enterprise software, but they are not the only option. You can also explore free or paid trials, proof of concept, or simply request information directly from different vendors. In some organizations, RFPs are simply a common part of a procurement process.
Some critical RFP scoring criteria to consider
- Are you looking for a process mining, task mining, or full-feature process intelligence solution that covers systems, processes, and work?
- Are you looking for a turnkey solution requiring limited implementation or more extensive deployment resources?
- Do you have or need the capabilities to deploy the solution across your organization, or will you require extensive support from the vendor or one of their partners?
- What is your timeframe and urgency for the project? Is it within weeks, months, or years?
- Are you looking for a solution that you can pilot and expand over time across your organization, or do you expect clear, measurable outcomes from the start?
- How important is the solution’s current state, and how much weight do you give the vision and future roadmap of your chosen partner?
- Can you estimate the total cost of ownership of the solution based on the proposal answers, and will it meet your budget requirements?
Limitations of Process Mining
Although process mining can do wonders for organizations and has been proving that for many years, there are still areas where process mining falls short.
Analysis of latest but not real-time data
Process mining analyzes the latest data extracted from information systems but that still doesn’t guarantee the full picture of a company's current performance. The data from event logs are first derived at a point in time, cleaned, and only then are they analyzed, resulting in more “static” offline analytics. As an outcome, traditional process mining tools aren't able to notify about possible process deviations on an ongoing basis.
High initial costs
In most cases the rollout of process mining tools requires a lot of effort and inputs from multiple units and teams that result in very high costs. As an example, an IT team would need to do some upfront development and software integrations before the software can start running.
In many cases integrations have to be created and maintained to all enterprise resource systems. For a company that has more than one ERP or uses cloud-based software in business operations the costs quickly add up.
Heavy reliance on human analysts
Although process mining ultimately aims to facilitate automation of the processes, it is still very much reliant on the human brain and the work of business analysts, data analysts, and IT people. Two big areas where people are absolutely necessary for process mining are:
- Interpretation of the data: once analyzed, insights alone are not sufficient, and a business analyst is needed to interpret that data and find concrete use cases in line with the initial objectives.
- Data cleansing and extraction: data from event logs might be incomplete, incorrect, or duplicated, and data analysts need to take time and clean the data to prepare for further use.
Long time-to-value
Depending on their complexity, different systems take a lot of “preparation” time before they can even start providing event logs. Because it’s a necessary step in process mining, it ends up affecting the whole time-to-value of the projects.
Because of the heavy data integration needs of most process mining solutions, it is not uncommon for implementations to take between 12-18 months.
Consider an alterative - process intelligence software
While process mining is undoubtedly popular - there is an alternative to consider. Process intelligence software, such as Workfellow, can deliver most of the use-cases of process mining without the need for data integration and data mining. Instead it uses advanced data capture technologies and AI to collect relevant business object data directly from the enterprise work stations.
Modern process intelligence software like Workfellow provide the upsides of process mining while also giving some additional benefits.
- Real-time data analysis
Workfellow analyses real-time data revealing the current performance of an organization and alerting in case of any deviations on an ongoing basis. It covers all business work-related applications, systems, and documents, leaving no space for undocumented “shadow” work that might not be covered by event logs.
- No upfront development needed
Process intelligence platform starts running and collecting data right away, without having to do multiple integrations and occupying IT peoples’ time for the data extraction.
- All prior and subsequent analysis is done automatically
Workfellow platform collects the data independently by observing all work-related activities within teams. This is the data that reveals the reality of work so no data scrubbing/cleansing is needed.
Once data insights are ready, the platform automatically analyses them and provides suggestions for further actions, taking up the role of a business analyst and data analyst.
- First results within weeks rather than months
Workfellow needs only 2 weeks before it starts generating business cases and data-backed solution recommendations for them.
Are you interested to finding out more about process intelligence? Book a demo today!
