Manal Laghmouch - Towards Full Population Testing in Auditing: How Many Process Deviations Should Be Labeled?

Conformance checking allows auditors to detect process deviations automatically, resulting in numerous deviations, with only a few being relevant. Identifying notable items amidst this large data set is challenging. Machine learning techniques offer potential solutions, but questions about the required number of labeled deviations and the impact of label quality remain. Our study investigates these factors' effects on Decision Trees and Random Forests. Results demonstrate these models' effectiveness in identifying notable items within imbalanced deviation populations. Achieving 90\% precision and recall is feasible with about 400 to 600 labeled deviations, depending on the notable items' population fraction. A higher fraction of notables reduces the required labeled deviations. Varying label quality produced similar results. Additionally, classifications identifying at least 90\% notable items are linked to less complex processes.

Hi! I'm Manal, a passionate fifth-year PhD researcher delving into the intriguing world of AI in financial auditing. Imagine the challenge auditors face when sifting through thousands of transactions to ensure the reliability of a company’s financial statements. In our tech-driven era, my research explores a game-changing approach – stepping away from traditional sampling and embracing automated testing for all financial transactions. But here's the catch: the automated approach floods auditors with alarms, while the auditor is genuinely interested in, let's say, 5% of them, as they might impact the financial statements. My project’s mission? Untangle this alarm overload by crafting an efficient solution for investigating each alarm and distinguishing true alarms from acceptable exceptions. I'm developing an interactive machine learning technique, paving the way for auditors to focus their valuable time on alarms that do really matter.