Asking AudRI | robotics at UBS
Data analytics, automation, robotics – technology in all its guises is now an essential fact of life and a vital tool for internal audit in today’s data-driven organisations. However, the key challenge is often not what technology to employ, but how to get people to use it effectively.
Technology alone is rarely the cause of innovation. It can speed things up and dramatically improve the scope and accuracy of data-intensive tasks; however, the real drivers of change are the people who have to use it each day. This is why the UBS internal audit team decided to shift focus to find new ways to help auditors use technology to do their jobs – a case of people using technology to drive audit innovation, rather than technological innovation forcing people to change the way they audit.
UBS first established a data analytics team consisting of experienced internal auditors and data analysts in 2012. Their remit was to make more data available for audit work, and to coach auditors to use this data more effectively. “From the start we knew that it was important that we built solutions that made real business sense, but soon realised that once we stopped coaching and stepped back, many auditors did not fully understand how they could meaningfully use the data to change how they perceived a risk or audited a control,” explains Thomas Nowacki, head of innovation and technology in UBS Group Internal Audit. “Over time, we also saw an increase in expectations from regulators and stakeholders, as well as auditors, for greater use of data and technology to drive impact and efficiency.”
UBS’s response was to create a new innovation and technology (I&T) team, adding project management, technology and automation teams to the data analytics remit. “The creation of the I&T team, with strong support from senior management, resulted in a marked difference in how we managed changes, which are now aligned to the department’s strategic goals,” Nowacki says. “Internal Audit launched a multi-year change programme, with a full-time programme manager, focusing on the holistic changes required to improve how we audit, leveraging next generation tools and technologies.”
This led to a fundamentally different approach to data and technology and the way it began to be applied across the function.
“The data alone doesn’t really matter – you can have all the data in the world, but it’s only useful if it changes the way people do things. We began to look more at what could be automated and to adopt concepts from behavioural science – how do you present information in a way that empowers people to do their jobs more effectively and efficiently,” Nowacki says.
The approach required behavioural “nudges”, as well as data analytics solutions. Nowacki points out that technology and access to data creates dangers as well as opportunities, not just from cyber security threats, but also in terms of human alienation and confusion. People can fear data or the complexity of statistical techniques, or can be so swamped by data that they are unable to use or process it meaningfully, he points out.
“The changes we are making are about enabling people to make the right decisions,” he says. “New technology doesn’t take away the importance of an internal auditor’s judgment and their ability to ask questions. They need accurate data at their fingertips, but they also need to know how to use it – otherwise they can get a false sense of security, or even find that audits take longer because they are sifting through too much data.”
To counter these risks, the team defined and communicated to the department the change programme’s aims and its clear (and measurable) benefits. “Everything we developed had to start with a genuine business problem that needed a solution, not with the technological capabilities,” Nowacki says. “We wanted people to see that everything we built resulted in something useful.”
Right data, right time, right place
Amandeep Khosa, head of data analytics and insights in the I&T team, was put in charge of ensuring that auditors got what they needed to improve the ways they did their jobs, while Steve Taylor, as head of automation, focused on how the right data and tools reached the right people at the right time.
“For example,” Khosa says. “We’re working on how we can make audit planning more efficient and easier. In the past we tried to get as much information as possible to people and then left them to draw conclusions. Now we’re automating more of the background work as standard, even providing a first draft of planning information, summarising the data, providing charts and bringing in comments from risk and methodology experts in the team.”
This first draft not only highlights the key points, but also includes “nudges” that draw attention to elements that may merit further audit attention. These could point to issues that the internal auditor may wish to include in their audit plans, or to underlying features or changes that could be missed. For instance, one area of the organisation may be fully resourced and look superficially straightforward, but if it has had a high staff turnover in the past six months then the report nudges auditors to consider risks around supervision.
“We want to free up auditors by automating the groundwork and enabling them to look more closely at what the data means in practice, or at specific changes relevant to that audit,” Khosa explains. “Our initial focus was to make as much relevant data as possible available to our auditors for audits and risk assessment, but over time we realised that to change behaviour sustainably, we had to derive the right insights in the right way and communicate it to auditors at the moment they needed it.”
Virtual support
Once Khosa’s team deliver the insights, Taylor’s group focuses on how this information can be automated to reach people in the best way. Automated reports and dashboards are part of this, but a key element is a new virtual colleague – a robot called AudRI (Audit Robotic Intelligence). The robot is now eight months old and has already had a significant impact. Auditors can email it questions and it pulls together the information and emails it back to them.
“Most auditors spend a large part of their job using emails,” Taylor says. “So we decided to embrace it. We’d created lots of smart dashboards, but you can’t make people remember to use these regularly, so we made AudRI available to provide the relevant information directly to their inboxes and run routine audit administrative tasks for them.”
Where previously auditors would have to visit different sites or dashboards to find information, AudRI becomes a central source for them. It can answer broad questions such as “How is my audit doing so far?” and email back details of current budgets and phase of the audit, or more specific queries such as “Send me a planning pack for my next audit” or “Can you run this audit test for me?”.
“It can’t do everything yet, but has already had thousands of requests and we believe these have saved over 1,000 person days that would have been spent trawling different locations for data,” Taylor says. “But this isn’t just about saving resources, it’s predominantly about freeing up auditors’ time from dull, repetitive administrative tasks. Some of the requests may only save a few minutes individually, but are done several times a day by lots of people; others may save several days’ work in one go. It adds up pretty quickly.”
When AudRI can’t answer a question, Taylor’s team uses these queries to decide how to develop the robot further. They would also like it to be able to respond to chat requests in future, and maybe even voice – an audit version of Alexa, perhaps?
Internal auditors are also actively involved in designing developments themselves. “An old adage maintains that if you give a man a fish you feed him for a day, but if you teach a man to fish you feed him for a lifetime, and we have focused on teaching auditors to be self-sufficient,” Taylor says. Whenever they see a data-centric task that they believe would benefit from being automated, they are encouraged to build this capability using software called Alteryx. This allows them to build and document audit and risk-assessment tasks, to save these for future use and to share them with others in the team. In the long term, Taylor believes it will replace conventional spreadsheets for preparation and analysis of data.
Once someone has created a new automation they can upload it to a “store”, similar to Apple’s App Store, for colleagues to access.
“We also get AudRI to prompt auditors about all the available automation from the store as soon as they begin an audit, without them asking for it. For example, if they are doing an audit within the investment bank, AudRI will share all its tasks relating to associated risks and businesses,” Taylor adds. “They can choose whether or not to run a test, but it means it’s there if they want it.”
The number of automated audit tests being uploaded to the store is clear proof that people are using the system, he adds. “We’ve seen about 150 created since January this year, so it’s moving fast.” Technical people from the I&T teams perform validation checks on the scripts before making them available to others.
Communication has been vital to the whole initiative. Khosa’s and Taylor’s teams have worked with Claire McGonigal, Group Internal Audit’s communications specialist, to ensure that AudRI and other forms of communications from the change programme are written clearly and are easy to understand. McGonigal has been in charge of making AudRI as human as possible and ensuring that the robot didn’t come over as bossy or threatening – emphasising the benefits it offers and the material ways in which it can help people to do their jobs more efficiently.
“It’s also been about getting the management team to talk about how they use AudRI and Alteryx and what they have achieved through leveraging analytics and automation,” McGonigal explains.
Each month she highlights a team of the month, publicises what they’ve done and the automation scripts they’ve built.
“We’re constantly getting requests about what people would like to see from AudRI in future. It’s such a refreshing change to see ideas coming in as people get excited about the way they could use it,” she says.
To encourage use further, the team is embedding many of the existing tools and methodologies used by internal audit into AudRI and the automated systems. This adds an element of “push” as well as “pull”, because people have to use it to access essential tools. They’ve found that the more people use it, the more they go on to use it in future.
“We’ve had to be clear that in no way is this replacing individual auditor judgment or opinion. It is purely there to help and support,” McGonigal says. “We’ve emphasised this, and shared positive stories from auditors and senior management to help lower the barriers to change. Once people find that using the new tools and technology saves them time, their enthusiasm helps to spread the message throughout the organisation.”
Sustainable innovation
Nowacki says they’ve been delighted by progress so far. “The way we do things in the whole internal audit department has changed significantly in just six months,” he explains. “It’s been a case of lots of small incremental changes and small bursts of activity, supported by strong project management and innovative and technically advanced solutions. And it’s primarily driven by the users, so it’s sustainable change.”
So what happens next? “We’ve gone from some teams using some data analytics to most auditors using analytics regularly to generate real insights,” Nowacki says. “So now we’re looking at how much analytics is enough to ensure they reach the best decision about how to act. You can always add more information, but does it lead to better audits? Are they starting to take longer without having any impact on risks identified or acted on?”
He wants auditors to feel confident in the data analytics output so that they don’t always have to look, for example, at a low-risk area in more depth because the information and insight they have obtained enables them to conclude that further review is not required. Instead, he wants to support them in being innovative and risk-based about their focus and the questions they ask. To do this they will have to maintain responsibility for, and control over, what they look for in an audit, even as more analytical insights are provided to them.
“Our industry is becoming more complex and if we just do what we’ve always done but slightly faster we will start to miss things,” he warns. “That’s why the focus on insights, rather than technology, was a real eye-opener. We are positioning ourselves to continue being a strong internal audit function, fit for a dynamic and digital future.”
This article was first published in November 2019.
