This originally aired as episode 49 of The Assurance Show.
In this episode, we discuss the importance of balance in your work program - covering each of the four objectives:
Conor:
Hello and welcome to the performance audit report. My name's Conor McGarrity.
Yusuf:
And I'm Yusuf Moolla. This episode is an older episode. It comes from The Assurance Show, which is a podcast that has been going for a little while. That podcast was focused a little bit more broadly. So internal auditors, performance auditors, and a few other people that have primarily data as a focus in their work. But this episode is specifically performance audit related. There's about 10 of them that will be putting onto the show from time to time. They are a bit older. Some of them are interviews with performance audit professionals in Supreme Audit Institutions. They will be relevant directly relevant to you as a performance audit report listener. Just to know that they are a bit older. So if you have listened to the assurance show before you may have come across this episode, and so apologies for the duplication, we just think that it's worthwhile bringing that across, into this podcast because of its relevance. Hope you enjoy it. So last episode about forward work plans. we left that at the creation of the forward work plan One thing that we didn't talk about then, that would come before that forward work plan is finalized, is what does the work plan comprise? In relation to ensuring a balance between the different types of objectives.
Conor:
To recap there, we're talking about the four main objectives, economy, efficiency, effectiveness, and compliance.
Yusuf:
And we'll, steer clear, for this discussion anyway, of limited assurance engagements or referrals. And focus on reasonable assurance engagements. Because limited assurance engagements and referrals largely will have a very specific matter to deal with. While your broader performance audit program, would cover one of those four broad objectives.
Conor:
Yeah. So you've gone through your assessment process to arrive at your forward work plan. So your three-year horizon and all those topics. So the next thing you need to consider is the balance of your program across those elements of economy, efficiency, effectiveness and compliance.
Yusuf:
Before we talk about how we do that, what are we seeing, or what have we seen historically, around that balance within forward work programs or within the reports that are produced in terms of the four key objectives.
Conor:
The topics have been heavily skewed towards audits that look at effectiveness and report on effectiveness, as opposed to the other elements we just mentioned then.
Yusuf:
What's the result of that? So what's the impact of the absence of focus on efficiency.
Conor:
Well, not just efficiency. But I guess if we start with efficiency. The primary effect initially for the audit office, is that they're not fully using their mandate, for example, as envisaged by the parliament or the Congress, or whoever gives them that power to do that. Because if they've suggested there are four aspects or elements you need to look at, and you're spending 80% of your work only looking at one of them being effectiveness, then you have to ask: Is that the use of resources, that parliamentarians envisaged, in setting up your organization to do performance audits?
Yusuf:
In some jurisdictions that may lean towards compliance as well. So effectiveness and compliance would be the top two focus areas out of the four. There's a basis for needing to do some sort of balancing, as in you want to be able to cover off on those objectives. What's a reasonable way forward in getting to that balance, given that we have so heavily been weighted towards effectiveness and compliance?
Conor:
It's probably too big of a leap to expect an audit office that's traditionally delivered the majority of its audits as effectiveness audits to automatically jump, to doing a huge number of efficiency, audits, or audits focused on economy. So that's probably overly ambitious and not realistic. However, there needs to be some more movement and acceptance that efficiency audits need to have a greater place in your overall mix of audits.
Yusuf:
Most people that will be listening to this will have an understanding of this, or, pretty good understanding of this topic. But when we are talking about the objective of an audit, there's often multiple objectives. So we could be covering effectiveness and compliance, effectiveness and economy in a particular audit. we're talking about there is moving towards more efficiency as an objective within the audit, either the sole objective or one of the multiple objectives. Is that fair to say.
Conor:
Yeah, it is. And there are probably more circumstances where you'll see efficiency alongside effectiveness rather than efficiency called out just by itself as the objective of the audit.
Yusuf:
And why is that?
Conor:
Quite often, you have to look at aspects of effectiveness when you're determining efficiency.
Yusuf:
Okay. So would there be some sort of overlap in either the criteria or the audit questions that are asked or the evidence that is obtained as between effectiveness and efficiency objectives?
Conor:
The last one you mentioned there Yusuf is probably the most prominent in that the evidence you'd have to obtain to measure effectiveness would also go a long way, if you're looking at efficiency.
Yusuf:
Given that you have that overlap in the design of the audit work program you might be able to cover off on both objectives which would be less than covering off on each objective individually within two audits.
Conor:
Absolutely. So that in itself gives you some efficiencies there on how you deliver your work..
Yusuf:
Sometime last year, we looked at the range of performance audits that were conducted across several jurisdictions, particularly in Australia, and a few other offices. But what we saw was that there was a significant skew towards effectiveness audits, a little bit compliance, efficiency audits accounted for less than 10% of the overall number of audits. And that is where efficiency is either the objective or one of the objectives. One of the things that potentially is holding us back is the fact that, historically, we haven't been doing much of it as performance audit teams. What else is it that deters us from going down that line.
Conor:
Based on the people we've spoken to from various performance audit backgrounds, there seems to be a perception that efficiency audits are more difficult to actually execute than effectiveness audits.
Yusuf:
And that perception, is that based on historically not doing much with it, or is it based on having done it and found it to be difficult.
Conor:
I think it's both. I think there's a bit of the fear factor there. We haven't really done much of them before. We think it's more difficult, it might be getting more into sort of economics territory. We may not have the skills, so there's certainly an aspect of that. Some entities may have tried to do some efficiency audits in the past. And they may not have gone as smoothly as they wanted. So either they took a lot longer to arrive at a report. Or maybe they just couldn't deliver it to the standard that they wanted. So this there's probably a mixture of things going on there.
Yusuf:
What about economy focused audits what's the hold up there. What's preventing us from doing more of those.
Conor:
I suspect the reasons are exactly the same. Although they economy audits don't seem to feature as prominently in discussions about why they're not being done as efficiency audits. I don't know why that is that they're not as prominent in discussions, but most of the discussions seem to focus on the efficiency aspects being difficult. But one would suspect that the reasons for more economy audits not being done are probably similar to the efficiency audits.
Yusuf:
So not withstanding that the performance audit mandate for most offices would cover all four. Is it that economy audits just aren't that valuable?
Conor:
That's quite a loaded question. Again, we'll come to perception. They may not be perceived as being as valuable because there are not that many of them we see around. we have no real yard stick to determine how good they've been or what impact they have had. In the absence of that, evidence, it's difficult to answer that
Yusuf:
But the bottom line is that we should be moving towards a better balance?
Conor:
Before we move on that we need to just mention the fact that compliance audits are still important, but it's pretty well held, universally that they are generally more straightforward audits to execute, because for example, there may well be a preset set of regulations, or metrics that have been well-established and define that we can just go automatically and test against. So they're generally considered to be less risky audits and more straightforward.
Yusuf:
It's almost basic checks and balances that you following your own sort of basic controls and preventing things like fraud and ensuring that you are compliant with privacy expectations and, security expectations.
Conor:
Yeah, black and white rules generally.
Yusuf:
Which, in an ideal world would be covered by internal audit functions within the individual agencies. And then the audit office really should be coming over the top of that. And just checking that across the board. However, we don't necessarily see that happening from within internal audit teams, which is when you need the auditor general to step in.
Conor:
Yep. Absolutely. And it does actually make a lot of sense, even from an efficiency perspective, if internal audit is picking up some of those compliance based stuff, when the external auditors or the Auditor-General comes in, they can see what's been done already and don't need to cover that off in their footprint.
Yusuf:
Okay. So one way that you can improve that is by measuring the effectiveness of internal audit functions the related boards. And there's obviously what we've seen over the last year is that there is more around that. So there are some audit officers that are starting to look at. Or looking at that again. Also because it just means that they can be more efficient as a team and make the whole system. Going back to efficiency then, So let's p ut efficiency and economy into one basket for for purposes of this discussion, we all know what effectiveness audits are. We all know what compliance audits are. What would we do in order to undertake an efficiency audit? What would the objective be? So what are we looking to achieve? And then if you haven't done an efficiency audit before, what are the sorts of things that you need to be thinking about, that will be different to when you're doing any effectiveness orders.
Conor:
So if we go back to our definition, that's trying to maximize the ratio between inputs and outputs. A place to start would be in trying to understand whether there's been historical challenges with where that the ratio as between outputs and inputs looks extreme or looks as if there's a heck of a lot of work on the input side, that doesn't seem to flow through to the output side. So that's trying to understand what that actually looks like in terms of service delivery let's talk about the health sphere, and that's always a target for efficiency, dividends, or efficiencies and savings. Particularly now in this current climate. There's always been concerns, generally held among parliaments and also members of the public that hospitals are very expensive to run. So that's the one thing, but there seems to be a lot of costs Service inputs that don't necessarily deliver a good return on patient throughput. So how many people we're getting moving through the system? So that's an example. Trying to find those types of efficiencies in the health sphere that has been front of mind for a lot of auditors general and various jurisdictions. So how can we get more of our members of the public through the health system without necessarily having all these extra costs, inputs to do that?
Yusuf:
Okay. So in that example, what we're saying is we still want to have patient outcomes and all sorts of governance in place. However, there are certain things that we are likely to find within the inputs that go into running a hospital, for example, or running a health service, for example, that could be eliminated without reducing those outputs or could be reduced without reducing the actual outputs or the outcomes that particular service is providing.
Conor:
One of the pitfalls is that, because there are so many variables, there's so many inputs into a massive system like that. Where do you commence? Trying to find efficiencies. How do you actually identify those activities or inputs that are not providing any value to the organization? And in fact, maybe actually detracting from your ability to do your job. So that's the one thing. So then identifying that that's, a significant global problem, we just try and zone in on a particular function type or a particular service or stream within that environment and focus on that firstly.
Yusuf:
What if you went back to front the one thing that you need to be able to determine is are the outputs outcomes effective, right? So you need to, possibly you want to go first and look at compliance within that. system and then go and look at effectiveness of the outcomes within that system. And then go and look at, okay, we know that they are compliant or they're working towards compliance. We know that they are effective or working towards effectiveness. now that we understand how the system works and what we're trying to get out, now we can go and see what there is that potentially, extra superfluous, whatever you want to call it within the inputs. So working back from the outcomes to what would generate those outcomes.
Conor:
Yeah, and that's a really good way to explain it. So it's almost like a hierarchy of activity. So you might do the compliance thing first, because we want to make sure that people are getting healthcare, according to legislation and appropriate health standards effectiveness might be is the health service being delivered properly? Is the quality there? So people are not leaving our hospital less well than they entered the hospital, that sort of thing. So effectiveness will take care of that. And then once, once we know that those two things are ticked off or headed, where they need to go, then how can we make sure that we're getting that same level of outcome potentially with fewer inputs.
Yusuf:
So that's great because similar thinking would apply to internal audit teams. If you're putting together plan for addressing internal audits. Now, like we said, before, many internal audit teams are going for this sort of three plus nine approach where you determine what your initial audits need to be. And you're reasonably secure on that. And then, do that for three months and then work out what do I need to do next? The certain topics that will remain year on year, just not because we want to have a rolling audit plan of any kind, but because they are so important. So in, certain industries, customer service may be the most important thing or revenue, maybe the most important thing or marketing or sales, whatever it might be. And within internal audit. If you are going to look at what is my work plan going to be, it might make sense to say, let me look at compliance first. And then for that particular topic, then look at effectiveness and then look at efficiency so that we slowly building up in terms of the value that we can provide to those departments or functions or business units. But also because it's more difficult to do efficiency traditionally, If we understand. How the system works through having done a compliance focused audit then an effectiveness focused audit that efficiency audit becomes that much easier.
Conor:
That's a common sense approach. If you've done those compliance audits, effectiveness audits you'll know that organization pretty well, by the third time you come around to do your efficiency audits. Which in itself will help you plan more quickly, deliver more quickly and report more accurately. Because you've already got that solid grounding. So just to turn that on its head slightly, it would almost seem unwise for a performance audit team that probably hasn't done a lot of efficiency audits to go in and expect to do an efficiency audit of, say, a brand new government program. So not only is it brand new and we don't know anything about it, we haven't even yet done a bedrock compliance, audit or effectiveness audit. That would seem really strange. And we'd suggest that teams don't take that approach at trying to do an efficiency audit straight off the bat.
Yusuf:
And unless bringing subject matter experts in.
Conor:
Unless you're bringing the subject matter expert in. But then again, if it is a new program, for example, or a new type of service then how much subject matter expertise is there going to be around the place?
Yusuf:
Oh, so you mean if it's a new type of program, generally, not a new type of program for your government.
Conor:
Correct.
Yusuf:
Cool. All right. So that was episode two of the performance audit mini series and the encouragement we tried to provide within this little discussion that we had is to focus on or try to have a level of focus on audits that fall outside of the traditional. Objective focus, being efficiency and economy audits. And what we ended on there was if you are struggling to understand how to do it, then one easy way to do it is to build it into your work program as a later audit that you use earlier compliance and effectiveness audits to build on in terms of your understanding of how it all works. And getting comfort that it is all working before you try to go and make it cheaper, to actually work.
Conor:
Yup. Sounds like a good plan.
Yusuf:
Good stuff. Thanks Conor.
Conor:
Cheers Yusuf.
