The Performance Audit Report

22 | Engaging citizens in performance auditing

January 25, 2023
Episode Summary

In this episode, we discuss how audit offices are engaging citizens in performance audits.

This summary of the five ways audit offices are engaging citizens is an excerpt from the full article here.

1. Forward work plans – consulting broadly for focus areas 

Many offices invite members of the public to suggest what topics or areas they should audit. They generally do this through their website, rather than via any broader advertising (e.g. through media advertising). However, it is possible to use a more focused approach to gathering public input on your forward work plan. 

The New Zealand Office of the Controller and Auditor-General took a direct approach to community consultation in developing its 2022 Annual Plan. After developing its draft work programme, the office surveyed New Zealanders to get their feedback on it. Respondents were asked to consider ‘performance areas’ the Auditor-General should prioritize. Feedback from the community led to the auditors adding a new PA topic to the work program (the Government’s progress to reducing child poverty).  

In Canada, the Auditor General of Alberta invites members of the public to provide input into potential audit topics by asking the broad question: 

  • Given your personal experience with government programs and services or what is most important to Albertans, what should we focus on and why? 

 Members of the public are also encouraged to provide comment on potential audits identified by the auditor general.  


2. Refining the broad focus areas for specific citizen concerns 

 As part of their strategic planning, some audit offices develop areas of focus or broad themes they intend to evaluate over the coming period. Some offices maintain a ‘watching brief’ on specific areas that helps inform their priorities.  

For example, the New York City Comptroller maintains an ‘agency watch list’ that spotlights city agencies that raise concerns due to rapidly increased spending and ‘meagre measurable results’. An agency that was on that watchlist was New York’s City Housing Authority. 

The Comptroller’s performance audit of the City’s Housing Authority took a ‘resident powered approach that included a survey of residents, visits to housing developments, listening sessions, participation in family days and ongoing engagement with residents. The aim of this engagement was to help focus the audits on the issues that matter most to residents. The survey used to identify residents’ priorities was made available online in the top languages spoken by Housing Authority residents and canvassed resident’s ideas for specific audits of the Housing Authority’s operations. The auditors will follow up with residents to ‘make sure recommendations are implemented and real change is made.’  


3. Audit planning – citizen input on audit scope 

Input from members of the public and representative organisations is an approach many audit offices take during the planning phase for their audits. This can involve a mixture of techniques to gather information in support of the audit. 

In the Philippines, the Commission on Audit has a dedicated focus on Citizen Participatory Audits.  

Civil society organisations (CSOs) may also work with the auditors in conducting audits. Citizens and CSOs are invited to get involved in the audit design and to formally join the engagement team. They then work with the auditors to develop the audit plan, assign responsibilities and conduct the audit through fieldwork, reporting and implementation of recommendations.  

The participatory audits are described as ‘a strategy to uphold the people’s right to a transparent government…built on the premise of that public accountability prospers with a vigilant and involved citizenry.’  

 The City Auditor of Austin (Texas) has taken a community engagement approach to its performance audit of the city’s delivery of animal services. It has developed an online survey for members of the public to provide input to the audit on matters such as the most important areas the audit should review, what skills an SME on animal practices should have and other suggested aspects for review.  

Some audit offices engage the cohort most impacted by the public service being audited. Audit Scotland’s PA on improving outcomes for young people through school education took this approach. Auditors worked with a panel of the country’s young advisors to help shape the scope of the audit and design questions for surveys and focus groups. Young advisors also assisted in facilitating some of the focus groups. 


4. Audit execution – seeking agency staff views during fieldwork 

 Sometimes auditors survey the audited agency staff as part of their evidence gathering techniques. They do this to get first-hand perceptions about the organisation or the services it delivers.  

One example of this is the (U.S.) District of Columbia Auditor’s survey of police as part of audit of sworn police officer staffing needs.

Another example is Washington State Auditor’s assessment of workplace culture at its department of Fish and Wildlife. The survey was sent to all departmental staff, receiving a response rate of 45 percent.  

Sometimes the audit office will survey a number of key individuals or agencies within a specific sector as part of gathering evidence. The Queensland Audit Office’s (Australia) report on managing public sector workforce agility took this approach. Evidence gathering included the development, issue and analysis of all twenty government departments in the state. Survey results were then incorporated into the public report. 


5. Reporting – tailoring reports for citizens and civil society organisations 

A relatively simple way to engage citizens.  

The idea is to produce reports that citizens can consume easily.  

This includes: 

  • Audit report infographics.  An example of this is the one page ‘Report Highlights’ that accompanies the Healthy Eating in Schools report produced by the Office of the Auditor General in Nova Scotia (Canada).
  • Audit report summaries.  The Pacific Association of Supreme Audit Institutions (PASAI) suggests producing citizen-friendly audit report summaries and/or delivering presentations on audit findings. It has also developed a good practice toolkit to help its members summarise an audit report into a single page for engagement with civil society organisations. 
  • Alternate media summaries. Examples include the videos produced by the Auditor General of British Columbia (Canada) and the podcast episodes produced by several audit offices, including Audit ATX produced by the Austin Office of the City Auditor (Texas, United States). There are a few others outlined in this article.
  • Reports written in plain English.  An example is the ‘easy read summary’ that accompanies the Audit Scotland report Tackling child poverty”. 

Episode Transcript


Hello and welcome to The Performance Audit Report. I'm Yusuf.


And I'm Conor.


This is the first episode that we are recording in 2023. We released one a couple of weeks ago, but that was an older one. This is the first real one for 23. Today we wanna talk about engaging citizens in performance auditing.


So we want to talk about this topic in particular because we're seeing more and more that performance auditors are engaging citizens, in their work, and it can actually improve audit impact. Decision makers responsible for implementing our recommendations will sit up and take notice if they can see how close citizens were to the audit process and to the outcomes. Audit offices' public reports are increasingly being consumed by members of the public. So it therefore makes sense to involve them in the audit process. So that's what we're gonna be talking about today. We've got an accompanying, little article. How do they do it? Citizens views and their input feature to varying degrees in our performance audit work. Some offices do it when they're selecting audits for their forward work plans. Some seek citizen involvement when planning individual audits. And a lot of them when they're crafting their audit reports. Obviously none of these are mutually exclusive and some audit offices combine all three approaches. So we'll be talking about five ways, that audit offices are engaging citizens, and we'll talk through some brief examples. And then we'll conclude, by describing some of the challenges for performance auditors to consider.


So the first of those, forward work plans where the team consult broadly or the team consults on broad focus areas. Many offices allow members of the public to suggest what topics or areas they should audit. And that generally happens through their website. Where the website will have a button or link or contact form or something else that says, please contribute to this article. But it is possible to use a more focused approach to gathering that public input. The New Zealand Office of the Controller and Auditor General recently took a direct approach to community consultation when they developed their last annual plan. This involved surveying local citizens to get their feedback on their draft work program, and survey respondents were able to look at different areas that the auditor general could prioritize in terms of performance, looking at performance. And that directly related to a new topic being added to their work program. And we've got details of that in the article. In Canada, the Auditor-General of Alberta invites members of the public to provide similar sort of input and they're encouraged to provide comment on potential audits that have been identified by the AG. So this is a a bit more proactive, outreach, and far broader than uh, an individual audit.


So sticking with forward work plans for number two, Yusuf, you mentioned there the broad approach, asking broad questions with examples. Another way in which citizens are engaged is where Are, they're involved in refining the focus areas for specific citizen concerns. So, as part of their strategic planning, some audit offices develop areas of focus or broad themes they intend to evaluate over the coming period. So some offices call them Variously a watching brief on specific areas that helps inform their priorities. The New York City Comptroller is an example of this. They maintain an agency watch list that spotlights city agencies that raise concerns due to, for example, rapidly increased spending or poor performance results. And a recent example of one of the agencies on their watch list in New York is the city's housing authority. So when it was actually conducting a performance audit of that authority, it took what it called a resident powered approach. Another example would be in the Philippines. So where their Commission on Audit has a dedicated focus on what they call citizen participatory audits. They also involve civil society organizations that actively get involved, so these are generally not-for-profits. And these participatory audits are described by the Commission on Audit as a strategy to uphold the people's right to transparent government.


Okay. The third one is straightforward. This is citizen input into audit scope as part of audit planning. So we said earlier that sometimes, audit offices will ask for input from members of the public, either before or during planning. As an example, this would be a button or form on the website. It could be, something like a, social media post that informs the public that an audit is coming up and that they can participate in, that and broadly it's about getting, public input. An example of that is the City Auditor of Austin in Texas. One of their recent audits was on the city's delivery of animal services, and they developed an online survey for members of the public to provide input to the audit. That then helped them to determine what the most important areas that the audit should look at would be and potentially other aspects for review. Sometimes, audit officers will, consider who is most directly impacted by a public service, that they're looking at. As an example, Audit Scotland did a PA on improving outcomes for young people, through school education. And they, talked to, young people as advisors to help shape scope of that audit, designing questions for surveys, focus groups, and actually facilitating some of the focus groups as well.


So the fourth example is a little bit different in that this is where performance auditors seek the views of agency staff, during the audit as opposed to citizens more broadly or members of the public. A lot of the time performance auditors survey audited agency staff as part of their evidence gathering techniques. They do this, for example, to get perceptions about the organization or the service that's being delivered. An example of this, The District of Columbia Auditor's survey of police in one of its recent audits in relation to a review of police officer staffing needs. Another example is the Washington State's auditors assessment of workplace culture at its Department of Fish and Wildlife. So both these performance audits used a survey of staff, as part of gathering information. Sometimes the audit, may survey a number of key individuals or agencies within a specific sector. An example would be Queensland Audit Office in Australia. Its report on managing public sector workforce took this approach, and in this example it did a survey of all 20 government departments in the state.


They used some of the survey results in the actual public report as well. Interesting. Okay. And then the fifth one, is about reporting. So this is not getting, citizens involved in planning or scoping way upfront, but this is about considering and improving citizen engagement by producing reports that citizens can read. So couple of things that this means. One is the report is written in plain English. An example of that is the Audit Scotland report on tackling child poverty. Another is using, or summarizing the results of the audit in a way that is easy to digest. We could also include infographics in the audit. So the Office of the Auditor-General in Nova Scotia, in Canada. And I think we spoke about this to Michael Pickup from British Columbia. The report highlights that come out of the Office of the Auditor-General in Nova Scotia as well. And then, the last one there is, just different ways of summarizing the report for different audiences through alternate media. So, Michael Pickup the AG in British Columbia produces videos. A range of audit offices, produce podcasts, so, Audit ATX from Austin in Texas and then there's a whole bunch of others.


We just described there the five categories where we see the growing use of citizen engagement, or members of the public, or even individuals working within agencies. But when, you're engaging people in your audit, there needs to be clarity on a few things. There needs to be clarity on the role of performance auditing, on the importance of objectivity and balancing competing views, and also the scope of performance auditing. So firstly on clarity for citizens on the role of performance auditing. So we are engaging them as part of our work, but it's vital that they understand the scope and limitations of the performance audit function. So as an example to this many audit offices performance audit mandate cannot query the merits of government policy. And this is a very important limitation for members of the public to understand but it can be quite difficult to navigate and apply in practice, even for auditors So, make sure this limitation is clearly communicated to members of the public.


So that's a broad office limitation. If it actually applies to your office. I know there are some offices in Europe, for example, where they do look at the merit of government policy.


Yeah, that's right. The second consideration is there needs to be clarity for citizens on objectivity and independence and balancing views in your work. Now, this all seems like bread and butter stuff to us, but if we're engaging citizens in performance audit, they're only gonna be one several voices that we need to listen to, and it's important that they understand this and the need for us as auditors to maintain objectivity with no perception that, for example, their views are being given greater weight than other sometimes competing views. So we all know people like the stakeholders and engaged citizens can get very passionate, for example, about certain things, and that's great. At the end of the day, we need to be balanced and promote fairness and mitigate any perception that calls into question our independence. So regular, open and consistent communication with citizens and other stakeholders will help provide this clarity. And the third consideration is we need clarity for citizens on the scope of the particular audit itself. So regardless the importance or potential impact of a particular audit, it's never gonna be the complete solution to concerns raised by citizens or service delivery issues or program risks. Again, we need to be very clear from the outset. It is just one of a number of ways in which we can address service delivery issues, but it might not be able to 100% address an individual's or a group of individuals' concerns about a service.


As we went through there. There's many benefits in engaging citizens in performance auditing. The World Bank and INTOSAI and Supreme Audit Governing bodies or coordination bodies, have been talking about this for some time. Consensus that it's important, needs to be thought through carefully. There's a whole bunch of real life examples and we'll have them in the article. And there's obviously a range of resources and references from INTOSAI the World Bank, CAAF others PASAI more recently, particularly the PASAI article is about civil society organizations. So a slightly different view. The resources are available. The examples are there. Follow the links on our article, and you could, get it all in one place. Thanks, Conor.


Thanks, Yusuf.

Listen to More Episodes Like This

Conor McGarrity
Your host

Conor McGarrity

A specialist in performance audit, Conor is an author, podcaster, and senior risk consultant with two decades experience, including leadership positions in several statutory bodies.
Yusuf Moolla
Your host

Yusuf Moolla

Podcaster, author, and senior risk consultant, Yusuf helps auditors confidently use data for more effective, better quality audits.
PA Reports Logo
Smart performance audit research.


Latest Podcast Episodes

New episodes of The Performance Audit Report are released fortnightly. Subscribe now to get notified on release.

Submit a Brief

Tell us about your audit topic/title, objective, keywords, keyphrases, and any other details we may need. We’ll confirm the details with you within 2 business days so we can deliver your research working paper within 5 business days.
Submit a Brief