Public Sector Accounting
Accountability for Results
Making the federal government accountable for results presents some big challenges that require innovative solutions.
FROM: SEP-OCT 2003 ISSUE | BY DAVID HUME
In the world of policy, politics, and government, the oft-whispered but seldom spoken truth is that, despite its airs of certainty, government has little knowledge of the actual effects its programs and policies have on the world, and only a general sense of what exactly Canadians are getting for their tax money.
But by no means should such a statement be used to cast government in a poor light. For most of its history, government has simply not had the tools at hand to measure the effects of its policies. Assembling the necessary data would have been too time consuming and costly to pursue on a large scale. But recent advances in information technology could mean changes.
Sophisticated data-collection and measurement systems will make data collection and analysis far less costly and far more effective. Though the technology will not make this process easy, it will dramatically increase government's capacity to measure results. Over the next 10 years, government will likely have access to a wealth of useful information, ranging from sophisticated quality-of-life indices to the results of individual government programs.
The effects of this development could be far-reaching. Policy makers stand to have better evidence on which to make decisions, and Canadians will have a better idea of what the government is achieving as it spends their tax dollars. The federal government has already identified results as one of its priorities. In his recent budget speech, Finance Minister John Manley said,"Simply put, Canadians want to know what they are paying for. They want results. They want value for money."
Compliance vs. Results
The new focus on results could reshape how government is held accountable. Traditionally, government has set up rules that govern the process of spending public funds and aim at preventing misappropriation, conflicts of interest and corruption. As accountants in the public sector well know, these rules are not meant to be broken. On this model, government is held accountable for how well it complies with the process rules it sets for itself.
The strength of this compliance model is simplicity — either the rule is followed or it is not. Its weakness lies in the ribbons of red-tape that have spread through government through years of rule-making. By contrast, holding government accountable for achieving its goals frees public sector managers to experiment, innovate, and take risks, even tolerating failure and error as they run these risks.
Almost everyone would agree that innovation and experimentation are good things, and that they should be a part of the way government does business to meet the challenges of an increasingly complex world. Making this happen will mean loosening the grip of red-tape on government, but it will not mean abandoning compliance altogether. Government cannot be allowed to pursue its goals without any checks on the means it uses.
Creating the right balance between compliance and results — what some call a performance-based model of accountability — thus becomes the goal.
How Results Reporting Works
A key challenge in developing a performance-based approach to accountability is to provide accurate and reliable information on results. A simple framework provides the foundation for understanding how results reporting is supposed to work and how the new information could be used to hold government accountable.
In an article in the April 2003 issue of the Centre for Collaborative Government's Policy, Politics & Governance newsletter, Don Lenihan, director of the Centre for Collaborative Government, John Godfrey, MP and John Williams, MP, CGA, described the framework in terms of three levels at which government can provide the public with information on what it does: inputs, outputs, and outcomes.
Inputs
Inputs are the resources that have been allocated to meet a department's needs. For example, it will need money for salaries, capital investment and maintenance. By itself, however, information on inputs tells us nothing about the achievement of results because it tells us nothing specific about what was produced with the inputs. For example, if an MRI machine was purchased with money from a hospital budget, how many times was it used?
Outputs
Outputs are the specific tasks that governments perform with their inputs. For example, if $10 million were set aside for capital investment, reporting on outputs would tell us what things were purchased with the money — such as new machinery. Output-based reporting tells us what the money was spent on. However, it tells us nothing about how effective the outputs were in achieving a policy goal. Perhaps the new machines were poorly suited to their task, or perhaps the task contributed little to the goal and did nothing to improve cost margins.
Outcomes
Although the public certainly cares about inputs and outputs, in the end, it is most interested in knowing the impact that a policy or program has had on society and on the government's bottom line. What has the program changed? If officials used $20 million to buy three new MRI machines (inputs), and if 3,000 patients were able to access those machines for diagnostic purposes (outputs), what did the sessions contribute to improving the health of the patients or the community (outcomes)?
A Stumbling Block
Though there are still practical questions about government's ability to assemble information on results, the most significant challenge in developing performance-based accountability is a conceptual one. Establishing that results information is a reliable guide to a policy's or program's impact can be difficult, as the following example shows.
Suppose a mayor is elected on the promise that she will lower crime rates in an area of her city. She initiates a study that concludes drug trafficking is the key cause of crime in the area. Her response is to increase police presence. A year later, another report by the mayor's office shows that there has been no reduction in crime levels. Should the citizens conclude that the initiative has failed? If so, should they hold the mayor accountable for the failure?
The conclusion is not so clear. High crime rates often have more than one cause, which can include poverty, racial tensions, and unemployment. It could be that a change in one of these conditions, such as a rise in racial tensions, has offset the gains that resulted from the increased police presence. Unfortunately, the crime rate was not reduced, even though the police presence may have had a significant effect.
This can also work in reverse. For example, a lessening of racial tensions may lead to a drop in the crime rate. The drop may have no connection to the increased police presence, but, because the crime rate is a key indicator for determining the impact of the police presence, it appears as evidence that the initiative is working.
The problem is that the results or outcomes governments want to achieve are often influenced by multiple factors. Non-governmental organizations, the private sector, and other governments may all be engaged in activities that contribute to, or prevent, the outcome. This raises a number of questions around efforts to provide information on results and holding governments accountable.
Which events or interventions actually caused a particular outcome? If there are multiple causes, can we measure the contribution that an intervention made to producing an outcome? What information do we need to track progress toward an outcome?
Improving Public Debate
The principal arena for ensuring government accountability in Canada is public debate. Elections, Parliament, the Auditor General, and the media all contribute to the ongoing discussion of the government's decisions and how well or how poorly it is making them.
But for even the most addicted political junkie, public debate often sounds like nothing but accusations and artful dodges. Its adversarial climate does not leave much room for close examination of the effectiveness of government programs. Nor does it create a reliable instrument for measuring how well tax dollars have been spent.
An informed, rigorous, and principled public debate of the issues is necessary to make accountability for results a reality. Below are three suggestions that might be first steps toward this goal.
- Make public debate more disciplined
Managing complexity requires ironclad discipline, which public debate sometimes lacks. Public debate tends to jump easily from discussion of information about inputs and outputs — such as how many MRI machines were purchased for hospitals — to conclusions about outcomes — such as how well or poorly the health system is working.
As information about outcomes becomes available, there will be more debate about the factors contributing to them. The inputs, outputs, and outcomes framework should be rigorously applied once that debate begins. This will clarify what conclusions are permissible to draw from the information and will ensure the debate does not become misleading.
- Embrace learning and error
In an environment that involves the spending of public funds, admitting a mistake can be very difficult. But mistakes present learning opportunities. Even though the desired outcomes of a given program are not achieved, the program may not have been wasteful.
New questions need to enter the language of public debate. These include questions about what barriers were encountered in implementing the program, how they were dealt with, and, most importantly, what kind of learning came from the experience.
These questions demand a different approach from those meant to hold government to account. The media and the Opposition are not known for asking, "What have you learned?" when faced with government mismanagement. A change is necessary to move forward.
- Set a new standard on the clarity of policy goals
Deciding what information is needed to provide a reliable report on the results of a policy or program is a complicated task. Government must examine, assess, and record the impact of a program on people and communities. Officials must decide what standards they will use to evaluate the program.
In theory, such choices should be guided by clearly stated policy goals. If the goals are not clear, officials will have no reliable basis for deciding what indicators should be used. It will be difficult to hold government accountable for the results because there will be no clear statement of what it was trying to achieve.
But some commentators maintain that vague policy goals are not always a bad thing. They can, and often do, serve a constructive end. Parties who disagree on a clear goal may reach agreement on a vague one. This allows a process to move forward that otherwise might have become bogged down in political differences. Moreover, results that are tied to a vague goal may provoke debate leading to greater clarity.
However, the general rule remains sound: clearly defined policy goals are an important standard in holding government accountable for results.
Making it Work
Government accountability involves a complex mixture of institutions, processes, people and information. Over the next decade huge amounts of results information will become available. If the information is of high quality, it could be used to strengthen accountability in a variety of forums, including Question Period, parliamentary committees, the news media, and election campaigns. It could make a dramatic impact on how Canadians see their taxes being spent. There is the real possibility that they could finally get a satisfying answer to the question, "Where did our tax money go?" Making it work well is in all of our interests.
[ TOP ]
David Hume is a researcher at the Centre for Collaborative Government, a public interest research organization located in Ottawa.