“One year later, we’re focused on delivering for Canada’s middle class, and those working hard to join it.” PM Justin Trudeau October 18, 2016
Within the Government of Canada, improving performance and delivering results is central to every department and agencies’ core mandate. The uniformity of the approach is best exemplified in the new Policy on Results and Directive on Results (July 1, 2016) which applies to all Ministries and the shift to a Results and Delivery Approach by the Trudeau government.
The Expected Results of the Policy are very concise and achievable, as spelled out in Section 2:
3.2.1 Departments are clear on what they are trying to achieve and how they assess success;
3.2.2 Departments measure and evaluate their performance, using the resulting information to manage and improve programs, policies and services;
3.2.3 Resources are allocated based on performance to optimize results, including through Treasury Board submissions, through resource alignment reviews, and internally by departments themselves; and
3.2.4 Parliamentarians and the public receive transparent, clear and useful information on the results that departments have achieved and the resources used to do so.
The new Policy on Results is of significant importance, replacing three substantive Treasury Board policies – the Policy on Reporting of Federal Institutions and Corporate Interests to Treasury Board of Canada Secretariat (2007), the Policy on Evaluation (2009), and the Policy on Management, Resources and Results Structures (2010).
Along the public sector management continuum, the Policy covers the full spectrum of planning, reporting and evaluating departments activities and performance. It introduces new tools and matrices, requiring clarity in identifying and choosing the right metrics based on three principles – (1) Is it meaningful? (2) Is it moveable? (3) Is it measureable? – and placing these within a Departmental Results Frameworks (DRF).
The clock is now ticking on changes that have to be implemented by Canadian Ministries and Departments. While departments continue to implement, maintain and use the Program Alignment Architectures (PAAs), Performance Measurement Frameworks (PMFs) and Performance Measurement Strategies (PMS), these all must be replaced by November 1, 2017 with the new instruments – Departmental Results Frameworks (DRFs), Department Program Inventory (DPIs) and Performance Information Profiles (PIPs).
Structural Aspects of Results and Delivery
A strong element of deliverology is to try to focus on organizational results; to effect this requires structural changes. Within the Canadian government, at the central planning unit levels, in the Privy Council Office (PCO), there is an Assistant Secretary to the Cabinet, “Results and Delivery”. At the Treasury Board Secretariat, the responsibilities have been unified under an Executive Director, “Results, Performance Measurement and Evaluation”. These individuals have the authority and responsibility to ensure implementation of the Results and Delivery approach within the bureaucratic structure. The PCO is developing new templates for federal Ministers for reporting and communicating, with a results annex.
Under Section 4.2 of the Policy on Results, Ministers and Department Heads are responsible for presenting and obtaining approvals from the Treasury Board for the initial Departmental Results Frameworks and for reporting on Departmental Results. The DRF sets out the departments’ Core Responsibilities, Departmental Results and Departmental Results Indicators. Deputy Heads/ Ministers, among numerous other responsibilities, are responsible for establishing, implementing and maintaining the Departmental Results Frameworks.
Importantly, with the focus on Results and Delivery Approach within each department, a head of performance measurement must be appointed to establish, implement and maintain the Department Program Inventory (DPIs), overseeing Performance Information Profiles (PIPs). In most cases, this will take the form of Chief Results and Delivery Officers (CRDOs).
Key Processes in the Results and Delivery Approach
Accountability is central to the Results and Delivery Approach. For the system to be effectively implemented, it means having comprehensive organizational, performance measurement and evaluation information, along with individuals assigned with the accountability for each Program in the Program Inventory. Accordingly, the Policy requires the appointment of an official who is responsible for establishing, implementing and maintaining the program’s Performance Information Profile, including ensuring data collection for it.
The Results and Delivery Approach embraces a clear planning approach, robust monitoring and vigorous departmental evaluation in the form of a five-year, rolling departmental evaluation plan. The information and processes effected will underpin reports to Parliament and provide critical performance information for resource alignment reviews.
Source: M. Barber, P. Kihn, A. Moffit (2011). Deliverology: From Idea to Implementation. P.33.
An emphasis on performance and results and delivery means having a substantive and valid evaluation system, as well as independent verification systems. Deputy Ministers (DM) must establish and chair the Performance Measurement and Evaluation (PM&E) Committee to oversee departmental performance and evaluation – (replacing Departmental evaluation committees). Further, it is required under sub-section 4.3.10 that departments establish and maintain a robust, neutral evaluation function and a designated head of evaluation, responsible for leading the evaluation function and who has direct, unencumbered access to the DM or Deputy Head of Department. Consequently, it could be concluded that one of the Implications of the new policy on evaluation means the planning and timing of evaluations should be more flexible, and respond to the need for learning, not strictly compliance.
This “Reporting to the top” is depicted as part of the system of effecting change, communicating imperatives and directly reporting on performance and delivery to decision makers. This is at the core of the approach to implementation – senior management will be more engaged in results and better able to define and measure success, while having the information required to make adjustments as needed.
To effectively execute the new approach, a three step process is espoused: (1) understanding the delivery challenge (2) planning for results and delivery and (3) driving results and delivery. And according to Nick Rodriguez, “data is the basis for effective decisions at all stages of the Results and Delivery journey.”
Ensuring Application and Understanding through Common Language and Training
Different people interpret words differently. Ensuring a common approach and interpretation for understanding is subsequently essential to ensuring proper delivery and implementation.
To support implementation of the system, a Results and Delivery manual has been developed and is available to Canadian public servants. In addition, a lexicon [see below] has been developed by the Treasury Board to ensure common language use.
Results and Delivery Manual “10 Rules”
Source: Nick Rodriguez, Delivery Associates, Oct. 18, 2016 PPX Learning Event.
Review of the application of the Results Based Management (RBM) and Performance Measurement system across government departments demonstrates uneven application of systems and concepts which are applied uniformly without training and support. Conversely, within Departments and Ministries, individuals that receive training in RBM at the policy, planning, programming and management levels demonstrate improved efficiency and effectiveness when applying governmental policy and completing required Program Alignment Architectures/ Activities, Performance Measurement Frameworks, and Performance Measurement Strategies.
To address this, the Treasury Board and the PCO are developing new templates for federal Ministers, with a results annex. They suggest a meeting every three weeks of the departmental RDOs with their Chief Data Officers for regular stocktaking of priorities, and to encourage departments to break down silos of communication and learn to better use their data to support decision-making.
The entire effort seeks to achieve a results culture which permeates the whole of government. A systematic, integrated approach focuses on three key aspects (1) Developing a System (2) Creating Tools and (3) Building Capacity as elaborated on the chart which follows.
Source: “The Privy Council Office – Results and Delivery Unit and Treasury Secretariat Approach” , Oct. 18, 2016 PPX Learning Event.
Universalia Management Group has decades of experience supporting learning, improving performance, measuring success and supporting organizations and institutions. This work includes working with bilateral agencies, such as Global Affairs Canada (formerly CIDA) in developing the Results Chain and Performance Measurement Plan for CIDA’s Knowledge Management Initiative, and most of the International Financial Institutions (IFIs) on Effectiveness in Managing for Development Results and Implementing the Common Approach to Assessing Multilateral Effectiveness and Evidence of Contribution to Development Results.
Universalia will continue to support the Government of Canada to implement Results and Delivery through monitoring and evaluation, and potentially in the future, level-specific and common training for public servants based on the Treasury Board Policies, Directives, Manuals and Lexicon. This will support efficiency, effectiveness and performance in preparing and maintaining Departmental Results Frameworks (DRFs), Department Program Inventory (DPIs) and Performance Information Profiles (PIPs) and delivery of required evaluation functions.
Treasury Board Policy Suite: Lexicon
The following definitions will be included in the glossary for the Treasury Board Policy Suite and should be taken as the common lexicon (2016-07-01).
|Departmental Result (résultat ministériel):||Departmental Results represent the changes departments seek to influence. Departmental Results are often outside departments’ immediate control, but they should be influenced by Program-level outcomes.|
|Departmental Results Framework (cadre ministériel des résultats):||Consists of the department’s Core Responsibilities, Departmental Results and Departmental Result Indicators.|
|Departmental Result Indicator (indicateur de résultat ministériel):||A factor or variable that provides a valid and reliable means to measure or describe progress on a Departmental Result.|
|Effectiveness (efficacité):||The impacts of a program, policy or other entity, or the extent to which it is achieving its expected outcomes.|
|Efficiency (efficience):||The extent to which resources are used such that a greater level of output/outcome is produced with the same level of input or, a lower level of input is used to produce the same level of output/outcome. The level of input and output/outcome could be increases or decreases in quantity, quality, or both.|
|In the Government of Canada, evaluation is the systematic and neutral collection and analysis of evidence to judge merit, worth or value. Evaluation informs decision making, improvements, innovation and accountability. Evaluations typically focus on programs, policies and priorities and examine questions related to relevance, effectiveness and efficiency. Depending on user needs, however, evaluations can also examine other units, themes and issues including alternatives to existing interventions. Evaluations generally employ social science research methods.|
|The financial and non-financial resources (e.g., funds, personnel, equipment and supplies) used by organizations to implement policies, programs and other activities to produce outputs and influence outcomes.|
|Impartiality in behaviour and process, ensuring that official, professional, personal or financial relationships or interests do not have an impact on evaluations. Neutrality also requires that preconceived ideas, prejudices or biases do not affect the decision whether to evaluate; analysis; findings, conclusions, recommendations; and the tone and content of evaluation reporting.|
|Organizational Profile (profil de l’organisation):
|Basic data used to identify organizations included in the institutional and corporate composition of the Government of Canada (e.g., name, appropriate Minister, enabling instrument, etc.).|
|Program Outcome (résultat du programme):
|A change or consequence attributable to outputs or to which outputs or other outcomes of the program contribute. Program outcomes may be a hierarchy of short-, medium-, or long-term, and are generally more directly influenced by departments and within their immediate control than Departmental Results. Program outcomes are found at the Program level, whereas Departmental Results are found at the Core Responsibility level.|
|Direct products, services or similar stemming from the activities of an organization, policy, program or initiative, and usually within the control of the organization itself.|
|Performance Information Profile (profil de l’information sur le rendement):||The document that identifies the performance information for each Program from the Program Inventory.|
|Program (programme):||Individual or groups of services, activities or combinations thereof that are managed together within the department and focus on a specific set of outputs, outcomes or service levels.|
|Program Inventory (répertoire des programmes)||Identifies all of the department’s programs and describes how resources are organized to contribute to the department’s Core Responsibilities and Results.|
|Program of grants and contributions (programme de subventions et de contributions):||Refers to the definition of “program” in subsection 42.1(2) of the Financial Administration Act, which defines program as a program of grants or contributions made to one or more recipients that are administered so as to achieve a common objective and for which spending authority is provided in an appropriation Act|
|The extent to which a program, policy or other entity addresses and is responsive to a demonstrable need. Relevance may also consider if a program, policy or other entity is a government priority or a federal responsibility.|
|Resource alignment review (examen de l’alignement des ressources):
|Cyclical or targeted exercises that consider the alignment with priorities, resources, and results of government programs in support of management excellence, good expenditure prioritization and innovation.|