Audit and Evaluationcae/DG, AEB Annual Report 2008-2009

CAE/DG, AEBAnnual Report

2008–2009

July 2009

1

Environment Canada

Audit and EvaluationCAE/DG, AEB Annual Report 2008-2009

Table of Contents

1.0Introduction

2.0 Report on Activities

2.1Achieving the Internal Audit work plan

2.2Achieving the Evaluation work plan

2.3Assurance, Liaison and Coordination Activities

2.4Summary of Branch Accomplishments

3.0 Status of the Audit and Evaluation Branch Functions......

3.1Audit and Evaluation Branch Priorities

3.1.1Implementation of EC’s Internal Audit Policy Implementation Plan

3.1.2 Quality Assurance – Internal Audit Function and strategic planning and Coordination

3.1.3Quality Assurance – Evaluation Function

3.1.4 Meeting the MAF Criteria Requirements

3.1.5.Implementing HR Strategies

4.0Resource Utilization

5.0General Lessons Learned

6.0Looking Ahead

Figures

Figure 1: Project-Related Person Months Utilization by Function

Figure 2: Percentage of Total Person Months Utilization by Function

Figure 3: Actual Expenditures by Function……………………………………………………….11

1

Environment Canada

Audit and Evaluation CAE/DG, AEB Annual Report 2008-2009

1.0Introduction

Environment Canada's Audit and Evaluation Branch (AEB) supportsthe Deputy Ministerand senior management in attaining strategic objectives of the Department by providing them with objective, independent, evidence-based information, and assurance and advice on the effectiveness and efficiency of departmental programs, policies and operations. In addition, the Branch supports the External Audit Advisory Committee (EAAC), the Departmental Evaluation Committee (DEC) andother related committees, provides services to liaise external auditors with the respective departmental representatives, and provides oversight in responding to environmental petitions.

Each fiscal year, the Branch prepares a departmental risk-based audit and evaluation multi-year plan and an annual report on its activities as well as astatus report on internal audit engagements and evaluation projects conducted during that year.[1]

The purpose of this document is to report to the EAAC and the DEC on accomplishments achieved against commitments and priorities for 2008–09,as set out in the Audit and Evaluation Plan 2008–11,and important lessons learned throughout the year.

2.0 Report on Activities

2.1Achieving the Internal Audit work plan

Ten new internal audit projects were planned for 2008–09. Three of these projects were completed, with theReview of Information in Support to Decision Making being tabled at EAAC’s January 2009 meeting, and the Audit of Staffing and Audit for Accounts Receivable being tabled at EAAC’s April 2009 meeting. One of these projects, the Audit for Technology Infrastructure Directorate, was cancelled, as it was deemed too low risk. The remaining six projects have been carried-over into 2009-10, with two scheduled for completion in the first quarter, and the remaining four in the second quarter.

Forty-one internal audit recommendations were followed-up.

Internal Audit also provided active leadership in the implementation of the Treasury Board Secretariat (TBS) Policy on Internal Audit across government by participating in Chief Audit Executive meetings and serving on the Senior Advisory Committee on Internal Audit Capacity, and several other TBS committees and working groups.

Currently, there are five Certified Internal Auditors, two Certified Information Systems Auditors, one Certified Government Audit Professional, oneCertified Internal Auditor with additional Certification in Control Self-Assessment, one Certified General Accountant, and one Chartered Accountantwithin the Branch.

Liaison and coordination with the OAG, CESD and Other External Auditors

The audit team coordinated its activities with the Office of the Auditor General (OAG) through regular discussions with OAG/Commissioner of the Environment and Sustainable Development (CESD) representatives about the latter’s audit work plan for the next five years and its impact on the department’s internal audit work plan. It is expected that the OAG will increasingly rely on the Department’s work.

In an ongoing process, the Audit and Evaluation Branch is also building its relationship and establishing working protocols with other external auditors such as the Public Service Commission and the Commissioner of Official Languages that are increasing the number of audits in which the department is included.

Environment Canada(EC) was involved in ten external audits that were tabled in 2008-2009. EC is included in eleven external audits that were initiated in 2008-2009, and that are scheduled for tabling in 2009-2010.

2.2Achieving the Evaluation work plan

The Evaluation Division planned to conduct twenty-eight evaluation projects in 2008–09, ten of which had been carried over from 2007-2008. Of that number, eleven projects were completed and approved during the past fiscal year. One project, the Ice Program Evaluation, was initially postponed due to Strategic Review. It was subsequently postponed again due to low risk and relative importance compared to other obligations and priorities. Of the sixteenprojects carrying over into next year (2009–10), eleven are scheduled for completion in the first quarter of 2009-10, while the remaining five are scheduled for completion in the second quarter.

Evaluation also provided leadership through membership on TBS committees on evaluation guidelines, performance measurement and evaluation competencies. In

2008-09, it gave a presentation to the Performance and Planning Exchange and several presentations at Canadian Evaluation Society conference. Significantly, it also hosted the inaugural two-day meeting of the Environmental Evaluators Network (Canada) Forum. Further, it was strongly engaged in the development of guidelines and handbook for Performance Measurement and Evaluation Plans for regulatory initiatives.

In addition, the Evaluation Division faced several staffing changes in 2008-09. Two evaluators plus the Evaluation Division's summer student participated full-time on the Strategic Review from April to August, while two other evaluators participated on the Strategic Review in March. In addition, one ES-06 retired in December, one ES-05 is on assignment at Treasury Board, oneES-06 is on assignment with EC from NRCan, and one ES-05 was deployed into the Evaluation Division.

2.3Assurance, Liaison and Coordination Activities

Strategic Planning and Coordination (SPC) provided secretariat, consultative and strategic support for the EAAC, with four meetings and six teleconferences in 2008-09. The EAAC was at the forefront in delivering on the eight core areas of responsibilities including the first EAAC annual report to the Deputy Minister. The EAAC also met with the Minister where this report was further discussed. SPC also provided secretariat support for the DEC with four meetings in 2008-09.

SPC coordinated EC’s response to the twenty-twopetitions addressed to the Minister of Environment during the past fiscal year. Of these twenty-two petitions, five petitions were sent for information only. Ten received responses within the 120-day legislated timeframe, six received late responses, and one response is under development. Factors that impact on the timeliness of responses, such as management delays or sign-offs are not within the control of SPC. It should be noted that the CESD reports biannually on responses to petitions received between July 1 and June 30. SPC also contributed to the development of the OAG’s guidance document, “Getting Answers: A Guide to the Environment Petitions Process.”

SPC continues to improve EC’s Risk-Based Audit and Evaluation Plan. The 2009-12 plan includes a well-defined and risk-ranked audit universe using the 2008-09 Program Activity Architecture, a detailed mapping of projects to the departmental priorities, the Corporate Risk Profile (CPR) and core management controls, and better alignment and integration with other audit assurance providers, such as the OAG, the Office of the Comptroller General (OCG), the Privacy Commissioner, the Commissioner of Official Languages and the Public Service Commission. It further includes the provision of fuller risk description, rationale and scope for all audit and evaluation projects, including carry-overs, and offers an improved focus on follow-ups on recommendations and management action plans.

2.4Summary of Branch Accomplishments

The fiscal year 2008-2009 has been a transition year, where key milestones in implementing the new Policy on Internal Audit were achieved. Efforts focused on Branch staffing, introducing the implementation of detailed management action plans to be presented with each report to the EAAC, initiating activities to strengthen reporting mechanisms, and to initiate the development of a long-term strategy for the provision of a holistic assurance.

Summary of Activities

3.0 Status of the Audit and Evaluation Branch Functions

During 2008–09, the AEB undertook a number of activities to strengthen the internal audit, evaluation and strategic planning and coordination functions. This section summarizes some of the key accomplishments.

3.1AEBPriorities

For fiscal year 2008–09, the AEBfocused its resources and efforts on a number of priority areas:

strengthening the follow-up process for audit and evaluation recommendations;

working with management to strengthen the process for responding to OAG environmental petitions by contributing the OAG’s guidance document, “Getting Answers: A Guide to the Environmental Petitions Process”;

conducting focused audit activities on People, Stewardship and Risk Management, in support of providing an annual holistic opinion to the Deputy Minister;

preparing to respond to the new TBS Evaluation Policy;

improvingmanagement action plans in response to recommendations;

ensuring appropriate and effective support for departmental strategic review;

providing ongoing support to EAAC and DEC; and

implementing the Branch Human Resources Plan.

3.1.1Implementation of EC’s Internal Audit Policy Implementation Plan

As a result of the 2006 Policy on Internal Audit, in 2007, the Treasury Board approved the creation of the a three-member, independent EAAC whose fundamental role is to ensure that the Deputy Minister has independent, objective advice, guidance and assurance on the adequacy of the Department’s control and accountability processes. They reinforce the independence of internal audit and, at the same time, challenge the function to achieve high levels of performance and professionalism.

For 2008-09, EAAC had four meetings and six teleconferences.

Being at the forefront, EC continues to play a lead role in working with central agencies and other government departments to address challenges identified in the implementation of the Internal Audit Policy. In 2008-09, the issues dealt with were related to the reimbursement of EAAC members for per diem and travel expenses, as well to security issues related to the receipt and handling of secure documents. EC continues to work with the OAG/OCG to address the EAAC’s concerns regarding the traditional timelines available for reviewing external audits thathave not been adjusted to consider the role of the EAAC in the process.

In June of 2008, the EAAC presented its first Annual Report to the Deputy Minister. This was a summary of the EAAC’s activities for 2007-08, and an assessment of EC’s system of internal controls, risk management and the capacity of its internal audit function. The EAAC members all met with the Minister to present and discuss their report.

3.1.2 Quality Assurance – Internal Audit Function and strategic planning and coordination

In April of 2008, a quality assessment of the internal audit function was completed. The assessment indicated that while the internal audit function was in general compliance with accepted Internal Audit standards, there was a level of non-compliance with its external quality review, and that more work was needed on the holistic opinion for the Deputy Minister.

In November of 2008, an internal quality assessment was performed against the upcoming January 2009 changes to the IIA standards. It was found that there were minor levels of non-conformance, and an action plan was developed and is currently being implemented to improve the internal audit function.

Following these changes, an external quality assessment was conducted by Ernst & Young. It was found that internal audit activity was in general compliance with the new IIA standards, with only partial compliance in the one area of follow-up. This area is being addressed. The results of the Ernst & Young report were discussed at the EAAC April 2009 meeting, and will be presented at their upcoming July 2009 meeting.

3.1.3Quality Assurance – Evaluation Function

As part of the AEB Quality Assurance Program, the Evaluation Division conducted an internal assessment of its compliance with evaluation standards. The standards used to guide the internal assessment are based on existing evaluation standards (e.g., TBS), as well as the adaptation of relevant Internal Audit standards. The assessment demonstrated that the Evaluation Division is in compliance with the standards, with the exception of one area of partial compliance.

Completion of evaluations within the expected timelines is an issue for the unit at this time. Feedback from clients/program managers suggests that evaluations are typically completed in a timely manner that serve program decision-making issues and have not impacted program functioning (such as submission to TBS for renewals). One issue in this context is related to expectations regarding the timing of completion of evaluations that span fiscal years. The final completion of evaluations scheduled to be carried out within a fiscal year, for example, taking the report to the DEC, may always have been intended to span into the next fiscal year. Our annual plan document, however, does not indicate the expected completion date for projects, only the fiscal year in which a project begins. This improved communication about timelines for evaluation projects should help to clarify perceptions of timeliness. Other delays do occur, some of which are external to the Evaluation Division, such as delays in the formal approvals process, delays in the cost recovery process, or the frequent need to review and revise materials submitted by contractors. That said, the Division recognizes that improvements in evaluation work planning to ensure feasible evaluations that can be completed on time, prioritization of human and fiscal resources to ensure the completion of key evaluation products, and careful monitoring of timelines for completion of products will help manage the Division’s ability to complete evaluations within expected timelines.

3.1.4 Meeting the MAF Criteria Requirements

The Management and Accountability Framework (MAF) VI results for the effectiveness of the internal audit function (criterion #18) were based on an assessment of EC’s 2008-09 activities and core functions, and are as follows:

18 Effectiveness of Internal Audit function (Acceptable)

18.1Internal Audit Governance (Strong)

18.2Internal Audit Planning (Opportunity for Improvement)

18.3Internal Audit Human Capacity (Acceptable)

18.4Expected Results (Strong)

In response to these opportunities for improvement, EC has improved its Risk-Based Audit and Evaluation Plan for 2009-2012 by including an Audit and Evaluation Universe and linking it to departmental priorities, CRP, and core controls. It includes not only the required risk-ranking of audit projects, rationale for resource estimates and carry-overs, and statement of constraints, but also a projected start date and tabling date for each project.

The AEB focused branch efforts to improve risk-based planning by providing greater context for risk and significance of projects. Further, the department is committed to more regular reporting on the follow-up of management action plans, and has implemented new guidelines and procedures.

The MAF VI results for the quality and use of the evaluation function (criterion #6) are as follows:

6 Quality and Use of Evaluation (Strong)

6.1 Quality (Strong)

6.2 Neutrality (Strong)

6.3 Coverage (Acceptable)

6.4 Usage (Acceptable)

The MAF VI recognized that EC continues to demonstrate that evaluation information is used to inform expenditure and policy decision-making in the department. It further noted that evaluations led by the department have addressed key issues of relevance and success using appropriate methodologies. It found the governance of the function to be appropriate and that the scope of its coverage to be increasing.

3.1.5.Implementing HR Strategies

During the past year, the Branch has made progress towards implementing the strategies identified in the Human Resources (HR) Plan for 2008–09. For example, the Branch provided coaching sessions to help employees determine their career objectives and the steps to achieve those objectives. Each employee developed a personal learning and development plan to acquire and maintain the knowledge, skills and competencies related to their level and functions, departmental business priorities, and career aspirations

The AEB remains committed to supporting professional development for its employees. Internal Audit resources are trained in using TeamMate and ACL in the conduct of audit services.

4.0Resource Utilization

For 2008–09, professional resource requirements to carry out audit engagements, evaluation projects, and strategic planning and coordination were 177 person months and $508,000. A breakdown of the person month allocation and professional services by function is shown below. These figures do not include resources dedicated to management activities.

Person Month (PM) Allocation and Professional Services

*Person month description by category: