Wildlife Tree Evaluation

1.  Are practices meeting the intent of policy?

2.  What are the timber supply impacts of current practices?

3.  How are WTR areas changing?

4.  Are there practices that achieve biological objectives and minimize costs?

¨  Six regions, 12 districts, Seven BEC zones -- 128 cutblocks

¨  Approximately $175,000 and the equivalent of 22 months of staff time.



Life is like a box of chocolates…evaluations are like a box of smarties.

(Forest Gump – Director’s Cut)



s Scale, scope and success

m Methodology

a Adaptable

r Realistic resources

t Team

i Information

e Evaluate the evaluation

s Statistics

S Sponsor scope, scale, stakeholders and success

·  Have a sponsor and client

·  Choose an appropriate scale (small and focused or large and complex)

·  Clearly define measurable objectives, deliverables and measures of success

·  Decide what is in the scope of the project and do your best to avoid “add ons”

·  Know and involve your stakeholders at every level/stage of the evaluation

M Methodology and Management

·  Field test the methodology and train the data collectors

·  Manage the project (facilitate the team, communicate, monitor progress, remove the roadblocks and find the resources i.e., money and expertise)

A Adaptable

·  Expect the unexpected and be ready and able to adapt

R Realistic, resources and risks

·  Have realistic expectations on timing, budget, staff time, availability of information…

·  Resources (people, time, money)

·  Analyse the risks

T Team and Timely

·  Involve the right people on the team at the right phase at the right time

·  Be realistic on timeline and then deliver on time

I Information management

·  Know how you want to report out at the beginning and have a plan to maximize the data

E Evaluate the evaluation

·  A critical review of what did and didn’t work is essential – and share this information with others

S Statistics

·  If you want to prove something, change something or influence someone’s decision, statistics will make it easier and/or defensible


Wildlife Tree Evaluation

-- Lessons learned

¨ The Team is the Evaluation

¨ Before you begin, know your destination and how long it will take to get there

¨ Get in shape before you try to run a marathon

¨ Focus on the big things

¨ Training -- seek to understand and then to be understood

¨ Know and embrace your stakeholders

¨ Don’t try to hit every possible destination on your evaluation trip

¨ Keep the tools sharp


Lessons we learned from the WLT evaluation

With hindsight being 20/20, there were things that we, as a project team, would do differently with this evaluation if it were to start again. The following, although not a complete listing of things we would change, highlights what might be called our version of the “seven habits of highly effective evaluators”. Some of the key lessons learned include:

Before you begin, know your destination and how long it will take to get there

This was a large project. In all, more than 11 000 trees were sampled in 128 cutblocks. At the onset of this evaluation, we spent many weeks debating and defining its objectives. In the end, although most of the original team agreed on where we wanted to go, we didn’t know that part of the route (namely the data input and compilation) would take many months longer than expected, thus delaying the entire report by several months. In addition, we added two destinations (CWD and Lichen assessment) at the last moment in the planning process – this resulted added field survey time and in some inconsistency in the data collection process, and therefore limited usefulness of this aspect of the project.

Get in shape before you try to run a marathon

With large evaluation projects, it would be advantageous to go beyond field-testing of the methodology and data compilation procedures before launching into a full-scale project. Time, money and effort could have been saved on this project had we performed a full scale operational testing (i.e. collected data on one district) of the data collection and input/compilation methodology.

Focus on the big things

A significant amount of time (data collection and data entry) was spent on small tree data. These small trees accounted for little timber volume and limited wildlife tree habitat. Given that each site had a maximum of one day for assessment, time, in some cases, could have been better spent on assessing more significant aspects of stand-level biodiversity – or finishing the site earlier and moving onto the next block.

Seek to understand and then to be understood and Training

Although a one-day training session was held in order to familarize field members of the evaluation team with the evaluation methodology, this was not sufficient training given the new, relatively untested and complex methodology. In order to help answer/clarify ongoing questions/issues that arose during the collection of field data, we utilized a style of email distribution.

Many of the small issues and concerns that caused some delay and time for field staff could have been avoided with a more rigorous training session. Furthermore, we would have identified some of the necessary methodology changes ahead of time and been able to “solve” the problems with the people who collected the data.

A two-day session, where the field people collected their own data on each type of reserve and presented the results to the larger group would have been a very cost-effective approach. More time spent asking questions, generating discussion and listening to the solutions generated by the field assessors themselves would have made the project more efficient and effective.

Know and embrace your stakeholders

Defining who the key stakeholders are and ensuring they are represented on the evaluation team (or kept well informed of the projects progress) is an important step towards achieving the final evaluation objectives. Although industry is a key stakeholder of this evaluation, the industry member that had agreed to be on the initial project team withdrew due to concerns over the field-based nature of the evaluation methodology. A new industry partner should have been brought on at that time, this however did not occur.

Don’t try to hit every possible destination on your evaluation trip

Truth be told, we tried to answer far too many questions and had too many objectives for a project with the limited funds and time allocated to us. A focused evaluation, answering one-two key questions would have been completed much quicker and at a lower cost.

Keep the tools sharp

Part of the reflective component of every evaluation should be a critical look at what did and did not work and how things can be improved the next time. An evaluation of wildlife tree retention could have been done successfully many different ways. Our evaluation was a success, nevertheless, it can be improved for the next time around. The suggestions in this section of the report are designed to facilitate that improvement and provide some “food for thought” for those embarking upon their own evaluations.



Life is like a box of chocolates…evaluations are like a box of smarties…

…Each handful is unique