Knowledge Management Capacity Assessmenttool: KM-CAST

KM-CAST 1

Knowledge Management Capacity AssessmentTool: KM-CAST

Introduction

International Development Knowledge Management and Learning Network[1]members (from a range of humanitarian and development organisations) have agreed to carry out a Knowledge Management benchmarking exercise utilising the Knowledge Management Capacity Assessment Tool (KM-CAST).

A KM-CAST assessment enables organisations to self-identify their level of knowledge management maturity ranging from Level 1 (Novice) up to Level 5 (Expert) across a range of indicative knowledge and learning areas.

The assessment may also help organisations to identify gaps, raise awareness, and determine areas where further attention or investment is required. It is also hoped that these assessments will encourage cross-sector learning and sharing of experiences and best practices.

Results, if shared, will help identify areas in which organisations have achieved advanced levels of knowledge management capacity. This, in turn, may encourage reflection and discussion of which approaches, actions or processes have supported improvements in knowledge management.

Objectives and benefits

KM-CAST has been designed to help:

Establish benchmarked levels of KM maturity (across a team, office or organisation)
Provide a common language and framework to discuss knowledge management and its constituent components
Identify areas of strength as well as areas for improvement in knowledge management within organisations
Provide evidence for network members to engage with senior management
Provide case studiesto feed into ongoing knowledge and learning assessment activities across the development sector

Using the tool: Guidance

The KM-CAST has nine indicator areas, each of which is split into five levels, from Level 1: Novice, to Level 5: Expert. For each indicator area, the organisation completing the tool should choose one Level which best reflects the current maturity of the organisation. A higher level should not be selected unless all the requirements of the lower levels within the same area are also met, i.e.level 5 should not be selected unless the requirements of Levels 2 to 4 have also been met. Where elements of higher levels have been partially met this can be noted within the comments section of the scoring table. When completing KM-CAST, thought should be given to potential actions and next steps. This may be captured and recorded in the benchmarking score table (Annex 1).

Scoring

Scoring is simple. A score (1-5) is given for each of the nine levels equal to the assessment level of maturity level e.g. achieving a Level 2 in Area 1 equals a score of 2 for that area. Scores should be added to the scoring table (Annex 1).

How to undertake a KM-CAST assessment?

A KM-CAST assessment can be implemented in a number of ways:

Desk based assessment by an expert or experts: The organisation’s knowledge / learning manager(s) or team complete the assessment as a desk based exercise. This may be supported by interviews with staff, organisational leaders and key stakeholders.
Facilitated workshop assessment: The tool may be applied using one or more workshops, during which participants are introduced to the tool and asked to self-assess against it. The knowledge / learning manager (s) or team should facilitate this process. This approach may be repeated for different areas of the organisation.
Staff survey: The tool is circulated to member of staff via a survey in which they are asked to complete the tool and their results collated.
Peer assessment/ Peer assisted assessment: Partnership with another peer organisation to share expertise and knowledge, enable greater impartiality and to validate results.
External assessment: External expert(s) invited to assess the organisation using the tool.
Any combination of the above

Case study:Facilitated workshop assessment.An organisation that recently completed a similar assessment took the following approach; A separate assessment was completed for each team within a single office. A one hour workshop was held for each team, facilitated by the knowledge manager. During the workshops every team member was allocated a number of stickers equal to the areas of the assessment (for this case it would be 9 stickers). Each team member then placed a sticker on the level that they judged to be appropriate for each area. To derive overall assessments for each area, the median level was taken (eg if there was 1 sticker on level 2, 3 stickers on level 3, 4 stickers on level 4 and 1 sticker on level 5 then the overall rating would be taken as level 4). Where votes were tied between two levels, the lower level was taken.

We hope agencies will document and share experiences of undertaking KM-CAST assessments through case studies, shared via the INGO KM and Learning website

We are a large organisation - should we complete more than one KM-CAST?

It is up to your organisation how to use KM-CAST. Medium or large organisations could choose to undertake a single assessment, or could instead undertake separate assessments for different functional areas, departments, or offices. Organisations undertaking more than one assessment could aggregate their scores to calculate averaged levels of maturity, or could take the lowest level per category as representing an organisational base level of maturity.

Sharing of Anonymised Results

After completing the KM-CAST assessment, please complete the scoring table (Annex 1 -page 12) and please also complete the accompanying questionnaire (Annex 2 page 13). Return both to the KM&L Network collators[2]. Results will be compiled, anonymised and shared amongst all participating organisations. This will allow organisations to see how their maturity in different areas compares to others, whilst maintaining anonymity. Results will not be shared outside of the KM&L network without prior discussion/agreement (the KM&L group may decide to share anonymised results with e.g. DFID).

Walter Mansfield, 1/07/2015

KM-CAST 1

Knowledge Management Capacity AssessmentTool: KM-CAST

Knowledge Management Area / Level 1
Novice / Level 2
Learner / Level 3
Intermediate / Level 4
Advanced / Level 5
Expert
1.Roles and responsibilities for Knowledge Management (KM) / We have not yet defined roles and responsibilities for knowledge management (KM).
We lack senior leadership forknowledge management. / We have begun to define roles and responsibilities for KM. Senior leaders have responsibility for improving KM within parts of the organisation. Some staff have defined KM responsibilities / A senior leader has taken responsibility for improving KM across the organisation. Roles and responsibilities for KM have been defined. / A senior leader has responsibility for improving KM across the organisation. Roles and responsibilities for KM have been defined. KM responsibilities are captured and monitored within individual workplans / A senior leader has taken responsibility for improving KM across the organisation. Roles and responsibilities for KM have been defined. All staff with KM responsibilities have these captured and monitored within individual workplans. Staff have the mandate and resources to fulfil their KM responsibilities.
2.Knowledge Management strategy / We do not have a knowledge management strategy or framework and we do not have a strategic approach to knowledge management. / We have started to develop a knowledge management strategy/framework (to underpin a strategic approach to knowledge management) but this is not yet complete. / We have a knowledge management strategy/framework, however it is not regarded as complete/up-to-date/fit for purpose (for ensuring a strategic approach to KM). / We have an effective knowledge management strategy/framework which is aligned to wider organisational aims and objectives. / We have an effective knowledge management strategy/framework which is aligned to wider organisational aims and objectives. All staff act on the strategy. The strategy is reviewed, monitored and updated on a regular basis.
3.Awareness and capacity / Our staff lack basic awareness of knowledge management concepts/methods/tools.
There is no-one for staff to turn to for support on knowledge management. / Some staff have some awareness of KM concepts/ methods/ tools.
There are no formal mechanisms for building staff capacity, though some staff (undesignated knowledge champions) offer colleagues support in applying knowledge management practices to their work on an informal/ad-hoc basis. / Our staff have some awareness of KM concepts/ methods/ tools and sometimes apply them to their work.
There have been some formal attempts to improve KM capacity (e.g. through training) though these are one-off/un-sustained. / Our staff have a good understanding of KM concepts / methods / tools and apply them to their work.
Designated staff (knowledge champions) are tasked with building staff capacity, using e.g. training, mentoring, sharing best practices. / All our staff have a good understanding of KM concepts /methods /tools and apply them to their work.
Designated staff (knowledge champions) proactively build staff capacity, with this remit formally captured in their objectives.
Our leaders drive KM activities and model good knowledge management behaviours.
4.Systems and technology / Our systems and technologies used to manage organisational knowledge are not fit for purpose or are unused. There is inadequate guidance on the use of our knowledge systems. Storage and sharing of documented knowledge takes place on an ad-hoc basis across systems such as Personal and Shared drives, email, document management systems. / Our systems and technologies used to manage organisational knowledge are not fit for purpose or are underused. We have identified some of the problems in our knowledge systems but there is no clear workplan/dedicated resources for improvement. / We have a good understanding of the gaps/deficiencies in our KM systems and have put in place a clear plan/dedicated resources for improvement. We do not have clear guidance on the use of our systems. There is good user uptake of our systems. / We have a good understanding of the gaps in our systems and have a clear work plan/dedicated resources for improvement. We have developed guidance on the use of our knowledge systems. There is good uptake of our systems. Users have been engaged in the design of our systems. Everyone knows how to access training and support. / Our knowledge systems meet the needs of our users.
All our staff use our knowledge systems; receive sufficient training, support and follow guidance.
We monitor our systems and address gaps.
We are able to quickly and easily find the knowledge we need to carry out our work.
5.Institutional memory / We do not have any processes or tools in place to adequately capture and share knowledge prior to staff changing roles or leaving. When staff leave their knowledge leaves with them which has led to institutional amnesia. We have no plans to address this issue. / We are aware of the problem of institutional memory and ad-hoc attempts are made to capture and share knowledge prior to staff changing roles or leaving. We haven’t yet produced organisation wide guidance or processes. We are vulnerable to knowledge loss when staff leave. / We encourage staff to engage in informal knowledge sharing activities prior to staff changing roles or leaving. We have documented guidance for this, however we don’t have a systematic procedure for ensuring this guidance is followed. We have problems capturing outgoing knowledge and are vulnerable to knowledge loss when staff leave. / We have put in place a formal knowledge transfer process for all instances of staff changing roles or leaving. This process aims to capture outgoing knowledge. Findings are documented and key lessons are made accessible wherever possible. / We have put in place a formal knowledge transfer process for all instances of staff changing roles or leaving. This process aims to capture outgoing knowledge. Findings are documented and key lessons are made accessible wherever possible. We review the lessons learned from outgoing staff and seek to apply learning to inform future activities.
6.Prioritising learning from outside the organisation (pulling knowledge into the organisation) / We do not prioritise or incentivise learning from outside the organisation. / We acknowledge the importance of learning from outside the organisation but this does not happen in a strategic way. Some staff members are incentivised to seek out knowledge and learning relevant to their work. / Our staff are encouraged to seek out knowledge and learning relevant to their work. Learning from others is reflected within individual objectives. Gaps in organisational knowledge are identified and acted upon. / Our staff actively seek out relevant external knowledge and learning and have specific goals for sourcing and capturing knowledge. Learning from others is documented, saved, and used to inform ongoing work. / Learning from others is an integral part of our organisation. Learning brought into our organisation is systematically shared with colleagues.
We have developed a learning culture and allocate, time recognition and resources to learning.
We reward and promote innovation and best practices in learning.
7.Internal knowledge sharing (organisation-wide Knowledge Sharing) / Knowledge resides in silos with little knowledge sharing across our organisation. We have little culture of sharing. / We encourage cross-organisational knowledge sharing though practices are patchy/inconsistent/lack integration. Some of our staff share knowledge though this is mainly via personal networks and relationships. / We have simple methodsfor sharing knowledge across our organisation. We have developed incentives and have defined spaces to encourage and reward sharing. Not all staff share knowledge. / We have defined methodsfor sharing knowledge across our organisation We have developed incentives and have defined spaces to encourage sharing. We have a culture of knowledge sharing / We have fit for purpose systems for sharing knowledge across our organisation. We have developed incentives and have defined spaces to encourage and reward sharing. Our staff have clear responsibilities for knowledge sharing in individual objectives. Internal knowledge is accessible across the organisation. We have a culture of knowledge sharing in which all our staff actively contribute to sharing and documenting knowledge.
8.Reflection: Learning and reflecting upon successes and failures / We do not prioritise reflection to critically review and improve practice, strategy or objectives. / We use reflection in an ad-hoc and unstructured way to critically review and improve key practices, strategies or objectives / We take time for reflection and have developed clear guidance on minimum expectations for ensuring space for reflection is prioritised, and to encourage personal accountability for learning. / Planned and structured reflection processes routinely take place for key areas of work. Reflection focuses upon changes that can be made to improve practice. We have created safe spaces to learn from failures. / Planned and structured reflection processes routinely take place for key areas of work. Lessons from reflection are used to shape and improve future practice. We acknowledge and learn from our failures
Our leaders model a reflection culture.
Reflection is integral to how the organisation works.
9.Measuring and assessing the impact of Knowledge Management / We have no processes or systems in place for monitoring actions to improve knowledge management. / We have some measures in place for knowledge management, but these are not consistent. We lack robust outcome/impact indicators for our knowledge management activities. / We have a clear measurement framework for knowledge management. We have some outcome/impact indicators. These indicators could be more robust, and we’d like to test and update them more regularly. We recognise the need to allocate sufficient resources to tackle problems found. Senior leaders are aware of the measurement framework. / We have good processes in place and have developed indicators for the majority of Knowledge Management activities. Many of these indicators are relevant and robust. We recognise the need to do more to ensure they are up to date, and remain relevant. Measurements are routinely reported to senior leaders. / We have strong processes in place and have developed robust measurements for almost all Knowledge Management activities. We assess Knowledge Management as part of organisational wide monitoring and evaluation cycles. Where shortcomings are found recommendations are made to senior leaders and these are acted upon with sufficient resources.

Walter Mansfield, 1/07/2015

KM-CAST 1

Annex 1: Benchmarking score table

Indicator Area / Score /5 / Comments / Potential Actions/Next Steps (optional)
1.Roles and responsibilities for Knowledge Management
2.Knowledge Management strategy
3.Awareness and Capacity
4.Systems and technology
5.Institutional memory
6.Prioritising learning from others
7.Internal Knowledge Sharing
8.Reflection: Learning and reflecting upon successes and failures
9. Measuring and assessing the impact of Knowledge Management
Total: / /45

Annex 2: Benchmarking questionnaire

Question: / Response:
Name of organisation (and unit within organisation where applicable):
Name and job title of organisation contact:
Who participated in the benchmarking assessment from your organisation? (No. of employees, areas/departments, levels of seniority?)
How big is your organisation? Number of employees? Annual turnover?
What resources did you allocate to completing the benchmarking assessment? – Number of employees, person hours, other resource inputs?
Where you have completed multiple assessments using KM-CAST please give the number of assessments completed and describe any methods of aggregation.
Do you have any feedback on the benchmarking tool? How can it be improved, what worked, what didn’t work? (If there are any indicators that you would prefer to see re-phrased, please suggest alternative wording).
Do you have any advice for others completing this process?
Would you be willing to offer support to another peer organisation to share knowledge and assist in their benchmarking?

Glossary of terms

INSTITUTIONAL MEMORY: a collective set of facts, concepts, experiences and know-how held by a group of people within an organisation

KNOWLEDGE MANGEMENT: the processes, tools and culture required to enable people to capture, manage, synthesise, share and re-apply knowledge to create and innovate and effective organisation.