This section provides a starting point for the use of evidence standards within OutNav.
When tracking the progress of a project, it is helpful to refer to your evidence standards to ensure your analysis is robust. Every organisation needs to agree the basis on which they will rate their progress towards outcomes and their confidence in their evidence within OutNav.
In OutNav we have included some set evidence standards that you can see when you are working on any stepping stone (just click the magnifying glass icon next to the ratings buttons on the Update Analysis page). These standards offer a guide as to how you will rate whether progress is low, some or good, and whether you have low, medium or high confidence in your evidence. They follow the usual ways of judging confidence in evidence as set out in academic work.
Many organisations use these standards, which are included by default in every new OutNav project. However, there might be instances where you want to change these to suit a specific project or programme. For example, there may not be any robust evidence available, meaning you want to base your standards on reflections. There may be funder requirements that you want to incorporate, or you may simply want to change them to suit your organisation's language or preferences.
To customise the evidence standards in your OutNav project, you can click the "edit" button from the evidence standards pop-up on the Update Analysis page, or go to your Project Settings. Please note: only Project Admins and Organisation Admins have permission to update evidence standards.
Our default evidence standards are outlined below.
Rating Progress
It is likely that you will come up with a rating of no progress in some places in your outcome map, especially if this is the first time you have scrutinised processes. The OutNav approach is flexible and allows for redirection and innovation if things don’t go as planned. You can change and adapt your outcome maps as you learn about what is working well and what isn’t. They are not intended to be set in stone and used as a way to judge performance.
Below are some suggestions for progress rating criteria:
Great progress |
|
Some progress |
|
No progress |
|
Other progress issues to consider include:
- Are you measuring current service users?
- The number of people we aim to serve, or a proportion of the population who we could serve?
- Is this a formative or summative judgement (formative to think about progress, summative to say something about overall programme effectiveness)?
- How often will you rate your progress?
- Will you measure progress as you go, or rate solely against your final aims?
- Will have a project plan specifying the timescales over which levels in a pathway will be achieved?
Rating Confidence
Confidence measures are the standards that your organisation seeks to achieve in data quality and robustness that will allow you to make solid progress ratings. Organisations take different approaches to this. For example, are you seeking evidence for every part of your outcomes map, or could you use reflection as the basis on which you rate progress?
In general, confidence in evidence is based on a combination of the following factors:
- How many sources of evidence are there?
- Is there agreement/disagreement between evidence sources?
- How robust is the evidence (how was it collected, how many people, purposefully collected data or opportunistic)?
Below are some suggestions for confidence rating criteria:
Very confident |
|
Some of the above criteria apply and/or:
Some confidence |
|
Few of the above criteria apply and/or:
Little confidence |
|
Remember, these evidence standards have been provided as a guide, you may decide on more specific evidence standards within your project team or as an organisation.
The evidence standards can be included in the reports you generate from OutNav, viewed from the Update Analysis page, or customised from your Project Settings.
Comments
0 comments
Please sign in to leave a comment.