enquiries@rodlaird.co.uk »
Freephone 0800 612 0910
Freefax 0800 612 0920
Now taking bookings
Bookmark and Share

How to assess the true impact community involvement: A seven step guide (05 Apr 18)

These priceless tips from a professional evaluator guide you through the maze


Have you been asked to design and conduct an impact assessment (IA) of community involvement?

Here are seven practical steps that can help you complete an IA that is useful for improving services and for reporting to a range of audiences.

Step 1: Learn the differences between inputs, activities, outputs, and outcomes

These terms may be jargon to you, but they are easy to learn and essential to designing and assessing your efforts in  community involvement. Figure 1 below shows you some common definitions.

Step 2: Develop a simple logic model of your community efforts

“Logic model” is more jargon, but it is becoming enormously common around the world, and you simply must know what it means. A logic model is a visual display, on one page, showing very clearly what resources your program has (inputs), what you do with those inputs (activities), how much you accomplish (outputs), and most importantly, what changes you achieve as a result of your activities (outcomes).

A good logic model also shows how the different outcomes relate to each other -- which are achieved first, which next, and so on all the way to the ultimate outcome .

Figure 2 below shows you a simple logic model for a programme to help non-English speaking men gain employment. It shows that three activities were conducted with the men (the boxes at the bottom) – literacy classes, workplaces skills classes, and one-on-one vocational counseling.

The other boxes show that the programme aimed to have the men learn English, acquire good interviewing skills, be able to fill out job applications, have key workplace skills, and gain full-time jobs.

Figure 1: Inputs, activities, outputs, and outcomes

 

Figure 2: An employment program for non-English speaking men

We strongly recommend you to develop a similar model for whatever your organisation is doing to increase community  involvement. Your activities should go in boxes along the bottom row, and your various desired outcomes should flow upward from those activities, with later outcomes building on earlier ones.

You may not see right now the value of creating this logic model, but you will find many ways to use it once it’s been developed.

For example, logic models help people like you to:

Step 3: Know what monitoring bodies require and suggest about outcomes

While monitoring bodies often offer considerable flexibility for you to tailor your IA to fit your own situation, many insist that you include two desired outcomes:

Other "suggested" outcomes could include:

Our advice is to include, if possible, these outcomes in your logic model, with other outcomes you first need to accomplish to reach the higher-level outcomes.

Step 4: Develop a performance measurement plan (PMP) for the desired outcomes you feel are important enough to measure on a regular basis

A good PMP includes your key desired outcomes, at least one measurable indicator for each outcome, a source of performance data, a method for gathering the needed data, and both a person responsible and a timeframe for collecting the data.

Measures should be SMART: Specific, Measurable, Achievable, Realistic, and Time-Bound. This is a good set of criteria, but perhaps an even more important criterion is that each indicator should capture the essence of the desired outcome – that is, that what you are actually measuring strikes to the heart of what you say you are trying to achieve.

Step 5: Develop a way to spot unexpected outcomes as they develop

It is not uncommon that a project creates outcomes never intended by its designers. For example, no one wants female tenants to be at greater risk because of their attendance at night-time meetings, yet possibly that could be an unexpected outcome of greater tenant involvement. If so, we obviously need to know immediately, in order to correct the situation. The best approach to finding unexpected outcomes is to be very aware that they are possible and to alert all involved persons to be constantly looking for them.

Step 6: Try to assess your organisation' s contribution to positive outcomes

“Causal attribution” is a term researchers use when they try to explain precisely what caused certain outcomes to occur. The Guide hints at requiring an attribution analysis, but it suggests only the very subjective approach of asking involved persons what they think. This can be useful, but you could also take advantage of some “natural experiments” to see if there is a relationship between your activities and the positive outcomes.

For example, did outcomes get better shortly after you started your efforts? If you phased-in your efforts at different times in different locations, did positive outcomes tend to appear each time you began new activities?

If you conducted different activities in different sites (or even no activities in some sites), is this related to the positive outcomes? And what about the intensity or “dosage” of your efforts? If you did more in some sites than in others, were the outcomes even more positive where you worked harder? If so, you will have much more compelling, and convincing, evidence that your good efforts in tenant involvement have paid off.

Step 7: Use the findings from your IA to improve services

While you might be tempted to view the IA as just another good idea, we hope you view it as a way to make your community involvement efforts as effective as possible.

We would simply add that the main reason to conduct your assessment is to find ways to improve your services. Knowing where we are on the map can help us plan the most-direct route to where we want to go.

Good luck!

Why not attend run our course An Introduction to Measuring Social Impact and Social Value at your offices? You can email me at rod@rodlaird.co.uk 



Why not join the discussion!

Or even better still offer your own advice and tell us about things that others can learn from.

We moderate comments lightly so bear with us and we'll get your thoughts listed as soon as we can.

You must be logged in to post comments.

Not registered yet? Simply fill in the box below.

Email
Password
Forgotten your password?

Like to have access to this and hundreds of other articles like it? Register now!

Just pop your details in the form below, and you'll have full access to our library as well as receiving the free articles you have requested.

Research and evaluation
Resident involvement
Community involvement
Patient involvement
Communications & marketing
Equality and diversity
Community safety
All of these
Your work email

Already subscribed? Want to manage your account? »

Log in

New?

Register now to benefit from hundreds of free hints, tips, articles and interviews

Your email address:

Contact us»

Latest Twitter updates

Courses you might like:

An introduction to measuring social impact and social value 

How to measure social return on investment
(Central and in house)

Project managment  
(In house)

Related articles:      

7 reasons to measure the outcomes of your projects

9 things to bear in mind when managing a project

12 values to base your impact assessments and evaluations on

A practical way to assess the impact of your financial inclusion

A step by step approach to using a logic model to solve a community problem

A step by step guide to developing a strategy 

A step by step guide  to developing outcome indicators

A step by step guide to measuring the impact of your work in the community

A step by step guide to using project outcome data

Another virtual workshop in Impact Assessment

How to show you’ve made a difference

Impact maps: A beginners guide

Make sure you control your projects - not the other way round

Measuring the impact of lots of small projects at the same time: one of your trickiest problems solved

Step by step guide for developing outcome indicators

Step by step guide to measuring costs, benefits and true value of your work in the community