Pieces to piece together (Quantifying outcomes and value)

Part of what I was asked to do was to look into a Social Return on Investment approach to making a case for the value of digital storytelling / object handling + reminiscence therapy for people with dementia.

The Guide to Social Return on Investment (SROI) is very helpful and takes people through the process of doing an SROI evaluation in a step-wise fashion (Nicholls et al., 2009). The key points are – keep the process transparent, including the assumptions that you make; develop the evaluation in consultation with stakeholder, ascertain the outcomes of your activity, work out the value of that work in a monetary equivalent. So… outcomes and value…

As I’ve noted earlier, the impact of reminiscence is an area of on-going research.

The systematic reviews by Woods et al. (2005a) and Olazarán et al. (2010) are positive about reminiscence in a cautious fashion but are nonetheless clear that the data doesn’t allow for strong conclusions about the clinical value of reminiscence. In their protocol for reminiscence therapy, Woods et al. (2009) maintain that position although they note the popularity of reminiscence amongst people with dementia and carers. (The popularity supports the bedrock of anecdotal observation suggesting the value of this type of work.)

Back in 2005, Diaz et al. identified the need for a consensus approach to assessing the impact of care for people with dementia. [Useful numbers: from the data they gathered they found that, of the patients reported, 380 (68.5%) had Alzheimer’s Disease, 57 (10.3%) had Vascular Dementia, 25 (4.5%) had Dementia with Lewy Bodies, 23 (4.1%) had Fronto-Temporal Dementia, 31 (5.6%) had Mixed Dementia, and 39 (7.0%) had another kind of dementia.]

Since Diaz’s 2005 paper there has been a pan-European endeavour to develop a consensus approach to researching the treatment of Alzheimer’s Disease (see Moniz-Cook et al., 2011 and Katona et al., 2007) such that the results from groups across the continent would be comparable. The various academics and practitioners worked through consensus approach to figure out what domains for impact should be considered and which tools could best be used to facilitate the evaluation of each domain (Moniz Cook et al., 2008). They don’t really summarise the findings in a neat way, so I will:

Cognition in the person with dementia: Alzheimer’s Disease Assessment Scale (ADAS-Cog), Nurses Observation Scale for Geriatric Patients (NOSGER), Cambridge Cognitive Examination – revised (CAMCOG-R) or the Mini-Mental State Examination (MMSE)
Patient Mood: Cornell Scale for Depression in Dementia (CSDD)
Family Mood: General Health Questionnaire (GHQ)
Patient Quality of Life: Quality of Life in Alzheimer’s Disease (QOL-AD)
Family Carer QoL: World Health Organisation Quality of Life Assessment Instrument (WHOQOL)
Family Carer Burden: Zarit Burden Interview (ZBI)
Patient Activities of Daily Living: Lawson-PMS-IADL
Patient Behaviour: Neuropsychiatric Inventory (NPI)
Staff Carer Morale: either GHQ or the Masach Burnout Inventory (MBI) – the authors are ambivalent
Global Patient Measures: either Clinicians Global Impressions of Change (CIBIC-Plus) or the Gottfreis-Brane-Steen (GBC) measures – again the authors are ambivalent and both of these require trained staff to implement.

Most of these are free (or easily accessible) and do not require specialist training to implement, with the exceptions of NPI, CIBIC-Plus and GBS.

Of course, with this sort of research you need to have a pre-determined idea about the domains where you expect there to be some impact:
– As noted previously, the review by Olazarán et al. (2010) found multi-component activities (involving cognitive stimulation plus other activities such as reminiscence) to have positive impacts on cognition, activities for daily living (ADL), behaviour and mood.

– In the protocol for randomised trials of reminiscence therapy, Woods and co-workers (2009) have identified primary and secondary outcomes. The primary outcomes looked for will be the quality of life of the person with dementia and the care-giver’s mental health. Their anticipated secondary outcomes are autobiographical memory in the person with dementia, quality of relationship between the person with dementia and their carer, depression and anxiety for both parties, stress in both parties, general quality of life for both and general functional ability.

What’s really nice about the protocol is that it lays out, clearly and in detail, the way that the researchers have structured their trials and what tools they are going to use for each domain. Another interesting feature of this protocol is that it carries within it an explicit shift from ‘person-centred’ care to ‘relationship centred’ care. (Note the inclusion of ‘quality of relationship’ as a domain.)

So, even if we cannot state what the impact is with clinical confidence – we can draw on this body of consensual work to set up research that will investigate the impact and will have parity with other work in the field.

Assume for a moment that these type of interventions do have an impact; for political and economic reasons, that impact has to be given a value in monetary terms. Putting a value on human life can feel morally difficult (Dye, 2008) but, as Hughes (2010) points out the demand on healthcare is potential infinite whilst the resources are resolutely finite. The solution lies in ‘health economics’.

The papers by Wimo (2007) and Knapp (2007) both outline the issues with this area very helpfully. The report by Hughes (2010) is meant to be a source of advise.

Either we do either a cost-benefit analysis – working out the cost of doing something with respect to the benefit accrued by not doing nothing – or we do a cost-offset analysis – working out the cost of doing something one way against the cost of achieving the same goal through a different means. The cost benefit analysis might for instance give some indication of the value of providing reminiscence therapy against the option of not doing anything – which is the approach that Woods et al. (2009) are taking and the idea behind SROI. The cost off-set might set the costs of reminiscence against the cost of achieving the same state of well-being (plus side-effects) through prescribing drugs – this is the line of argument taken by Olazarán et al. (2010).

I suspect that cost effectiveness is harder calculate. Wimo (2007) gives an equation to follow…

ΔC/ΔE = (CA − CB)/(EA(T1−T0) − EB(T1−T0))
where C = costs, E = effect (consequence, outcome), A, B = treatments, T0,
T1 = measurements.

… so you need to be able to quantify the outcomes in a way that can be handled like this. In a cost off-set approach I think you can get away without quantifying outcomes in the same way because you merely have to agree that two treatment approaches have the same consequences, however you describe them. Knapp noted that, at the time of publication (2007), there were no published cost-benefit analyses of dementia interventions.

There are tools to help do this: the Personal Social Services Research Unit (University of Kent) compiles a report on the ‘Unit Costs of Health and Social Care’ (Curtis, 2010) which documents the financial value of different services and these could be fed into calculations of value.

However, there is a great deal of complexity lurking under the surface. As Knapp (2007) notes, “People with dementia are likely to use a number of services, each of which needs to be captured for a fully comprehensive economic evaluation. In addition, there will be many and important inputs from family and other (unpaid) caregivers. How those latter inputs are costed is a source of contention. Many studies have shown the costs of informal care to be considerably greater than the costs of services provided to people with dementia but the costing of caregiver inputs is not straightforward.” Knapp goes on to discuss the subtleties of this approach but one important point is that these decisions drift over into wider political concerns. “Bodies such as NICE are required to take a health and social care system perspective, which means that they necessarily attach a zero value to caregiver inputs.”

“An additional approach is to calculate “utility,” generated from health-related quality of life scales. Combining such a measure with information on costs allows the economist to carry out what is often called a cost-utility analysis. The best-known and most robust measure of “utility” is the Quality-Adjusted Life Year (QALY).” […]

“Many evaluations of new interventions find them to be both more effective (the outcome profiles are better than for old or current interventions) but simultaneously more expensive. […] How is the trade-off to be made between the better outcomes and the higher expenditure necessary to achieve them? The methodological approach now widely used is to construct cost-effectiveness acceptability curves (CEACs). These [curves] show the probability that a new intervention will be cost-effective for each of a number of pre-specified or implicit valuations of an outcome improvement by the decision-maker. The CEAC is a comparatively recent development, and there are as yet few published applications of it in the dementia field, but its use will grow. One advantage of the approach is that it makes transparent the trade-offs faced by decision-makers.” Helpfully, Knapp and co-workers have published an analysis of the cost-effectiveness of cognitive stimulation therapy for people with dementia that takes this approach (Knapp et al. 2006), thereby giving us a precedent to follow.

Similarly the protocol devised by Woods et al. (2009) includes measures to capture the costs of the intervention, using the validated Client Services Receipt Inventory (CSRI). Again, this is another model to follow.

Despite the complexities of assessing the value, the advantage of the tools for measuring impact (listed earlier) reduce the different aspects of a person’s life into a series of numbers which lend themselves to such a quantitative approach One of the intentions behind the consensus approach was to ensure that those numbers mean the same thing whenever they are used across Europe. So, despite my qualms about such reductionism, the numbers form the basis of a common language.

The questions that come out of this are: Are the measures advocated above common amongst practitioners working with people with dementia? Are these approaches politically persuasive?

Having got this far, and not knowing the answers to those questions, I’m going let this thread lie for a little while.


Curtis, L. (2010) Unit Costs of Health & Social Care 2010, PSSRU, University of Kent: Kent. Available at http://www.pssru.ac.uk/uc/uc2010contents.htm [Last accessed 13/10/11]

Diaz, S.P.R. et al. (2005) ‘The need for a consensus in the use of assessment tools for Alzheimer’s disease: The Feasibility Study (assessment tools for dementia in Alzheimer Centres across Europe), a European Alzheimer’s Disease Consortium’s (EADC) survey’, International Journal of Geriatric Psychiatry, 20, 744–748

Dye, C. (2008) What is a human life worth? Public Lecture at Gresham College, 24 April, 2008. Available at http://www.gresham.ac.uk/lectures-and-events/what-is-a-human-life-worth [Last accessed October 10, 2011]

Hughes, D. (2010) Health Economics: an essential tool for decision-makers, Liverpool: National Prescribing Centre. Available at http://www.npc.co.uk/local_decision_making/resources/Health_Economics.pdf [Last accessed 13/10/11]

Katona, C., Livingston, G., Cooper, C., Ames, D., Brodaty, H. & Chiu, E. on behalf of the Consensus Group (2007) ‘International Psychogeriatric Association consensus statement on defining and measuring treatment benefits in dementia’, International Psychogeriatrics, 19, 3, 345–354

Knapp, M. (2007) ‘Economic outcomes and levers: impacts for individuals and society’, International Psychogeriatrics, 19, 3, 483–495

Knapp, M., Thorgrimsen, L., Patel, A., Spector, A., Hallam, A., Woods, B., & Orrell, M. (2006) ‘Cognitive stimulation therapy for people with dementia: cost-effectiveness analysis’, British Journal Of Psychiatry, 188 , 574-580

Moniz-Cook, E., Vernooij-Dassen, M., Woods, R., Verhey, F., Chattat, R., de Vugt, M., Mountain, G., O’Connell, M., Harrison, J., Vasse, E., Dröes, R.M. & Orrell M. For The Interdem Group (2008) ‘A European consensus on outcome measures for psychosocial intervention research in dementia care’, Aging & Mental Health, 12, 1, 14-29

Moniz-Cook, E., Vernooij-Dassen, M., Woods, B., Orrell, M. & Interdem Network (2011) ‘Psychosocial interventions in dementia care research: The INTERDEM manifesto’, Aging & Mental Health, 15, 3, 283-290

Nicholls, J. et al. (2009) A guide to Social Return on Investment. Available at http://www.neweconomics.org/publications/guide-social-return-investment [Last accessed 13/10/11]

Olazarán, J. et al. (2010) ‘Nonpharmacological Therapies in Alzheimer’s Disease: A Systematic Review of Efficacy’, Dementia & Geriatric Cognitive Disorders, 30, 161–178

Wimo, A. (2007) ‘Clinical and economic outcomes – friend or foe?’, International Psychogeriatrics, 19, 3, 497–507

Woods, B., Spector, A.E., Jones, C.A., Orrell, M. & Davies, S.P. (2005a) ‘Reminiscence therapy for dementia (Review)’, Cochrane Database of Systematic Reviews, Issue 2. Art. No.: CD001120. DOI: 10.1002/14651858.CD001120.pub2.

Woods, R.T et al. (2009) ‘Reminiscence groups for people with dementia and their family carers: pragmatic eight-centre randomised trial of joint reminiscence and maintenance versus usual treatment: a protocol’, Trials, 10, 64ff


About Bruce Davenport

Museum educator and researcher.
This entry was posted in dementia, measurement of impact. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s