Cornwall’s STP: Why we need to see what’s going on

— This post can be downloaded as a pdf here. —

When we are told ‘Don’t worry: no decisions will be taken before the public are consulted’ we have every reason to worry!

Right now work is going ahead on preparing a Sustainability and Transformation Plan (STP) for health and social care in Cornwall and the Isles of Scilly. Under the brand name Shaping our Future (SoF), a team of people are beavering away to produce a ‘plan’ of some kind. It seems that this is due to be unveiled in a few months’ time, and is to take the form of a Pre-Consultation Business Case.

For anyone who is concerned about what the ‘plan’ will mean for them and their family and neighbourhood, the problem is this. During the planning process, as in all planning processes, things happen to narrow down the options that are available at the point when the public are eventually consulted. For example:

♦  Time and staff resources are used up. Once a deadline has been set, there is simply not the time and manpower to go back to square one and start again. If no alternatives have been investigated and considered, there is no time now to investigate them. Even if alternatives have been considered, these will have been reduced to a shortlist, and it won’t be possible to resurrect any that didn’t make it on to the shortlist.

♦  The staff working on the plan become psychologically committed. They take decisions about the kind of plan they want to see, and about working methods, i.e. how they are going to produce the plan: to go back on these decisions, and write off some of the work that has been done, will generate stress and involve loss of face. Similarly, people make assumptions and become wedded to these: even if evidence turns up that shows that their assumptions were unrealistic or – like budgetary limitations – could have harmful consequences, they may disbelieve and deny that evidence (the phenomenon of cognitive dissonance).

And the staff working on the plan will have their own motivations. They will have personal ambitions, such as advancing their careers, and typically will want to produce something distinctive and striking. Such ambitions too generate commitment.

♦  The planning team may have a budget for research, and agree a contract with an outside research outfit for it to do this work. This contract will set out ‘terms of reference’ for the work, e.g. to explore and evaluate the implications of possible plans X and Y. Then as the planning team forges ahead it may become apparent that there’s a third possibility, option Z. But it may be that the research budget has all been used up, and there is no money to explore and evaluate option Z. So option Z gets ruled out by default, long before a point of formal decision is reached.

♦  Some people, particularly members of powerful interest groups, will have easier – ‘preferential’ – access to the planning process than others. For example, in the NHS senior managers and consultants may get together privately to find and agree a course of action that suits both groups. Once that agreement has been reached, it will be very difficult for members of allied health professions, let alone the public, to get it reviewed and altered.

♦  The situation on the ground changes. A facility may be closed for repairs, and it happens that staff drift away, local people at first protest and then find ways of coping, the building gets starved of maintenance and begins to decay, so the option of restoring it to use becomes increasingly expensive and consequently difficult to justify (the ‘planning blight’ syndrome). Or a service that up to now has been provided at local level gets abruptly withdrawn, to be replaced on grounds of economy by centralized provision, transferring costs to patients who have more travelling to do and importantly making it difficult to restore the service at local level.

Never underestimate the effect of these dynamics. By closing off options they pre-empt formal decisions. As a former very senior civil servant has put it: ‘The experience of anyone who has worked in Whitehall is that there is an early stage in any project when things are fluid; when, if you are in touch with those concerned and get hold of the facts it is fairly easy to influence decisions. But after a scheme has been worked on for weeks and months, and has hardened into a particular shape, and come up for formal decisions, then it is often very difficult to do anything except either approve it or throw it overboard.’* He might have added that there is invariably a huge penalty attached to throwing it overboard at that stage.

It is true that the widespread criticism of the previous ‘engagement’ over the STP for Cornwall caused the process to be restarted. However, this appears to be very much the exception that proves (in the sense of ‘proofs’, or tests) the rule. Anyone who bets on this happening again would be virtually certain to lose their money.

We have to be alert to the fact that managers who are familiar with these phenomena can deliberately take advantage of them. Thus they can limit the number of staff working on the plan and impose unrealistic deadlines, they can impose stringent budget limitations, they can reach deals with the more powerful interest groups, and they can change the situation on the ground. Indeed, they may have reached their present positions in their organizations precisely because they are skilled in using strategies like these.

So while these managers may be absolutely correct when they say that no formal decisions will be taken before the public are consulted, they may have a host of strategies under way to guarantee that they get the result they want. We have every reason to worry!

What can we do about it?

Public money is being spent on producing plans for new health and social care arrangements. It is my contention that those spending it should be directly accountable to the public for their work, not via paymasters with their own personal and political agendas. Such accountability requires – and it is an essential and fundamental requirement – transparency.

Transparency needs to be good enough to enable us to follow every step in the planning process. So we need to see what terms of reference the planning team have been given. We need to see what they tell the Transformation Board or whoever it is that supervises them, and what instructions and ‘steers’ they get from that Board. We need to know what skills the members of the planning team have, so we can see whether there are gaps in their range of skills. We need to know what contracts they give to researchers and management consultants, so we can see for ourselves what limitations are built in to these contracts, and we need to see the reports that these people produce. We need to know about deadlines, so we can gauge the impact of these on the process. And we need to keep a close eye on the situation on the ground, so we can judge how day-to-day decisions are closing off options for the future.

In a nutshell, we need to see what’s going on!

– – –

*Lord Bridges (Cabinet Secretary 1938-46, Permanent Secretary to the Treasury and Head of the Home Civil Service 1946-56), ‘Whitehall and Beyond’, The Listener, 25 June 1964. Cited in Peter Levin, ‘Opening up the Planning Process’, in Stephen Hatch (ed), Towards Participation in Local Services, Fabian Tract 419, 1973 and in Peter Levin, Making Social Policy, Open University Press 1997 (p.44).

 

 

Community hospitals under threat: Are decisions being taken on scrappy information and limited understanding?

— This post can be downloaded as a pdf here. —

Work on the Sustainability and Transformation Plan (STP) for health and social care in Cornwall and the Isles of Scilly, now rebranded as Shaping Our Future (SOF), has reached a critical stage. We risk being gulled into earnest round-table discussions while an organizational bulldozer lightly camouflaged with poor quality information lumbers towards us. Community hospitals are under threat and decisions are evidently being taken on scrappy information and limited understanding.

This report examines recent developments and draws some pessimistic conclusions, but goes on to suggest a more positive way forward.

Recent development (1): Local area workshops have been held.
In July 2017 half-a-dozen ‘local area workshops’ were held across Cornwall. They brought together people from three groups: people who work in the health and social care services, people who work for voluntary bodies (the ‘third sector’) in delivering those services, and people who have recently received support from these services, or their full-time carers.

The stated aim of these three-hour events was ‘to ensure that any changes to the health and social care system achieve our aims of improving the health and wellbeing of our local population by improving the quality of local health and care services in ways that are financially sustainable’, and the idea was that the three groups of participants would work together, setting aside organizational roles, ‘to help produce detailed options for what local people can do themselves, for local community support, and for local services’. The term ‘co-production’ is sometimes used as shorthand for working-together endeavours of this kind.

So patients and carers have sat round a table with service providers and members of voluntary organizations, and (as I have witnessed) some worthwhile exchanges have taken place. At the time of writing this, we wait to see what written records emerge from these exchanges and how frankly they reflect them. And, importantly, what can be built on them.

Recent development (2): Information packs of poor quality have been handed out.
At each local area workshop, an information pack was handed out. It included a section on ‘The case for change’, and here we find four snippets of quantitative data on hospital services (p.5):

(1) ‘Around 60 people each day are staying in acute hospital beds in Cornwall and they don’t need to be there.’

Comment This statement could be extremely misleading, because it does not necessarily follow that people who no longer need acute treatment are fit to go home, although this seems to be the conclusion we are intended to draw. It may be that many of these people are awaiting transfer to a community hospital. Evidently the Royal Cornwall Hospitals Trust does not collect data on the fitness of patients for a particular destination. It exists within its own ‘silo’, not looking beyond its own boundaries, so we are given this number out of context. Also we aren’t told what date or period the figure of 60 people refers to: it is a snapshot taken at an unspecified time.

(2) ‘35% of community hospital bed days are being used by people who are fit to leave.’

Comment Similarly, we are not told what destinations the people in community hospital beds who are judged ‘fit to leave’ are fit to go to, or on what date or over which period this percentage figure was gathered.

(3) ‘83% of admissions to community hospitals are from acute services compared to 42% nationally.’

Comment While this statement does recognise the connection between acute and community hospitals, and it does provide a national comparison (though we aren’t told whether ‘national’ denotes England, Great Britain or the UK), we could draw quite different inferences from it.

We could infer that community hospitals are coming under greater pressure in Cornwall than elsewhere to take patients who have been discharged following acute treatment, and conclude from this that Cornwall needs more community hospital beds or more provision in care homes and/or people’s own homes.

Or we could infer that among the ‘national’ population there is a need for care that is different from ‘step-down’ (rehabilitation) from acute treatment, and that in other parts of the country this need is being met by community hospitals, whereas in Cornwall the community hospitals are close to being monopolized by patients moving out of the acute hospital at Treliske. Maybe part of the pressure that we know is experienced by the Emergency Department at Treliske comes from patients who, elsewhere, would be admitted directly into a community hospital. Again, we could conclude that Cornwall needs more community hospital beds.

(4) ‘62% of hospital bed days are occupied by people over 65 years old.

Comment It is impossible to draw any sensible conclusion at all from this piece of information. We are given nothing to compare it with. We aren’t told whether this figure applies to Cornwall & IoS only or to a larger area, and when, or whether the proportion has increased over recent years. 65 years was formerly the state pension age in the UK for men. For women state pension age was formerly 60. At present those ages are in the process of being both raised and equalized, so the very benchmark of 65 years seems entirely arbitrary.

As we see, the information presented in the information pack to justify reorganizing our hospitals – a mere four statistics – is of extremely poor quality. It is scanty and arbitrary. It is silo-dominated, and indeed it could be taken to imply that managers do not care where discharged patients go so long as they vacate their beds. The fact that acute hospitals count numbers of patients while community hospitals count percentages (another silo effect) makes it very difficult to compare the stress that they are under.

Moreover, the information presented is in the form of snapshots. These represent situations at particular points in time or over particular periods. We are not told at what particular points in time, or over what particular periods, the data were collected. This is not good professional practice. And such truncated information ignores dynamics, how situations change over time, and – importantly – it largely ignores processes that link organizations together, such as the movements of patients through the health and social care system. Failing to state the sources of data or the relevant dates or periods it applies to is an unprofessional way of managing and presenting information.

To sum up: As a basis for taking decisions, information of this calibre is not fit for purpose and is likely to pull wool over the eyes of people who don’t have specialized knowledge. It is not the demonstrated outcome of careful research, and no serious reasoning from it is presented. So we may conclude that it is offered to support a case, to sell a message, the message seemingly being that Cornwall is over-supplied with community hospital beds and consequently some community hospitals should be closed.

Recent development (3): The Transformation Board say they support the co-production approach, but they are not taking responsibility for ensuring it is put into practice.
At the head of the Shaping Our Future (SOF) operation is the Transformation Board, which has 20-plus members, mainly appointed as representatives of various organizations. The key members are Kate Kennally (Chief Executive, Cornwall Council), who is in the chair; Kathy Byrne (Chief Executive, Royal Cornwall Hospitals NHS Trust); Phil Confue (Chief Executive, Cornwall Partnership Foundation NHS Trust); and Jackie Pendleton (Interim Chief Officer, Kernow Clinical Commissioning Group).

Until recently the Transformation Board met every month, but as from May 2017 it will meet only every two months. The papers that are presented to the Board are not published, and nor are the minutes of its meetings until they have been approved by the following meeting, so it can be the best part of four months before the public gets to know what was on the agenda and what was agreed. (The latest minutes currently available are those of the May 2017 meeting.) It appears, then, that the four leading members will have considerable autonomy.

At the May 2017 meeting of the Transformation Board it was reported, under the heading of ‘Communications and engagement strategy: co-production plan’, that ‘senior communication and engagement support had been secured … with a view to them taking over the strategic lead for this work’. This refers to the secondment to SOF of the current NHS Regional Head of Stakeholder Engagement and the Regional Head of Communications and Engagement Specialist Projects. The minutes also record that ‘[there] was full support from Transformation Board members for the co-production approach’. And Jackie Pendleton of KCCG is recorded as saying: ‘It is important to be able to give answers to questions raised in the first phase of events to show the public that we have listened and we will continue to listen.’

While we might take encouragement from this, other items recorded in the minutes of the May meeting are worrying. We learn that the leading Board members want to be a ‘first wave Accountable Care System’ and have put in an application to NHS England to that effect. And we learn that it was proposed to establish two ‘delivery groups’ (‘Model of Care’ and ‘System Reform’), and that the ‘Programme Board’ was to be renamed the ‘Portfolio Board’, ‘to reflect the scale and scope of work’. No mention is made of publishing the minutes of these bodies. In a nutshell, this is all about fashioning the organizational bulldozer.

What recent developments tell us
1. Experience of the local area workshops tells us that it is possible to bring together round a table people who work in the health and social care services, people who work in the voluntary sector and people on the receiving end of services. Particularly noticeable is the part played by stories – accounts of personal experiences – in these conversations.

2. The quality of the information packs on the subject of how the system works is so abysmally poor, although lavishly illustrated, that it could be taken to indicate either incompetence, i.e. a lack of appreciation of what data are significant and why, or, more sinisterly, an attempt to coerce the reader into conceding without quibble that a case has been made for change. A third possible explanation for these poor quality scraps of information on hospital services presented to the public in the information pack may simply be that they were deemed good enough for us. If that was so, it indicates an attitude of condescension, if not contempt, towards the public. This can only breed suspicion and contempt in return: not a healthy state of affairs for our society and no basis for genuine co-production.

3. At Transformation Board level there is no shortage of expressions of goodwill towards public engagement and co-production. But it may be that in some quarters this is seen as a useful manipulative tool, a means of ‘selling the message’ and ‘nobbling’ influential lay people so they won’t be obstructive when proposals are published. Tellingly, among the proposals for reorganizing the SOF enterprise there is no mention of a co-production delivery group. And the two engagement specialists who have been brought in from regional level have been given the absolute bare minimum of support staff. Judged on these criteria, within the SOF set-up engagement and co-production have a very low priority indeed.

The way forward
1. The local area workshops have shown that when people talk about their experiences of health and social care services they are invariably telling a story. One of the great values of stories, along with their authenticity (the fact that they represent genuine experiences as recounted by the people who experienced them) is that they are dynamic. Unlike the information presented as snapshots in the information pack, they tell us what took place over a period of time and how the different parts of the system interacted with one another. It follows that those in charge of ‘engagement’ have the task of finding a way of taking these stories and using them, drawing lessons that can be fed into the planning process.

There is an unfortunate tendency among academically-schooled planners, in all disciplines, to disparage the worth of individual stories. If they are acknowledged at all, they are placed in a category labelled ‘anecdotal evidence’, a rank some levels below statistical evidence. (Some will recognise this as a variant of Gresham’s law: ‘Quantitative evidence drives out qualitative.’)

This was vividly illustrated in the information pack, where the reader was presented with a jumbled mass of data, collated and summarized in the form of figures and charts, and presumably expected to draw conclusions from it, or to accept the conclusions that other people have drawn.

There is no obvious, meaningful way in which individuals’ stories about their experiences can be integrated into the kind of ‘framework’ presented in the information pack.

Here is a little true story that illustrates that point. A 95-year-old Penzance woman living in her own home fell and broke her hip (fractured neck of femur). An ambulance took her to Treliske, where they mended her hip very efficiently, but she was left with some damage to the skin of her leg, so she was not fit to go home. She needed to go to a community hospital for rehabilitation. A bed became available for her in Edward Hain community hospital, but only after several days, during which she was kept in the noisy and disorientating Trauma Ward, where she visibly deteriorated. After a week in Edward Hain, however, which provided the necessary therapy in a calm atmosphere, she was able to return home and continue her recovery in the care of her daughter, district nurses and carers.

What this story does, like many others, is to draw our attention not only to this one ‘statistic’ but also to patients’ ‘trajectories’ through the hospital and social care system. These trajectories reveal with perfect clarity the interactions that linked the parts of the system: acute hospital, community hospital and home in this case.

Here, then, is a role for the public, to contribute the stories that transcend organizational boundaries, and give an insight into the interactions at work (or that fail to work) and the obstacles that hinder effective interactions.

And the corresponding role for those in charge of ‘engagement’ must be to find ways of searching out these stories, of acknowledging their validity, and – in feeding them into the planning process – of drawing out and responding to the lessons to be gained from them. This is the role that future local area workshops should play. And the people in charge of this stage must be given the human resources that they need to enable them to perform this role.

2. The information pack, as we have seen, provided numerical information about hospital services that was scanty, arbitrarily selected, silo-dominated, and in the form of snapshots, and its presentation did not meet professional standards. If this is all that is available to the planners, they will be forced to work on the basis of ‘design principles’, ‘models’ and other such general abstractions, and any attempts on their part to foster public engagement will be – and will be seen to be – a waste of time and effort. If this very poor presentation denotes an attitude of condescension towards the public, again, attempts to foster public engagement will come to nothing.

In either case, the planners need to be challenged to demonstrate their professional competence by researching and presenting a much sounder – properly investigated and more complete – information base, one that both takes a view of the system as a whole and covers patients’ encounters with and trajectories through the health and social care system. They need to know why, for example, 83% of admissions to community hospitals are from acute hospitals compared to 42% nationally. This information needs to be set alongside, checked against and integrated with information from patients’ stories.

We don’t know who the planners are. Are they KCCG staff? People hired on short-term contracts to work on SOF? Or are they employees of the American firm of consultants, GE Healthcare Finnamore who are also working on SOF? The information pack contains no information about who wrote it, who supplied the information in it, who supervised it, or who authorized it for publication. It would be reassuring if these people were prepared to be publicly accountable for what they do. It is to be hoped that they will respond to this challenge, whoever they are, because it is seriously against the public interest for far-reaching plans, policies and public expenditure to be based on the pathetically flimsy foundation that we have seen so far.

3. Within the Transformation Board, some transformation needs to take place. Halving the number of meetings, holding meetings in private, not publishing the agenda in advance and the minutes for several months after the meeting: these are characteristics of a body that is uncomfortable with transparency and shuns the light. It and the people working for it seem to be more concerned with securing power, autonomy and status than with doing a good job for the public they ostensibly serve. For starters, let us have more transparency and let us have a co-production delivery group.

Peter Levin

7 August 2017

 

Appendix: Bed occupancy in community hospitals

Here is a table drawn from data supplied two years ago by Peninsula Community Health (PCH), which formerly administered community hospitals in Cornwall: it shows that in May 2015 all of the community hospitals in Cornwall were running at an average bed occupancy rate of 90% or more, with several registering more than 95% and in one case a staggering 99.4%.

The position was clearly very tight in 2015, and presumably it still is. On average, in only five of the 17 community hospitals in Cornwall itself was it likely that a bed could be found on a randomly-picked night for an emergency case (see Column G). Strikingly, in 2017, there was no information whatever in the information pack on bed usage and availability in community hospitals. Is that information no longer collected? Or is it collected but someone consciously choose not to include it in the presentation? Certainly, the bed occupancy figures in the table provide a prima facie case for more community hospital beds in Cornwall, not fewer.

The information presented in this table is informative but it is not a sufficient basis for understanding how the system works, because it is silo-dominated. No doubt PCH managers could pride themselves on making extremely efficient use of their resources two years ago, but what were the consequences for the managers at Treliske? They can have had no day-to-day control whatever over resources on which they crucially depended. The two sets of managers were working in separate compartments – silos – from one another.

Occupancy of Community Hospital Beds in Cornwall, May 2015

A B C D* E F G
Hospital ward May 2015 average occupancy (%) Beds (as at June 22nd) Bed nights available in May [C x 31] Bed nights taken up in May

[B x D]

Bed nights not taken up in May

[D – E]

On average, one or more beds available every night
Bodmin Anchor 94.1 12 372 350 22 No
Bodmin Harbour 92.8 23 713 662 51 Yes
Bodmin Woodfield (stroke) 98.2 9 279 274 5 No
CRCH Lamorna (Camborne-Redruth) 99.0 23 713 706 7 No
CRCH Lanyon (stroke)
(Camborne-Redruth)
99.4 21 651 647 4 No
Edward Hain (St Ives) 92.7 11 341 316 25 No
Falmouth 97.3 24 744 724 20 No
Fowey 90.6 10 310 281 29 No
Helston 98.0 24 744 729 15 No
Launceston 93.3 19 589 550 39 Yes
Liskeard Oak 94.2 25 775 730 45 Yes
Liskeard Willow 94.5 19 589 557 32 Yes
Newquay 95.8 19 589 564 25 No
SACH Harold White
(St Austell)
95.6 22 682 652 30 No
SACH Heligan
(St Austell)
95.7 22 682 653 29 No
Stratton 90.6 13 403 365 38 Yes
St Barnabas (Saltash) 90.3 9 279 252 27 No
St Mary’s, IoS 23.5 10 310 73 237 Yes

*For Column D it is assumed that the June 2015 figure for the number of beds was applicable throughout May.

Note 1 The above table contains no information on turned-away patients. It appears that figures on these were not kept.

Note 2 In May 2015 the number of community hospital beds in Cornwall (excluding the Isles of Scilly) was 305. Of these, only 11, at Edward Hain Hospital, were located in Penwith. There is clearly some geographical imbalance here, especially as Edward Hain is not now in use. These are supposed to be community hospitals.


NB As stated on the front page of this website: ‘All posts on this website are my own responsibility … and should not be taken as representing the views of West Cornwall HealthWatch or the Citizen Advisory Panel.’

 

Communications and engagement in health and social care: A cautionary tale from Cornwall

This post can be downloaded as a pdf here

Executive summary
This report demonstrates why, in the field of health and social care, responsibility for communications and engagement should not be combined in a single position. It tells the story of Cornwall’s Sustainability and Transformation Plan from late 2016 to mid-2017, focusing on the problems of securing ‘engagement’ by the public. It reveals how communications between health bodies and the local authority, Cornwall Council, have come close to breaking down, and identifies the root cause of this as the combining of responsibilities for communications and engagement in a single position.

Communication: necessary but difficult
Under the terms of the Cornwall devolution deal, the organizations responsible for health and social care in the Duchy are supposed to be getting together to produce supposed to be getting together to produce a business plan for the integration of health and social care services. Health care is the responsibility of NHS Kernow (the clinical commissioning group), a number of ‘provider’ Trusts, and independent GP practices, while social care is the responsibility of Cornwall Council, the local authority, with its elected members and salaried officers. The ‘third sector’, made up of charities and voluntary bodies, also plays a significant part in both health and social care.

With such a wide range of people involved, from a wide variety of types of organization, it is crucial that they communicate with one another. Communication is not easy. There are always obstacles to be overcome – lack of trust, differences in rank, organizational loyalties pulling in different directions, people covering their own backs, professional rivalries, different views of what is important, competition for funds, and so on. So to communicate effectively everyone has to make an effort. And just one person can foul everything up.

The STP survey
Recent work on the integrated care plan for Cornwall – it used to be known as the ‘Sustainability and Transformation Plan'(STP) but has now been rebranded as the ‘Shaping our Future’ plan (SoF) – reveals the problem.

Over the winter of 2016-17 a questionnaire-based survey was carried out in Cornwall. Its wording was widely criticized. The questionnaire elicited responses from 1896 people (fewer than 1 in 250 from Cornwall’s total population of more than half a million). I learned four weeks ago, by accident, that a team based at the University of Exeter had examined the data collected and produced two reports: a summary report and a detailed analysis. These reports, which can be read here and here, have never before been placed in the public domain.

We learn from the University of Exeter reports that while 1896 people completed the survey, around 30% of them – i.e. around 570 – were a ‘health and care professional or support worker’. So only 1330 or so were ‘lay’ respondents, members of the general public. Moreover, among the 1896 there were no fewer than 762 references (around 40% of all respondents) to ‘needing more information’ and 352 references (around 18% of all respondents) to ‘not understanding the question’.

Further analysis of the data will show whether the 570 health and care staff had difficulty with the questions: if in the main they did not, it could follow that as many as 50% of lay respondents felt they needed more information and as many as 25% of them did not understand the question. These are not insignificant proportions.

In their summary report (p.16) the University of Exeter authors noted that the consequence of many participants needing more information and not understanding the question is that ‘caution should be taken in drawing firm conclusions from the data. Specifically with regard to inferring that a high level of ‘agreement with priorities’ translates to an overall participant endorsement of the STP’.

The involvement of Cornwall councillors
Cornwall Council has a Health and Adult Social Care Overview and Scrutiny Committee (HASCOSC). It set up an STP sub-committee which took a close interest in the STP survey. It held several information-gathering sessions, with the final one on March 10th, 2017.

The University of Exeter summary report is dated March 3rd, 2017, i.e. a week before the final STP sub-committee’s final information-gathering session. (The team’s detailed analysis is dated March 30th, but a version was presumably available by March 3rd or it would not have been possible to summarize it.) However, the summary report was not shown to the sub-committee.

Following its meeting on March 10th, the STP sub-committee went on to submit a position statement to HASCOSC, in which it concluded, among other things, that ‘the process of engagement with the public was inadequate and seriously flawed [and that] the questionnaire contained closed questions, was ill conceived and was unprofessional’. They were ‘disappointed that the report from the engagement events was not available when they were considering this issue’. ‘The engagement process was poor and ill-judged.’

HASCOSC met on March 15th. The Committee had in front of it the STP sub-committee’s position statement and an update report from the SoF Communications and Engagement Lead, Garth Davies (who is also Associate Director of Communications and Engagement at the Royal Cornwall Hospital Trust). Tabled at the meeting was Mr Davies’s own report on the ‘engagement activity’. He was asked if advice had been sought on the structure and nature of the questionnaire. The minutes of that meeting tell us that he ‘confirmed that advice was taken from the University of Exeter’. Later on March 15th I emailed Dr Michael Leyshon of the University of Exeter team and asked him: ‘Were you involved in designing the survey questionnaire?’ He replied the following day: ‘The simple answer to your question is no.’

So Mr Davies had withheld the University of Exeter reports and instead written and published his own views on the ‘engagement activity’. His report included (p.5) the following statement:

Overall, it is clear that respondents supported the priorities and case for change set out in the Shaping Our Future documents with many saying it is hard to disagree with the positive approach. Respondents said the top priorities should be ‘Prevention and improving population health’ and ‘Integrated care in the community’.

As we see, he effectively disregarded the academics’ warning against inferring that a high level of ‘agreement with priorities’ translates to an overall participant endorsement of the STP.

It is relevant here to note that both of the University of Exeter reports carry the statement: ‘This report has been produced for the sole use of the NHS Communications team, namely Garth Davies, in order to write the STP Engagement Report. The report is not for circulation nor use by any other party.’ Given that the reports draw on responses from the public and have been paid for with public funds, that prohibition is clearly wholly unmerited.

Interestingly, while the summary report reveals that among the 1896 responses to the STP survey there were 762 references to ‘needing more information’ and 352 references to ‘not understanding the question’, these figures do not appear in the detailed report: evidently some editing to remove embarrassing findings took place between March 3rd and March 30th.

In an email to a colleague on April 24th, Mr Davies wrote: ‘I will publish the analysis reports and survey summary report on the website.’ Assuming that the website to which he referred is www.shapingourfuture.info, by 11th July 2017, more than 11 weeks later, the reports had still not appeared.

On March 17th, the SoF Transformation Board (see below) met. The chair, Kathy Byrne, is recorded in the minutes as saying that the STP sub-committee’s position statement ‘was not particularly helpful for public confidence and noted that the framing of [it] has made the job harder’. She also ‘noted the importance of working more closely in the future’.

Members of Cornwall’s public might perhaps share my view that withholding information from councillors and misrepresenting survey findings – and continuing to do so – does the very opposite of inspiring me with confidence, nor can I see how it might contribute to ‘working more closely’.

The engagement report: what it tells us and what it doesn’t
I noted above that while 1896 people completed the survey, around 30% of them – i.e. around 570 – were a ‘health and care professional or support worker’. So only 1330 or so were ‘lay’ respondents, members of the general public. The engagement report tells us more. Of the 1896, 1258 were of working age (19-65). These working-age respondents will have included nearly if not all of the health and care professionals/support workers. So that group will have contributed around 570 to the 1258 working-age respondents. In other words, nearly half of the working-age respondents (45%) were health and care professionals or support workers.

People employed in health and social care amounted to around 13% of the working-age population of Cornwall in 2011. So they were considerably over-represented among respondents to the survey. By no stretch of the imagination, then, can the survey be said to have been a survey of a representative sample of the general public.

Unfortunately the engagement report does not distinguish between the responses of lay people and health/care workers. It lumps everyone together as ‘respondents’ or ‘people’. So when Mr Davies concludes in his engagement report ‘Overall, it is clear that respondents supported the priorities and case for change set out in the Shaping Our Future documents with many saying it is hard to disagree with the positive approach’ we have no idea of how far these results could have been skewed by the over-representation of health/care workers. And if we look at how those priorities were set out in the survey, we see that they mostly took the form of ‘motherhood and apple pie’ aspirations, inherently impossible to argue against.

And here’s another thing. If you look at the pie chart below, which shows the six priorities offered in the STP questionnaire, you will see that another priority, ‘Transforming urgent and emergency care’, tied for second place (at 19%) with ‘Integrated care in the community’.


Evidently Mr Davies chose to include ‘Prevention and improving population health’, along with ‘Integrated care in the community’, as top priorities, but to exclude ‘Transforming urgent and emergency care’ although it came equal second in the ratings. This ‘cherry picking’ is not an acceptable way of presenting the results of a survey. And it leads us to ask what else has been cherry picked to go in the engagement report.

When we read on, we learn that ‘The main concerns people identified were:

Potential reduction in community hospitals with concerns about travel times and the impact on the major hospitals without alternatives yet in place.

Financial with people wanting to see more investment in community services and many questioning whether the plans could be delivered within the budget.

Workforce with people wanting to see more investment in community staff and training.’

It is hard to match these up with the ‘top priorities’. And the second and third of these concerns could well reflect the over-representation of health/care workers among respondents.

The use of impressionistic language in the engagement report – ‘Many people commented …’, ‘Many said …’, ‘People wanted …’, ‘Others thought …’, ‘People felt …’, ‘Many respondents said …’,‘An overwhelming majority of people agreed …’ – and the withholding of actual numbers (we are never told how many is ‘many’) means that we have to depend on the author’s predilections and judgment. The language used and the instance of cherry picking from the pie chart must raise doubts as to the reliability of everything the engagement report says.

Recent developments: (1) The Transformation Board
Ideally one would want to see the unfortunate STP survey and engagement report consigned to history. Is this happening? Seemingly not.

Work on the ‘Shaping our Future’ plan is now being overseen by a Transformation Board, which is chaired by Kathy Byrne, who is also Chief Executive of the Royal Cornwall Hospital Trust. Its membership includes some officers and elected members of Cornwall Council, as well as numerous ‘system leaders’ from health provider bodies. (Members’ names and affiliations and the minutes of its meetings can be seen here.)

Ms Byrne submitted a report on Shaping our Future for the meeting of HASCOSC on July 12th, 2017. It includes a section on ‘Consultation and Engagement’ which reads as follows:

Between November 2016 and February 2017, local people were asked to give their views on the Shaping Our Future outline proposals through a survey, written responses or by attending a series of community and stakeholder events.

Over 5000 local people responded or took part in the events and said top priorities should be ‘Prevention and improving population health’ and ‘Integrated care in the community’.

As we can see, this is simply talking up the numbers and parroting the engagement report. As in that report, ‘Transforming urgent and emergency care’ does not appear as a top priority despite tying in the ratings with ‘Integrated care in the community’. What are we to conclude? It appears that Ms Byrne has rubber-stamped the engagement report, rather than subjecting it to careful and critical examination.

Sadly, we may expect similar behaviour from other members of the Transformation Board. Having been set up as a ‘stakeholder body’, its members are likely to read documents that come their way solely with a view seeing how the interests of the body that they represent could be affected. 

It follows that there is a great need for members of Cornwall Council’s Overview and Scrutiny Committee to bring their knowledge and critical faculties to bear and do some thorough scrutinizing. They should have investigative staff to help them in this: it is not a job for the secretariat that normally supports council committees. Ideally such staff should be able to link to voluntary and campaigning bodies, since many of these have good local knowledge and grasp of the policy process. 

Recent developments: (2) Working with people from local communities
In her report to Cornwall Council’s Health and Adult Social Care Overview and Scrutiny Committee on July 12th, Ms Byrne noted: ‘Two engagement experts have been secured from NHS England and the South West Commissioning Support Unit to help shape our engagement programme.’ One of these, Dr Lou Farbus, is now running a series of ‘co-production’ workshops across Cornwall and the Isles of Scilly, in which people who have recently received or who care for someone who has recently received some kind of health or social care support can meet people who are involved in delivering a service, to exchange views and perspectives and explore possible ways forward. Having myself taken part in one such workshop, I can vouch for their effectiveness in broadening one’s mind, in helping one to appreciate the work that other people do and the issues that they face, and in stimulating constructive, ‘out of the box’ thinking.

A fatal flaw in the NHS
A common feature of NHS bodies is that they bundle together responsibilities for communications and engagement. These are fundamentally different activities. Communications experts, who in many cases have had a training in journalism, are trained to use the media, to ‘put the message out’ and put a positive gloss on it, even to ‘spin’. In contrast, successful engagement involves dialogue, two-way communication: it requires skills in listening, in appreciating what others are saying, in responding appropriately, gaining other people’s confidence, being open and ‘straight’ with them, and able to negotiate compromises. These are two different skillsets.

The experience of the STP in Cornwall constitutes just a single case study, but it shows what can happen, and did happen in a particular situation. It has highlighted the differences between communications and engagement, and there are indications that these may cause problems elsewhere, not just in Cornwall. So perhaps the main lesson to draw from the experience of the STP in Cornwall is this: Do not combine responsibilities for communications and engagement in a single position. And indeed, beware the communications expert who fancies himself (or herself) to be an expert in engagement too.

Peter Levin

www.spr4cornwall.net

 

The STP engagement survey: What have we learned?

This post can be downloaded as a pdf here.

Background
Work is currently going on to produce a health and social care plan for Cornwall and the Isles of Scilly. It is officially known as a Sustainability and Transformation Plan (STP). A draft Outline Business Case (OBC) was published in October 2016, followed shortly after by what was described as a summary of the OBC, entitled Taking Control, Shaping our Future. In that document some elements of the OBC don’t appear, while others appear to have been added. The latter publication incorporated a questionnaire-based survey, the STP survey.

The STP survey was the second to be undertaken within the space of a year. In January 2016 NHS Kernow (Kernow Clinical Commissioning Group) and Cornwall Council published a health and social care integration questionnaire. As a survey questionnaire this was seriously defective in a number of ways, as I described in a report for West Cornwall HealthWatch entitled How not to run a health and social care survey.

The recent STP survey questionnaire put serious problems in the way of anyone trying to respond to it. In a post in January this year I set out the reasons why I was finding it so difficult to answer the questions in it. Last month (March 2017), Cornwall Council’s sub-committee on the STP, on the basis of the evidence they received, said they found the survey to be ill-conceived’ and ‘unprofessional’. So what can be done with the responses received to it? In March the STP team produced two reports on the engagement process, an interim ‘update report’ and a ‘final report’, which drew heavily on the responses to the STP survey. In the following sections of this report I examine the final report and ask what credence can be attached to its conclusions, given the defects of the survey.

Nearly half the working-age respondents were health and care professionals or support workers
The update report tells us that 1896 people completed the survey. But the final report reveals that 30% of them – i.e. around 570 – were a ‘health and care professional or support worker’. So only 1330 or so were likely to approach the survey primarily as members of the general public. Of the 1896, 1258 were of working age (19-65). These working-age respondents will have included nearly if not all of the health and care professionals/support workers. So that group will have contributed around 570 to the 1258 working-age respondents. In other words, nearly half of the working-age respondents (45%) were health and care professionals or support workers.

People employed in health and social care amounted to around 13% of the working-age population of Cornwall in 2011. So they were considerably over-represented among respondents to the survey. By no stretch of the imagination, then, can the survey be said to have been a survey of a representative sample of the general public.

The response to the survey was tiny
To get a sense of scale: the figure of 1896 is about one-third of one per cent of the current resident population of Cornwall and the Isles of Scilly (around 550,000) so it’s a tiny sample. Notably, the figure of 1330 non-health or social care workers who responded to the survey is little over half the 2450 who completed the previous health and social care survey in early 2016, further evidence of the off-putting nature of this latest survey.

The reports present very little quantitative data
We see that in the final report there is a remarkable dearth of quantitative data. Other than a breakdown of respondents by age, and an unclear map showing responses by postcode, a pie chart on p.13 showing ‘First choice priority’ is the only significant quantitative evidence presented. We are told: ‘Overall, people responded favourably to the priorities set out in the Shaping our Future documents and many thought it was hard to argue with the positive approach.’ Since those priorities, as set out in the survey, mostly took the form of ‘motherhood and apple pie’ aspirations, difficulty in arguing against them should not be surprising.

Very little analysis is put forward
Even with the limited data collected, some search for correlations would have been possible. Were there significant differences in responses by age or geographical location? Asking such a question could have generated useful information about the likely impact of future ageing of the population or difficulties with physical access to healthcare facilities in certain parts of Cornwall. And were there significant differences between health and care professionals or support workers and the general public? The ‘top priorities’ could have been very different for the two groups, and indeed it is hard to see the man and woman in the street spontaneously saying what is needed is ‘System reform to achieve better care’ or ‘Preventing and improving population health’ (sic) as in the pie chart below. It is a matter for regret that no search for correlations appears to have been carried out.

This approach lends itself to ‘cherry picking’
There is a striking illustration of the way that this approach allows the writers of reports to pick and choose what they include. The final engagement report says: ‘Respondents said the top priorities should be ‘Prevention and improving population health’ and ‘Integrated care in the community’’. If you look at the pie chart below you will see that another priority, ‘Transforming urgent and emergency care’, tied for second place (at 19%).


Evidently the author of this document felt able to pick and choose for presentation the information that supported a particular case. That is not an acceptable way to present the results of a survey.

Impressions are no substitute for facts
Instead of figures, what we find – on almost every page of the engagement report – are impressions: ‘Many people commented …’, ‘Many said …’, ‘People wanted …’, ‘Others thought …’, ‘People felt …’, ‘Many respondents said …’,‘An overwhelming majority of people agreed …’. We are very rarely given the actual numbers. We are never told how many is ‘many’. We must depend on the author’s predilections and judgment, his interpretation of responses and their impression of how close together these responses are. This is what allows them to be lumped together. The language used and the instance of cherry picking from the pie chart, described above, must raise doubts as to the quality of analysis and interpretation.

Using impressionistic language like this allows cherry picking of survey responses that support a particular view. Considerable passages of the survey questionnaire and the reports are couched in ‘management-speak’. This language may be appropriate shorthand for internal documents, but it is not appropriate for those that are public facing. The use of this style may in itself influence the choice of responses to be quoted in the report, making it more likely that those using a similar linguistic style will be chosen.  At the time of writing it is understood that the completed questionnaires will be made available for independent scrutiny, so it may be possible to discover whether this was indeed the case.

The University of Exeter: a key player?
The introduction to the final report has this to say about its authorship:

This report has been written by Garth Davies, Associate Director, Communications and Engagement for the Shaping Our Future programme but with special thanks to the Social Innovation Group at the Centre for Geography, Environment and Society at the University of Exeter for their data analysis and independent view on the findings. Dr Michael Leyshon at the University of Exeter has confirmed that the report is a ‘fair account of our analysis’.

Suffice it to point out (1) that the report does not actually present any data analysis; (2) that Dr Leyshon, a social and cultural geographer whose academic profile does not mention expertise in the NHS or social survey research methods, was not involved in the design of the survey questionnaire; and (3) that the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care, South West Peninsula, which includes the Universities of Plymouth and Exeter as well as local NHS organizations, and undertakes high-quality applied health research focused on the needs of patients and supports the translation of research evidence into practice in the NHS, was not involved in any aspect of the STP survey process.

Evidently what we have here is an attempt to give the STP survey an academic stamp of approval which it does not deserve.

‘Engagement’: At best nothing more than a case of unsystematic communal brainstorming
The process by which those in charge of the STP attempted to engage with the public took two main forms: a series of meetings and the STP survey. The meetings provided an opportunity for local people to give vent to their feelings about health and social care issues, such as the closure of community hospitals, and these have been reported, albeit in a general and impressionistic fashion, in the final engagement report.

As a survey, this exercise has been seriously deficient. At best it has amounted to nothing more than a kind of unsystematic communal brainstorming. Such an exercise may be productive in terms of ideas and highlighting matters that have been overlooked, but it is not a serious tool of social investigation. That much could have been predicted from the questionnaire itself, composed as it was of unanswerable questions, leading questions and questions that look like multiple choice but don’t actually offer a choice (such as questions which ask ‘To what extent …?’ but don’t offer a scale on which to register ‘extent’).

Governance issues
Work on the STP is in the hands of a team of officers under the supervision of a Transformation Board. The Board has four leading members: the Chief Executives of Cornwall Council (who is in the chair), Royal Cornwall Hospitals NHS Trust and Cornwall Partnership NHS Foundation Trust, together with the interim Chief Officer of NHS Kernow (Kernow Clinical Commissioning Group).

There are two important questions to be asked about this organizational structure:

(1) How was it that the design and implementation of the STP survey came to be entrusted to a specialist in public relations, self-identified as Associate Director, Communications and Engagement, and an employee of the Royal Cornwall Hospitals NHS Trust, rather than to someone with experience in social surveys and research methods?

(2) Over the past year West Cornwall HealthWatch has pointed out the inadequacies of the surveys that are supposed to underpin the Sustainability and Transformation Plan, and in an attempt to be constructive we have actually produced a set of guidelines for managers on the design and use of survey questionnaires. Why has the Transformation Board shown no sign of taking on board what has been spelled out to them?

We don’t know what the process was that led to the assigning of a social research project to a public relations specialist. But the failure to learn from the experience of the earlier health and social care survey is seriously worrying. If there is one thing that an innovative project, such as transformation, calls for it is the ability to learn.

The inability of an organisation to learn has a number of unhappy consequences. Mistakes are repeated, with a consequent waste of time and money. Communication channels become narrowed or blocked off, creating a hierarchical structure that supports the status quo. This prevents learning from other people engaged on a similar task elsewhere who are doing things differently. Addressing the public inappropriately with ‘management-speak’, using words such as ‘theme’, ‘vision’ and ‘priorities’, may lead to further spiralling misunderstanding and frustration. We have seen all of these in the present case.

When it comes to public engagement, we need to ask whether the four leading official bodies actually ‘get it’. NHS Kernow’s Governing Body includes a Lay Member, who is supposed (by law) to help to ensure that, in all aspects of its business, the public voice of the local population is heard and that it responds in an effective and timely way to feedback and recommendations from patients, carers and the public. The individual concerned worked for 31 years in the NHS, and at the point when he retired was Chief Executive of two primary care trusts. While his experience may well make him a valued member of the Governing Body, we may legitimately question whether it enables him to speak with authority for the public.

Recently the Director of Healthwatch Cornwall (no connection with West Cornwall HealthWatch!) has been invited to join in the deliberations of the Transformation Board. While this arrangement may be of value, the fact is that she has spent 18 years in the NHS in a variety of senior management and leadership roles. Again, we may legitimately ask whether such experience is an unmixed blessing when it comes to listening to people’s experiences of publicly funded health and social care services and thereby contributing these to the decision-making process.

To negotiate between the institutional world of health and social care provision and the ‘real’ world in which we all live requires an openness of mind along with the ability to learn and the ability to mediate between very different cultures. It is not immediately apparent that long experience in the institutional world confers and demonstrates such openness and abilities.

Postscript
The final engagement report has a message on branding. It says: ‘To give our plan a strong sense of local identity going forward we are referring to our five year plan as ‘Shaping our Future’ rather than the STP.’ We may justifiably wonder how excluding any reference to ‘health’, ‘social care’, ‘plan’ and Cornwall and the Isles of Scilly is calculated to impart ‘a strong sense of local identity going forward’.

Peter Levin

9/4/2017


Update
We have recently learned that since the final engagement report was published, the STP team has taken on additional expertise in the form of Dr Lou Farbus. She is the NHS’s Regional Head of Stakeholder Engagement and is currently on secondment to the Cornwall and Isles of Scilly STP team. She has many years’ experience of teaching questionnaire design and applying this skill specifically to service redesign programmes. She is currently in the process of putting together a package of engagement related training modules, to include questionnaire and survey design, which will be delivered to relevant members of the STP team.

West Cornwall HealthWatch warmly welcomes Dr Farbus in her new role, especially in the light of our experience with STP engagement so far, and we shall be glad to cooperate with her in any way we can. It is heartening to learn that someone in NHS Kernow has – albeit belatedly, we have to say – taken our point about the need to strengthen the STP team to handle engagement and surveys.

13/4/2017

 

Cornwall councillors’ verdict on STP engagement process: ‘ill-conceived’ and ‘unprofessional’

STP survey was ‘ill-conceived’ and ‘unprofessional’
On March 15, 2017 the Cornwall Council sub-committee set up to keep tabs on the Sustainability and Transformation Plan (STP) produced a report on the STP Outline Business Case and the process of engagement with the public, undertaken through a series of meetings and a survey. About the engagement process it had this to say:

In our view the process of engagement with the public was inadequate and seriously flawed. There was inconsistency of information provided dependent on the facilitator and this has to be rectified. The questionnaire contained closed questions, was ill conceived and was unprofessional. Members of the public were left feeling alienated, angry and frustrated. (My italics)

You can read their report, described as a ‘position statement’, here. The chair of the sub-committee told the Health and Adult Social Care Overview and Scrutiny Committee (its ‘parent’ committee), that a fuller report, complete with supporting evidence, would follow.

At the time of writing this post, that fuller report is still awaited. Meanwhile, there is some grim satisfaction to be afforded by a Cornwall Council body having investigated what lies behind the ‘spin’ rather than merely rubber-stamping papers put before them.

Peter Levin

 

An introduction to the design and use of survey questionnaires, for NHS and local authority managers

Some good-practice guidelines

by Dr Peter Levin (West Cornwall HealthWatch), January 2017

[This report can be downloaded in pdf format here.]

Guideline 1.  Surveys are research tools: employ trained and experienced people to design and administer survey questionnaires.

Guideline 2:  Be clear about the purpose of the survey: is it to be used for monitoring, to aid problem-solving, or to inform a planning process?        

Guideline 3.  Acknowledge ‘political’ motivations, and set them aside. 

Guideline 4.  Be clear about what you want to find out from the survey, and how you might use the findings.

Guideline 5.  Allow for the possibility that respondents may want to use the questionnaire to tell you things that you haven’t thought of.

Guideline 6.  Your survey should be focused and concise, and not used for a ‘fishing expedition’.

Guideline 7.  Identify your target population.

Guideline 8.  Take expert advice on the size and composition of the sample that you need in order to get reliable data, and on how to reach your sample.

Guideline 9.  Take precautions when commissioning ‘easy-read’ and other versions of questionnaires.

Guideline 10.  Decide what information you are going to present. 

Guideline 11.  Don’t formulate questions in ‘management-speak’, and do ask respondents to draw on their own experience.

Guideline 12.  When asking a question that invites a judgment, offer an     appropriate scale.

Guideline 13.  Avoid complex questions.

Guideline 14.  Avoid leading questions. 

Guideline 15.  Run a pilot version of your questionnaire. 

Guideline 16.  Give trained and experienced researchers the task of collating, analysing and reporting findings and drawing conclusions.

Preface

Clinical commissioning groups (CCGs) are required, under their constitutions, to involve the public in planning, developing and considering proposals for changes. They are expected to encourage patients and the public to examine and give feedback on plans for commissioning services and contribute towards decision-making on any proposals for changes in commissioning arrangements that would have an impact on service delivery or the range of services provided. CCGs should work in partnership with patients and the local community to secure the best care for them. The same expectations can be found in the contracts held by the frontline bodies that actually provide health care services. Local authorities too are under duties to consult and ‘engage’ with ‘stakeholders’, duties given extra force by the expectation that policies should be ‘evidence-based’.

In Cornwall, the Kernow Clinical Commissioning Group, the Cornwall Partnership Foundation Trust and Cornwall Council (the local authority with responsibility for social services) have alighted on the questionnaire-based survey as an appropriate means of securing such ‘engagement’. But to date these bodies sometimes seem not to have fully appreciated that designing a questionnaire to be completed by members of the public, and then analysing the responses received, require specialist knowledge, skills and understanding. In short, running a questionnaire-based survey is a social research project.

Social research is the process of systematically gathering, analysing and interpreting information about the behaviour, knowledge, beliefs, attitudes and values of human populations. Such research is intrinsically difficult: people are highly complex, and language is imprecise; human beliefs, values, attitudes and motivations are hard to pin down; there are invariably ethical issues to be faced; memory is fallible, and research respondents are not always able or willing to report their feelings or behaviour accurately or honestly; and there are considerable statistical problems in drawing valid inferences about large and shifting human populations.1

Researchers have to grapple with these problems afresh in every project. So good research needs craft skills (of observation and reasoning) and intelligent creativity and ingenuity in the way those skills are applied, together with the ability to understand the way people think, feel and behave.

The aim behind this paper is that it should serve as an initial step towards a set of guidelines that NHS managers – the managers of healthcare providers and commissioning bodies – and local authority managers can use to draw up a clear brief for the researchers and assure themselves that at the very least some basic errors in questionnaire design will be avoided.

Peter Levin

Penzance, January 2017


Guideline 1. Surveys are research tools: employ trained and experienced people to design and administer survey questionnaires.

The task of designing a survey questionnaire should be undertaken by people who have the requisite training and experience, so they can bring social research skills and an appreciation of ethical issues to bear. It follows that designing a questionnaire – including writing the ‘preamble’, the information that accompanies it – is not a task to be assigned to a public relations specialist or back-office administrator. It requires trained and experienced researchers.

Bear in mind that recommendations based on the findings from a questionnaire that has been rigorously designed, carefully deployed and written up on the basis of wide experience are likely to be given more careful attention, carry more conviction and be given more weight than others that lack these qualities.

Guideline 2: Be clear about the purpose of the survey: is it to be used for monitoring, to aid problem-solving, or to inform a planning process?

Tasks of different kinds require different approaches. Is there an on-going situation, such as the day-to-day delivery of a service, that you want to monitor? Or is there a problem, a ‘what shall we do about X?’ issue, that needs to be resolved? Or are you in the process of creating a plan, e.g. to reorganize a service or create a new one, in which case you may want to use one questionnaire early in the exercise to discover opportunities and possible ‘elephant traps’, and another, different one later in the process to help choose between alternatives that you have identified. Because monitoring, problem-solving and planning (early and late in the process) require different approaches, it is essential not to muddle them.

Guideline 3. Acknowledge ‘political’ motivations, and set them aside.

In the fields of health and social care there are always many interests in play. Senior managers have an interest in gaining a reputation for decisiveness and meeting their targets and ‘running a tight ship’; politicians have an interest in getting their constituents to re-elect them; professionals have an interest in doing a job successfully and having their success recognised, and in defending their autonomy; providers, who are always under pressure to limit their spending, have an interest in successfully bidding for funds; treasurers and finance officers have an interest in showing that they are tough in scrutinizing those bids; and so on.

Hence there is the danger that people may propose a survey because they see it as a means of advancing their interests, especially if they anticipate being able to control the analysis and publication of the findings and conclusions. They may also, in the present-day era of ‘engagement’, and especially in the case of planning major changes, see a survey as a means of showing that there is ‘community support’ for their proposals. (The use of leading questions is one familiar technique for doing this: see Guideline 14 below.) Having control of completed questionnaires allows selective ‘cherry-picking’ of supportive comments, while those not supporting the desired proposals can be ignored. This behaviour is unethical. It may also be counterproductive, since outside observers, in campaign groups and the media, may be able to reveal and publicize that strategy, with consequent reputational damage.

The only ethical use for a survey is to elicit information – facts and opinions – in a way that is unbiased, that does not favour or penalize any particular interest. 

Guideline 4. Be clear about what you want to find out from the survey, and how you might use the findings.

Monitoring: Here you might want to collect facts about the use of a service that are not collected as a matter of routine through the service’s immediate providers, e.g. about the journeys that service-users make. And if you want the service-users’ opinions of the service,  these can be gathered by asking them to rate the qualities of a service, e.g. on a scale from 1 (Very Poor) to 10 (Excellent) or by asking them to set out their opinions in a comment box.

An important potential use of a monitoring survey is to establish correlations between categories of service-user and their use and opinion of the service. Researchers should be tasked with identifying significant (relevant) categories: these might typically include age, gender, marital or partnership status, household type, and access to transport, as well as types and levels of need (e.g. disability) for the service. A search for correlations can show whether there exist certain groups of service-user who are receiving a lower standard of service than others, a situation of which you will want to be aware.

A monitoring survey that is repeated annually, with little or no change from year to year, will enable a comparison to be made of performance over time, so managers can see whether there is a trend, or indeed a significant departure from a trend, that demands attention.

Problem-solving: A ‘what shall we do about X?’ issue faced by a large and complex organization is liable to be centred in one or two departments but have ramifications throughout the organization and the world beyond. Questionnaire-based surveys can help you to explore these ramifications and use them to build a picture of the situation, understand the mechanisms in play, and go on to formulate possible courses of action and sketch out the likely impact, the consequences, of each. Such a questionnaire should be tailored to each of the groups involved – e.g. actual and potential service-users, staff in the department or departments most affected, members of different professional groups, staff in other departments. It should ask respondents to describe the situation from their own experience and give their opinions (on causes and effects and possible ways forward) based on that experience, rather than asking them to put themselves in the place of managers.

Planning: Early in the process, the task will be to identify and clarify issues. Typically, there will be one or more ‘imperatives’, already existing or foreseen, such as political pressures, spending heading for an overrun, staff shortages or an impending review by an outside body. Some of these issues may become apparent from monitoring exercises, but a survey that is focused on an issue and invites views on ways of responding can be better suited to the situation. Again, a questionnaire should be tailored to the different groups involved.

Later in the process, when possible alternative courses of action have been identified, the objective will be to evaluate and compare these possibilities. A survey can be used to discover what respondents anticipate will be the likely impact of each one on themselves, i.e. what they see as the likely real-life consequences (outcomes) of implementing each alternative, and whether there is agreement that the criteria being applied to evaluate them are appropriate. Respondents should always be asked to make judgments on the basis of their own experience, not as if they were managers having to allocate resources among competing services.

Guideline 5. Allow for the possibility that respondents may want to use the questionnaire to tell you things that you haven’t thought of.

While you do want to be clear about what you want from the survey (see Guideline 4 above), the very fact of issuing a questionnaire implies that you accept an obligation to take note of the responses that you get, and, if you don’t accept some requests or suggestions, to explain why not. So you need to recognise that to some extent you are surrendering control of the situation. You need to keep an open mind and be prepared for this, rather than go in with a determination to get what you want, otherwise the situation may become confrontational, with unpleasant results.

A practical step that you can take is to include at the end of your questionnaire the question: ‘Is there anything we haven’t covered so far in this questionnaire that you would like to tell us about?’ This would be particularly appropriate in a pilot (trial) questionnaire – See Guideline 15 below – but should not be out of place in the version that you actually use.

Guideline 6. Your survey should be focused and concise, and not used for a ‘fishing expedition’.

The guidelines presented here consistently follow the principle that surveys need to be focused. You may be tempted to ‘include a question about X’ in a questionnaire on the basis that it might attract some interesting answers. Respondents may judge such a question to be irrelevant and distracting, and possibly intrusive too. Indeed respondents may be antagonized, which could affect how they respond to the whole of the questionnaire. By all means invite comments, e.g. by providing boxes for them, but questions that amount to ‘fishing expeditions’ will detract from the validity and effectiveness of the survey as a whole, so the temptation to include such questions in a questionnaire should be firmly resisted.

As a general rule, questionnaires should be kept as concise as possible. A good test that can be applied is to ask about every potential question: ‘What shall I be able to do with the answer?’ If there will be nothing you can do except say ‘Fancy that!’ the question should not be included.

Guideline 7. Identify your target population.

Who are the people whose experiences are of concern to you? Are they past or present service-users (patients or clients), or potential future ones (the public at large)? Of service-users, are they children? or adults who depend on carers to speak for them? or people who need help with completing questionnaires? (The report on a questionnaire-based survey carried out in Cornwall in early 2016 stated that 0.3% of the 2,450 respondents were aged under 11: one has to wonder how able these 70 or so children were to answer questions such as ‘What are the three most important things to you when you experience health and social care services and support in Cornwall?’2) Do you want – or should you want? – to hear from the people who actually provide services face-to-face – in GP surgeries, acute hospitals, community hospitals, urgent care centres, minor injury units and other clinical settings, or in patients’ and clients’ own homes? Your questionnaire will need to be tailored to each target group.

Guideline 8. Take expert advice on the size and composition of the sample that you need in order to get reliable data, and on how to reach your sample.

‘Reliable data’ is data on which you can safely base your organization’s policies for future action, data in which you have confidence. For example, if you want to gauge support for proposed changes to a service, you will want many more than a handful of replies to a survey of opinion, because the mere handful might be completely unrepresentative of the population likely to be affected. You will want to be confident that you have replies from a representative sample of your target population – e.g. one which includes all age groups, all the different localities within the geographical area, patients from all the GP surgeries within that area, and people from across the whole range of health and social care ‘need’ groups – and a sufficiently large number within each group. (You may also want to ensure that responses from individual patients and clients are not outweighed by organized responses from groups with special interests.)  Precisely what constitutes a sufficiently large number is a matter for a statistician to determine: lay people should not take it upon themselves to make such judgments.

There is a great deal of accumulated experience of distributing survey questionnaires, among official bodies, market research organizations, etc. Since not everybody is online and not everybody has a landline telephone connection, the only way to ensure blanket coverage of a geographical area is by door-to-door distribution of printed questionnaires, although even this has the possible disadvantage that a multi-person household receives only a single questionnaire. Coverage of a group of patients or clients can of course be achieved by sending questionnaires through the post to members of that group only.

Lately we have seen questionnaires made available by placing them on a website (such as  that of Cornwall Council), leaving copies at public libraries and in the waiting rooms of doctors’ surgeries, and handing them out at public meetings. The coverage attained in this way may be poor: the health and social care survey carried out in Cornwall in early 2016 achieved 2450 responses from the public: not a tiny number in total but a mere 0.5% of the Duchy’s half a million population. In effect, this survey amounted to nothing more than a brain-storming exercise. But unfortunately even this minimal response rate yielded enough comments for the organizers to cherry-pick ones that enabled them to say: ‘You told us that you want to see …’ No compilation of all the comments received has ever been published. This is not acceptable.

Guideline 9. Take precautions when commissioning ‘easy-read’ and other versions of questionnaires.

Nowadays it is possible to have a draft questionnaire ‘translated’ into an easy-read version, to be published alongside the original one. Typically the easy-read one will be profusely illustrated with sketches, cartoons and pictograms, and the language will be simpler. The easy-read version should not be adopted and published without careful scrutiny. The search for simpler wording may have led to changes in meaning, possibly not ones that are visible to the translators, resulting in inconsistencies between the two versions. As a consequence the responses that are received to the two versions might not be comparable in some respects, and this might not be apparent at first sight. So the two versions should be rigorously checked for consistency.3

Of course, if one version of a questionnaire is easy to read, it may be that the other is in some respects difficult to read. As a matter of good practice, you should always refer back to the original to see if its wording could be simplified, or indeed if the wording of the easy-read version could be substituted.

It is now common for a questionnaire to be published in paper-based form (including a pdf format for printing out and completing by hand) and on-line, where the responses can be typed in on the screen and submitted electronically. Again, paper-based and on-line versions should be closely checked for consistency.

Guideline 10. Decide what information you are going to present.

The preamble, or introduction, to a questionnaire gives you an opportunity to present background information, such as your reasons for carrying out the survey. At the forefront should be a description of the situation that respondents can recognise: ‘At present we provide …’ and/or ‘You may have noticed …’. This immediately conveys to respondents that they have a stake in the process that is under way. Likewise, if you are engaged in a comprehensive review of health services in a locality and seeking respondents’ views on current provision and future possibilities, you could open with a simple list of what is available – e.g. GP surgeries during working hours, out of hours service, urgent care centre, minor injury unit, A&E Unit of District General Hospital, NHS 111, Ambulance (999) – and continue with the statement ‘We are going to ask you which of these you would prefer to use in certain circumstances’. This would be comprehensible to potential respondents and would give them the confidence that they know what they are being asked about. It would also help to reassure them that those who designed the questionnaire know what they are doing.

By the same token you should definitely not open your preamble by outlining the financial or other contextual pressures that managers are under: this conveys the impression that these come first in managers’ minds, and it also fails to provide an immediate ‘hook’ to capture respondents’ attention.4

Guideline 11.   Don’t formulate questions in ‘management-speak’, and do ask respondents to draw on their own experience.

A common failing of health and social care questionnaires is that the questions are formulated in ‘management-speak’, and this is a language that is foreign to respondents, using as it does terms such as ‘priorities’, ‘model’, ‘integrated’ (as in ‘An integrated 111, GP out of hours and urgent care centre model’), ‘pathways of care’ and ‘system reform’. Not only do questions such as ‘To what extent do you agree with each of our priorities?’ make use of language that will mean nothing to an ordinary member of the public: such questions require respondents to put themselves in the position of managers, a considerable intellectual feat.5

A good question will be expressed in language that allows respondents to relate it to, and answer from, their own experience. Where comments are invited, it will allow and encourage them to tell their story, in their own words. And this is where lessons can be learned about their experiences of ‘the system’, and about how they reacted to them.

Guideline 12. When asking a question that invites a judgment, offer an appropriate scale.

If you want to ask a question that invites a judgment – such as ‘How good …?’ or ‘To what extent do you agree …?’, how do you do it? You can simply provide a box for comments, so that people can use their own words to reply. This presents a difficulty for respondents in finding the right words and for you in aggregating their responses. There are two alternatives. One is to provide a sequence of labelled comment boxes –  e.g. ‘Very poor’, ‘Poor’, ‘Adequate’, ‘Good’, ‘Excellent’ – and ask respondents to tick one. The other is to offer a scale, e.g. from 1 (Very poor) to 10 (Excellent) and ask respondents to pick a point on that scale that represents their view. What you should not do (to take a recent actual example) is ask a question of the form ‘To what extent do you agree …?’ and then offer the boxes ‘Agree’, ‘Neither agree nor disagree’, ‘Disagree’ or ‘Don’t know’: this question invites a ‘scale’ answer – a measure of ‘extent’ – but the offered format does not permit one.6 Again, it is a basic principle of questionnaire design that questions should not confuse the respondent.

Guideline 13. Avoid complex questions.

Here, from a recent survey on ‘NHS 111 and out of hours service integration’, is an example of a complex question: ‘What was your reason for contacting NHS 111?’7 This is difficult to answer because it packs two questions into one: ‘What prompted you to call a health advice/treatment service?’ and ‘Why did you choose to call NHS 111as opposed to some other service or to visit a service in person?’ These two questions should be asked separately. To take another example, the same questionnaire asked: ‘How do you feel the NHS 111 service dealt with your call?’, to be answered on a scale from ‘very poor’ (1/10) to ‘excellent’ (10/10). This did not allow the respondent to say ‘I was kept waiting a long time but the advice I got was very helpful’. There should have been separate questions about waiting time and quality of the response. Note too that complex questions are not only difficult to answer: they distract the respondent from ‘going with the flow’ of the questionnaire.

Guideline 14. Avoid leading questions.

A leading question is one that suggests to respondents the answer that should be given to it, that puts an answer into their mouths. There are striking examples in a questionnaire recently distributed to clients of one of the NHS Trusts in Cornwall and designed by the Trust itself. It contains seven questions of the form: ‘Do you agree that … services that we provide are doing well?’ Respondents are offered the choice of ‘Yes’, ‘No’ or ‘Don’t Know’ as their reply.[8]

This is a doubly leading question: first, because asking ‘Do you agree?’ puts the onus on the respondent to justify not agreeing with the proposition (agreeing with it requires no justification); and second, because offering no choice other than ‘doing well’ similarly puts the onus on the respondent to justify not agreeing. Because of these built-in biases against disagreement, the results are inherently not trustworthy: they cannot be taken as genuine indicators of client satisfaction. The lesson for the designers of questionnaires is that leading questions should be avoided like the plague.

Respondents could of course have been asked ‘How well are our services doing?’ and offered a numerical scale on which to respond: this would have been a neutral way of posing the question. (Interestingly, this questionnaire also invited respondents to provide feedback in the form of comments: ‘From your experience, do you have any suggestions on how we could improve … services?’ Putting the emphasis on ‘your experience’ makes this a good question.)

Guideline 15. Run a pilot version of your questionnaire.

It is a cardinal principle of good practice that a questionnaire should be ‘piloted’, tried out on a sample of respondents. This is best done by sitting with a person who is filling it in, noting any difficulties that they have, discussing those difficulties with them and trying alternative wordings out on them. A trial enables you to see whether a question is self-explanatory and easy to comprehend. If a respondent answers a question with a query of their own, e.g. by saying ‘What does that mean?’, it is not a good question. Likewise, if the respondent finds it difficult to answer, or can see more than one possible answer, that is an indication that the question needs changing. Piloting a questionnaire can also show that a question is using overly-technical language or is complex and needs to be ‘unpacked’. (See Guidelines 11 and 13 above.)

However much experience you have, and with the best will in the world, having drafted a questionnaire and ‘lived with it’ through the drafting process, your perception of it will be different from that of a respondent freshly confronted with it. Piloting enables you to see it from the respondent’s point of view, and this can be invaluable in helping you to ‘get it right’, with consequent benefits for the insights you gain and the value of the report that you produce.

Guideline 16. Give trained and experienced researchers the task of collating, analysing and reporting findings and drawing conclusions.

Ideally the researchers who drew up the questionnaire will have the task of analysing and reporting the findings. They will be sensitive to subtleties and nuances in the ways that respondents respond, especially if they have also piloted the survey.

Most questionnaires in the field of health and social care will yield information of two kinds: quantifiable data, capable of being aggregated and examined for correlations, etc., and comments, which will be much less easy to collate and draw worthwhile conclusions from. Working with comments is undoubtedly a task for experienced researchers: they will have an eye for recurring topics and will be alert to the significance of variations in the language that respondents use.

Most importantly, as people with well-developed skills of observation and (usually) a background in science subjects, experienced researchers are used to seeing the world in terms of phenomena, to looking for behaviours and mechanisms, and to applying rigour in observation and the treatment of observations. In contrast, graduates in subjects where essay-writing, rather than field and laboratory work, is the norm, tend to see the world in terms of ‘themes’, a word which repeatedly appears – sometimes as ‘common themes’ and ‘priority themes’, and themes which ’emerge’ – in the report on the ‘engagement’ that took place in Cornwall in early 2016 (Shaping the future of Health and Social Care in Cornwall and the Isles of Scilly).[9] Such people tend to use terms without defining them (e.g. ‘priorities’, ‘pathways of care’ and indeed ‘theme’) and sometimes technical terms wrongly (e.g. ‘variance’, when they mean ‘variation’), perhaps because they aren’t aware that these are technical terms, with precise meanings.

It cannot be emphasized strongly enough that a survey, if its results are to be useful and taken seriously, needs to be undertaken and written up by people who know what they are doing and can approach their task with rigour and understanding.

Notes and references

1. The  UK-based Social Research Association has useful resources on its publications page: http://bit.ly/2jbA6sq  (Last accessed 14/01/2017)

2. Shaping the future of Health and Social Care in Cornwall and the Isles of Scilly: A short report to present the findings from engagement with public and providers to help shape future health and social care provision and improve the wellbeing of our residents (May 2016)  http://bit.ly/2iwtpTb  (Last accessed 14/01/2017)

3. See How not to run a Health and Social Care Survey, as demonstrated by Cornwall Council and NHS Kernow (May 2016) for an illustration of discrepancies between types of questionnaire.  http://bit.ly/2jviIyp  (Last accessed 14/01/2017)

4. For an illustration of this defect, see Questioning a Questionnaire: Was this one fit for purpose? (On the NHS 111 and out of hours service integration questionnaire published by NHS Kernow in October 2016)  (December 2016)  http://bit.ly/2jWOtQe   (Last accessed 14/01/2017)

5. Have your say on the Cornwall and the Isles of Scilly Health and Social Care Plan 2016-2021  (December 2016)   http://bit.ly/2jQG19f   (Last accessed 14/01/2017)

6. As Note 5.

7. As Note 4.

8. Cornwall Partnership NHS Foundation Trust, Council of Governors Members Survey 16.  (Undated but 2016)  http://bit.ly/2juZ1GU

9. As Note 2.

This post can be downloaded in pdf format here.

The STP survey: Why am I finding it so difficult to have my say?

A letter to the ‘Transformation Board’

Dear Transformation Board

I’ve been reading your document ‘Have your say on the Cornwall and Isles of Scilly Health and Social Care Plan 2016-2021’, the STP. It invites me to have my say and comes with a questionnaire which is supposed to enable me to do that. I’m finding it a real struggle to have my say using your questionnaire. Eventually I worked out why.

The questions all ask to what extent I agree with their ‘priorities’. Question 1 listed the following six priorities:

•   Prevention and improving population health
•   Integrated
care in the community
•  
Transforming urgent and emergency care
•  
Redesigning pathways of care
        (best practice treatment for specific conditions)
•  
Improving productivity and efficiency
•  
System reform to achieve better care

As you can see, some of these so-called priorities take the form of ‘motherhood and apple pie’ objectives, with which no sane person could possibly disagree: e.g. ‘improving population health’, achieving ‘better care’, ‘improving productivity and efficiency’. And others have no built-in objectives at all: e.g. ‘transforming urgent and emergency care’ and ‘redesigning pathways of care’. Transform and redesign with what objective, I asked myself. I didn’t get an answer. Why not? Because this is looking like gobbledygook.

The very idea of ‘priorities’ is a nonsense in this context. We need everything here that contributes to keeping the population in good health, and a spread of resources among them. ‘Priority’ implies that you deal with the highest priority first, then the next: that is clearly not appropriate here.

Maybe managers have a different view of the world, and are comfortable with the language of priorities. But in asking me questions framed like this, they are asking me to put myself in the position of a manager, and to take an overview of the whole system. I am just a member of the public, not a manager, and ‘management-speak’ is not my language.

I can only judge your ‘priorities’ and recommended approaches by envisaging how they would work out in practice. You aren’t giving me that information, information that I need in order to ‘engage’ in the planning process.

Finally on Question 1, you ask ‘To what extent …’ but you aren’t offering me a scale to register ‘extent’, just the two extremes of ‘Agree’ and ‘Disagree’ and the indeterminate one of ‘Neither agree [nor] disagree’. It’s looking as though you are deliberately trying to confuse me!

Question 2 opens with the statement: ‘Health and care services must be delivered within the budget available.’ That is so objectionable! It implies that we must accept whatever that budget is: I want to see a case made, by demonstrating the impact on the ground, for increasing the budget to meet the needs of the people of Cornwall. So I have a question for the Transformation Board: ‘Whose side are you on?’ If you were on our side you would be showing us the impact of the cuts you’re saying we must face up to.

Question 3 says: ‘We recommend investing a minimum of £20 million over 5 years in preventing people getting ill, supporting self-care and targeting citizens who are most likely to have health problems. We believe that focusing resources on preventing ill health is simple common sense and we can do more to keep people healthy, happy and well. Fundamentally, we must also ask people to do more for themselves and support each other in their community. To what extent do you agree with our recommended approach and our prevention priorities?’

Well, I have no way of telling whether £20 million over 5 years is the right amount or not. You are not giving me the information that I need to make a judgment about this: for example what you would spend this money on, whether it represents an increase on last year’s spending, whether the money would come from another programme.

Question 4: ‘We recommend changes to community hospitals so that they become community hubs which offer multiple services to prevent or reduce acute hospital visits. Community hubs will be linked to GP practices providing co-ordinated care and personalised support to keep people well, help people stay out of hospital or leave hospital quicker. Better community and home care should mean less need for community hospital beds and sites so we may reduce these over time, particularly if they need major financial investment. … let us know if you have any alternative suggestions to reducing community hospital beds and sites.’

No-one could possibly object to providing co-ordinated care and personalised support to keep people well’: this is another ‘motherhood and apple pie’ goal. But I have heard so much from senior executives at KCCG and RCHT about the ‘outdated bed-based model of care’ that I suspect that agreeing with your recommended approach will be taken as consenting to your closing community hospitals and continuing your policy of running these down by minimizing maintenance until ‘they need major financial investment’ when – in their blighted state – they will be ‘ripe’ for closure.

This recommendation takes no account of the valuable function that community hospital beds perform by allowing patients who are recovering from acute treatment to ‘step down’ to recuperative care.

Question 5: ‘We recommend changes to General Practice and grouping more GP practices together so they can better meet rising demand and expand the range of services. Right now GPs are spending too much time on administration and their work load could be reduced through targeted actions such as more effective self-care, early detection, better use of technology and a more flexible workforce. … let us know if you have any alternative suggestions to improve the sustainability of GPs.’

I can only judge these recommendations by envisaging how they would work out in practice. You aren’t giving me the information that I need. But agreeing with your approach could be taken as assenting to closing single practices in outlying places (e.g. St Just), which I would strongly disagree with.

If GPs are indeed ‘spending too much time on administration’ surely what should be done is to identify how the burden of administration can be reduced, and take steps to achieve that. And what would ‘a more flexible workforce’ look like in practice? You don’t tell me, so how can I possibly say whether I agree or not?

Question 6: ‘We recommend an urgent care service that is accessible, reliable and co-ordinated with clinicians at the end of a phone if you need advice. With clinicians visiting you when essential or in Urgent Care Centres so that you only need to visit an Emergency Department in an actual emergency. Better location of Urgent Care Centres (accessible within 30 minutes from homes in Cornwall, on average) should mean we can provide a better, more reliable service than Minor Injury Units but would probably need to be on less sites (sic) so that we could afford them and resource them. To what extent do you agree with our recommended approach and our urgent care priorities?’

Again, I am being invited to agree with ‘motherhood and apple pie’ objectives (accessible, reliable etc), but not told how these would work out in practice. The STP and draft Outline Business Case propose closing all 13 Minor Injury Units and having just three Urgent Care Centres, a reduction of more than three-quarters in places where one can go for treatment. This can only mean a severe reduction in accessibility. Not good!

Question 7: ‘We recommend changing our approach to caring for people with specific conditions such as diabetes, heart disease, stroke, cancer, joint problems and dementia so that citizens get equitable access to high standards of care regardless of where they live or their individual clinician, within the resources available. We want to reduce the number of out of county mental health placements. We also want to explore what other services we can provide locally or what makes sense clinically to provide outside of Cornwall and the Isles of Scilly. To what extent do you agree with our recommended approach and our priorities for redesigning pathways of care?’

‘Equitable access to high standards of care’ – more ‘motherhood and apple pie’ stuff. You don’t say how you would change your approach to caring for people, so – again – I have no idea what the impact on people would be. Why does it take an STP for you to make such changes? And ‘within the resources available’ gives you an easy way out of doing anything genuinely constructive. By all means aim to reduce the number of out of county mental health placements, but – once more – why does it take an STP for you to do that? And you are asking us to comment on your ‘priorities for redesigning pathways of care’, but not actually telling us what you mean by ‘pathways of care’ (if they are just best practice treatments for specific conditions why not just call them ‘treatments’?) or how you would redesign them.

Question 8: We recommend that local care providers change the way they work together to enable joined up care, share expertise and information more effectively and use public sector properties efficiently. A large proportion of the savings we want to make can come from the way we operate and function. Our aim will be to modernise and change organisational form with minimal impact on clinical staff and services. To what extent do you agree with our recommended approach and our priorities for system reform and improving productivity and efficiency?’

Clearly joined-up care, effective sharing of expertise and information and efficient use of public sector properties have to be good things – how could they not be? – but what does it take to bring these about? If I agree to efficient use of public sector properties’ am I consenting to closure of certain (as yet undisclosed) facilities? I need to know what the likely impact ‘on the ground’ would be if I am to be able to answer your questions.’

Peter Levin

17/01/2017

This letter can be downloaded in pdf format here.

The survey form is currently available on the Cornwall Council website here. It will continue to be available here.

NHS Kernow’s NHS 111 and out-of-hours questionnaire: not fit for purpose

Questioning a Questionnaire

In October 2016 NHS Kernow published a ‘Have your say’ questionnaire on the subject of NHS 111 and out of hours service integration. Was it fit for purpose? This post summarises my assessment on its fitness, or lack of it. The full report, which again comprises the original questionnaire together with annotations, can be downloaded in pdf format here. In this post, the annotations are shown in blue.


NHS Kernow’s goal is to commission an excellent health care system for Cornwall and the Isles of Scilly, to improve people’s health and reduce health inequalities.

This ‘motherhood and apple pie’ mission statement – with which no-one could possibly disagree, of course – has nothing to do with the particular problem posed by the existence of two parallel services, 111 and out-of-hours (OOH).

Like the wider NHS, we face a serious and challenging financial position, largely due to the growing demand for healthcare to meet the needs of an ageing population and more chronic conditions. We have a fragmented health and care system, which leads to increasing pressures, particularly in the emergency department.

This paragraph gives prominence, and hence demonstrates priority, not to the problem faced by someone needing health treatment or advice but to the CCG’s financial position and the pressure on ‘the emergency department’ (presumably the A&E departments of acute hospitals).

We have an unauthorised overdraft that grows by around £1million a month. With increasing costs and needs, our projected year end deficit is £53 million. We need to be realistic about our future and just improving what the NHS does is not enough; we need to do things differently across the whole system to close gaps in health and wellbeing, finance and care and quality. Our vision is to create a health and care system that empowers people to improve their health and wellbeing and reduce their dependency on services.

Again, the emphasis is on the ‘whole-system’ finance issue, not on whether there is scope for rationalizing the 111/OOH services. The ‘vision’ of reducing people’s dependency on services is evidently also prompted by financial considerations and has no apparent bearing on the issue of how to integrate 111/OOH services. Empowering people ‘to improve their health and well being’ with no reference to how this might be achieved can be read as suggesting that people adopt a DIY approach to their health care and ‘don’t bother the doctor’.

We need to ensure there are a range of safe and effective services that meet people’s needs, when they need them, including GPs – both in and out of hours; minor injury units and urgent care centres; and NHS 111. Having these in place will support the emergency departments to focus on treating people with the most serious and life-threatening conditions.

This continuing preamble still fails to put the issue from the patient’s point of view. So by not mentioning the whole range of services that someone in need might call or visit – ambulance services are conspicuous by their absence – dilemmas such as ‘Do I call 111 or 999?’ are not addressed. In other words, the emphasis is on institutional rationalizing rather than on assisting patients to make informed choices.

NHS Kernow is now looking to commission a fully-integrated urgent care access, treatment and clinical access service incorporating NHS 111 and primary care out of hours’ services in place for 1 December, 2017. The new service will be compliant with national guidance specified in the Commissioning Standards Integrated Urgent Care document (ww.england.nhs.uk/wp-content/uploads/2015/10/integrtd-urgnt-care-comms-standrds-oct15.pdf).

The new service will provide rapid access to anyone who needs medical advice and treatment when they cannot get to see their GP. We want to improve people’s outcomes and experience of care; make better use of the limited system-wide clinical capacity; reduce unplanned demand for 999 calls and transfers and Emergency Department attendances. It is vital the new service delivers the A&E Delivery Board’s priority that NHS 111 should increase the number of calls transferred for clinical advice in order to decrease call transfers to ambulance services and reduce Emergency Department attendances.

Mainland Cornwall, NHS Kernow’s territory, has, besides a single A&E department, one urgent care centre and 11 minor injury units (MIUs). Given the stated need (see above) for ‘a range of safe and effective services’, and the reiterated importance of reducing demand for ambulance and Emergency Department services, it is surprising that the questionnaire focuses exclusively on 111 and OOH services and ignores the others. Even though the telephone numbers of the MIUs are published, the services they provide are not covered in this questionnaire.

We would expect the successful provider(s) to collaborate with other providers in the system to deliver the standards for an integrated urgent care system. In particular:
· To make better use of limited GP time across all parts of the urgent care system eg minor injury units, emergency departments and urgent care centres.
· To increase clinical triage and intervention in NHS 111 and out of hours to improve public confidence and reduce unnecessary use of the Emergency Department.
· To reduce duplication and costs to create reliable and easy-to-access alternatives to the Emergency Department.

Given that stress is explicitly placed here on integrating ‘all parts of the urgent care system’, it is again surprising that the use of the emergency services, urgent care centre and MIUs is not covered by the questionnaire. No allowance is made for the difficulty that members of the public can experience in deciding whether a situation is an emergency or merely ‘urgent’.

We expect to enter into a contract that provides sufficient sustainability for the potential provider(s) to enable them to invest in our community. The contract will be one that rewards both positive outcomes and experience for patients and incentivises efficiency and value for money.

NHS Kernow is considering a number of potential service models:
1.  An integrated NHS 111 and GP out of hours’ with NHS 111 provision available 24 hours a day, seven days a week.
2.  An integrated NHS 111, GP out of hours’ GP service and Urgent Care Centre model.
3.  An integrated NHS 111, GP out of hours’ service and Urgent Care Centre model that is integrated with in-hours primary care.
4.  An integrated 111 and GP out of hours’ service with reduced in-hours NHS 111 provision.

The language here is doubtless familiar to those who drew up this questionnaire, but it is not apparent to the lay person what is meant by ‘integrated’ and what the implications could be for patients and the public of ‘services’ and ‘models’ being integrated. No explanation is given of what each of the four alternatives ‘would mean for you’.

We are committed to involving people in Cornwall and the Isles of Scilly to ensure the future service meets their needs. We can only do this with your help and would like your views on our proposals. What you tell us will be considered in the planning of the service. Please tell us your thoughts.

To sum up at this point: It is worth looking back at the foregoing script and (a) noting the language and especially the technical terms used, and asking what a lay person would make of them; (b) considering what this tells us about the gulf of understanding between NHS managers and the public; (c) checking the script against the questions themselves, which will reveal the extent to which the questions are designed to shed light on the issues faced and the relative merits of the four potential alternatives; and (d) asking ourselves whether the questionnaire actually meets some basic standards of intelligibility and consistency.

NHS 111 and out of hours service integration

1. Do you know the difference between NHS 111 and the Cornwall GP out of hours’ service?
☐   Yes                   ☐   No                ☐   Not sure

This question should have started by asking: ‘Are you aware of (a) the NHS111 and (b) the Cornwall GP out of hours services?’ It could then have gone on to ask: ‘What would you say is the difference between them?’ But ‘the difference’ is an abstract concept. It would have been better to ask: ‘Which one would you call if the need arose?’

The redundant apostrophe in out of hours’ does not inspire confidence in the proofreading of this questionnaire.

2. Have you used the NHS 111 service?
☐   I use the service      ☐   My friend/family member uses the service
☐   I do not have direct experience of the service

Note the confusion between tenses here. ‘Have you used …?’ relates to the past: ‘I use …’ relates to the present. It is not clear whether the questionnaire is seeking information about past behaviour or present, on-going behaviour. A good questionnaire does not confuse the respondent.

Note too that, looking over the whole questionnaire, we can see that Questions 2-5 and 8-9 are all to do with NHS 111 while Questions 6 & 7 are to do with the relationship between NHS 111 and GP practices. (Questions 10-16 are all on the Cornwall GP out of hours service.) This questionnaire could well have been ordered in a more logical and systematic way, and perhaps divided into sections, which would have made it easier for respondents to comprehend it and get to grips with it.

3. How often have you used the NHS 111 Service in the past 12 months?
☐   Never               ☐   1 to 5 times      ☐   6 to 10 times

More than 10 times, please state number: [                    ]

It may seem like a quibble to point out that, given the multiple-choice answers offered, this question should have read ‘How many times …?’, but a well-designed question will not interrupt a respondent’s mental process by inserting a task requiring mental agility, as this one does. Incongruously, while the respondent is not asked to state a precise number between 1 and 5 or between 6 and 10, for more than 10 – and correspondingly more difficult to recollect – a precise number is requested. Moreover, it is difficult to see how a scatter of responses above 10 could be taken into account in any analysis.

4. What was your reason for contacting NHS 111?       Yes     No
I have used this service before                                      ☐         ☐
I was told to call NHS 111 by another service                ☐         ☐
I wanted to know where the service was                      ☐         ☐
I wanted information on dental services                      ☐         ☐
I wanted advice on contacting a pharmacy                  ☐         ☐
I wanted advice on replacing a repeat prescription     ☐         ☐
I wanted advice on an illness/injury                             ☐         ☐
I could not get a GP appointment                                 ☐         ☐
I wanted to speak to an out of hours’ clinician            ☐         ☐
I wanted to know the opening time/location of a        ☐         ☐
service
Other, please specify:
[                                                                                             ]

The concept of ‘reason’ is a problematic one to use here. We may have a reason that prompts us to call for advice or treatment in the first place and a further reason for choosing one service provider rather than another. Strangely, the list of ‘reasons’ does not include physical prompts such as: ‘I had this sudden pain in my stomach.’ No distinction is made between categories such as needs for treatment, needs for advice, and needs for information. Moreover the question as phrased makes sense only in relation to a single contact: it doesn’t allow for the respondent having had different ‘reasons’ on different occasions.

5. How do you feel the NHS 111 service dealt with your call? On a scale of one to ten, with one being very poor and ten as excellent, please could you state how you feel the NHS 111 service dealt with your call? Please circle your answer.

1         2         3         4         5         6         7         8         9         10
_____________________________________
Very poor                                                                        Excellent

What is meant by ‘dealt with your call’? Conceivably it could have taken a very long time to speak to someone, or the person who first answered asked a lot of irrelevant questions, reading from a script, but then a clinician took over and was extremely helpful. Where on the scale should that response be marked?

6. Who do you think people should call for healthcare advice when their GP surgery is open?                Yes                           No
NHS 111                                                  ☐                              ☐
Registered GP practice                         ☐                              ☐

Very little can be done with answers to this question because it fails to go on to ask respondents why they have given that answer. And it may be that on some occasions they feel they need face-to-face contact with a clinician whereas on others they don’t. 

7. In the future would you like to be able to make an appointment with your GP by calling NHS 111 rather than your GP practice?
☐   Yes                     ☐   No

Again, this question calls for a follow-up question: ‘Why?’

8. Do you feel you/your friend/family member was advised appropriately by NHS 111?
☐   Yes, all the time       ☐   Yes, most of the time
☐   Yes, sometimes        ☐   I have not used the NHS 111 service
☐   No, please state why:   [                                                         ]

The references to ‘the time’ in the optional answers to this question are at odds with the fact that the service is there to be consulted on discrete occasions, not in an on-going way, over time.

9. If you have used the NHS 111 service, do you think it could help with all your health-related requirements?
☐   Yes
☐   No, please state why:        [                                                     ]

This question is confusing in a number of ways. It does not specify what is meant by ‘health-related requirements’: these could include not only requirements for advice or treatment but also requirements for commonplace pharmaceutical products. Nor does it allow the respondent to say that the service could help with some (rather than all) such requirements. And crucially, it does not specify a point or period in time to which it applies. A good question would direct the respondent’s attention to their past or present experiences, not leave it open as to whether they are invited to speculate about the future.

10. Have you used the Cornwall GP out of hours’ service?
☐   I use the service      ☐   My friend/family uses the service
☐   I do not have direct experience of the service

Given that the following questions 11-13 make no reference to the experiences of friend or family, it is hard to see the purpose of including the second of these choices in the list of possible answers to this question.

11.     How often have you used the Cornwall GP out of hours’ service in the last 12 months?
☐   Never        ☐   1 to 5 times        ☐   6 to 10 times
☐   More than 10 times, please state number:         [                  ]

As with Question 3: It may seem like a quibble to point out that, given the multiple-choice answers offered, this question should have read ‘How many times …?’, but a well-designed question will not interrupt a respondent’s mental process by inserting a task requiring mental agility, as this one does. Incongruously, while the respondent is not asked to state a precise number between 1 and 5 or between 6 and 10, for more than 10 – and correspondingly more difficult to recollect – a precise number is requested. Moreover, it is difficult to see how a scatter of responses above 10 could be taken into account in any analysis.

12. What was the reason for using the Cornwall GP out of hours’ service?
I have used this service before                                 Yes ☐   No ☐
I was unable to get a GP appointment                           ☐         ☐
I wanted to request a repeat prescription                      ☐         ☐
I wanted to speak to a clinician for advice                     ☐         ☐
I needed to be seen by a clinician at one of their           ☐         ☐
treatment centres
Did you have transport to get to a treatment centre     ☐          ☐
I needed to be seen by a clinician at home                     ☐          ☐
Other, please specify:
[                                                                                                      ]

As with Question 4: The concept of ‘reason’ is a problematic one to use here. We may have a reason that prompts us to call for advice or treatment in the first place and a further reason for choosing one service provider rather than another. Strangely, the list of ‘reasons’ does not include physical prompts such as: ‘I had this sudden pain in my stomach.’ No distinction is made between categories such as needs for treatment, needs for advice, and needs for information. Moreover the question as phrased makes sense only in relation to a single contact: it doesn’t allow for the respondent having had different ‘reasons’ on different occasions.

13. What was your experience of the Cornwall GP out of hours’ service? On a scale of one to ten, with one being very poor and ten as excellent, please could you state how you feel the GP out of hours’ service dealt with your call? Please circle your answer.
1         2         3         4         5         6         7         8         9          10
_____________________________________
Very poor                                                                        Excellent

This question illustrates the difficulty that is created when two different questions are packaged into one. ‘What was your experience?’ invites a narrative response, a factual description of what happened when the respondent contacted the service. No space is provided on the questionnaire form for such a description. Instead, provision is made only for grading how the respondent felt, and on one occasion only.

Moreover, as with Question 5: It is not clear what is meant by ‘dealt with your call’. Conceivably it could have taken a very long time to speak to someone, or the person who first answered asked a lot of irrelevant questions, reading from a script, but then a clinician took over and was extremely helpful. Where on the scale should that response be marked?

14. Do you think the Cornwall GP out of hours’ service treatment centres should be based in the same building with other health services, such as minor injury units and urgent care centres?
☐   Yes                ☐   No

This is the first reference in the questionnaire (including the preamble) to ‘GP out of hours service treatment centres’. There is no description of what they are and where they are to be found. Consequently it is difficult to see how a respondent can give an informed answer. And there is no provision for answering ‘Don’t know’ or ‘No preference’ to this question.

15. Do you feel you/your friend/family member was advised appropriately by the Cornwall GP out of hours’ service?
☐   Yes, all the time         ☐   Yes, most of the time
☐   Yes, sometimes          ☐   I have not used the NHS 111 service
☐   No, please state why:      [                                                       ]

As with Question 8: The references to ‘the time’ in the optional answers to this question are at odds with the fact that the service is there to be consulted on discrete occasions, not in an on-going way, over time.

And note that the box here that says ‘I have not used the NHS 111 service’ clearly has no relevance to a question about the GP out of hours service: it appears to have been erroneously copied and pasted from the draft of Question 8. This is a good illustration of the need to have a questionnaire carefully proofread before it is published. 

16. Do you think the Cornwall GP out of hours’ service meets all your health-related needs?
☐   Yes
☐   No, please state why:    [                                                       ]

As with Question 9: This question is confusing in a number of ways. It does not specify what is meant by ‘health-related needs’: these could include not only needs for advice or treatment but also needs for commonplace pharmaceutical products. Nor does it allow the respondent to say that the service could help with some (rather than all) such requirements. And crucially, it does not specify a point or period in time to which it applies. A good question would direct the respondent’s attention to their past or present experiences, not leave it open as to whether they are invited to speculate about the future.

Note that this question refers to ‘needs’, whereas the similar Question 9 refers to ‘requirements’. A good questionnaire is consistent in its terminology, and so doesn’t distract the respondent’s mind by posing the puzzle of whether different terms mean the same thing.

17. Do you think that either the NHS 111 or Cornwall GP out of hours’ service should be changed?
☐   Yes                      ☐   No
If yes,  what changes would you like to see?

This question, as worded, appears to rule out the possibility of the response that both services should be changed. It also requires the respondent to imagine possible changes, as none are put forward, even as prompts to thinking. If the question had been worded along the lines of ‘Can you suggest ways in which either or both services could be improved?’ it would be easier to respond to.

18. NHS Kernow is considering one of four options to provide NHS 111 and GP out of hours’ services in the future. Please tell us which one would
meet your needs:                                                           Yes        No
A joined-up (one number) NHS 111 and GP out of       ☐           ☐
hours’ service that is available 24-hours a day,
seven days a week)
An integrated NHS 111, GP out of hours and urgent     ☐           ☐
care centre model
An integrated 111, GP out of hours’ service and            ☐           ☐
urgent care centre model that is also integrated
with in-hours primary care (GPs)
An integrated 111 and GP out of hours service with      ☐            ☐
reduced in-hours 111 provision

I would not recommend these but my choice is:
[                                                                       ]

Note that the reference in the first of these options to ‘A joined-up (one number) NHS 111 and GP out of hours service’ is the first reference in the whole questionnaire to a telephone number. The other three options all refer to an ‘integrated’ service without spelling out what is meant by ‘integration’ in this context.

West Cornwall HealthWatch asked NHS Kernow for further information about the four options. The following reply was received: 

‘The options are deliberately only specified as high level concepts. One of the reasons for seeking feedback is to determine if any of the options should be worked up in more detail at which point we would undertake further public engagement.

‘Firstly none of the options describe the current model. The reason being that we currently have separate 111 and Out of Hours services provided by two separate organisations, namely SWASFT (the NHS 111 element) and Cornwall Health Out of Hours (The GP Out of Hours element).

‘Option 1 “An Integrated 111 and Out of Hours Primary Care Service” is closest to what we have now but assumes integration under a lead provider to reduce duplication of management overheads and processes.

‘Option 2:  An Integrated 111, Primary Care Out of Hours & In and Out of Hours Urgent Care Centre Service This option would see clinicians who work in the 111 & Out of Hours Service co-located with Urgent Care Centres in the out of hours service.  Option 1 does not assume any co-location.

‘Option 3:  An Integrated 111, Primary Care (In and Out of Hours) Urgent Care Centre  This option would build on the above, but also see patients being able to book appointments with their GP practice by calling 111.

‘Option 4:  An Integrated Out of Hours, 111 and Primary Care Service (No In-Hours 111 Provision)  This option would replicate Option 1 apart from a key difference that on Monday to Friday in the day-time, the 111 number would go to a pre-recorded message advising patients to call their own GP and/or include options to have the call passed directly to services such as Dental Helpline.’

While this reply clearly helps us to comprehend the implications of each of the options – importantly, it spells out that ‘integration’ can take the form of a single organization, a single location, and/or a single telephone number – the fact that ‘The options are deliberately only specified as high level concepts’ in a questionnaire ostensibly directed to ‘involving people in Cornwall and the Isles of Scilly is, we may justifiably say, remarkable.

19. Please feel free to add in any further comments in the box below:
[                                                                                           ]

SUMMARY AND CONCLUSIONS

As the content and language of the preamble show, this questionnaire was written with issues for management rather than for patients at the forefront of the authors’ minds. The authors were evidently confused about its purpose: the objective of ‘We … would like your views on our proposals’ is not consistent with the options being ‘deliberately only specified as high level concepts’.

Although, creditably, an attempt has been made to allow respondents to reply on the basis of their own experience, many of the questions are complex. Some use terms that need to be ‘unpacked’, broken down, e.g. ‘reasons’, ‘dealt with your call’, ‘health-related requirements’. Many are expressed in language that requires respondents to perform mental gymnastics, e.g. asking ‘how often?’ when ‘how many times?’ is meant.

Evidently the questionnaire was not piloted, or even proofread, as that would have revealed these problems.

This questionnaire is not fit for purpose. It is not a professional piece of work.

 
 

 

 

 

 

 

 

 

 

 

 

An assessment of the STP: Health and Social Care at risk in Cornwall

This post provides access to a recent publication by West Cornwall HealthWatch, a campaigning ‘watchdog’ group which aims to safeguard and improve services provided in West Cornwall by the National Health Service. Its own website is here.

5 January 2017

Health and Social Care at risk in Cornwall is an assessment of the Sustainability and Transformation Plan for Cornwall and the Isles of Scilly, published in late 2016 by the ‘Transformation Board’, a body which comprises all the leaders from the major public sector health and care organisations, including NHS Kernow (Kernow Clinical Commissioning Group), Cornwall Partnership NHS Foundation Trust and Cornwall Council. The assessment can be downloaded here in pdf format.

The official publications, including the draft Outline Business Case and an ‘engagement document’, Taking Control, Shaping Our Future, can currently be found here.

How not to run a Health and Social Care Survey, as demonstrated by Cornwall Council and NHS Kernow

This post, which is a revised version of one originally posted on April 17th, 2016, can be downloaded as a pdf file here.

Introduction
In January 2016 Cornwall Council and NHS Kernow (KCCG), partners in the ‘devolution deal’ for Cornwall, published a ‘Health and Social Care integration questionnaire’ under the headline ‘Have your say on health, care and wellbeing’. Both the design of the questionnaire and the administration of the survey left a great deal to be desired. This report presents a critique of both. It also draws on findings from the public events that were held in conjunction with the survey, and offers some lessons for future exercises of this kind.

The questionnaire
A detailed analysis of the Health and Social Care integration survey can be found below. In a nutshell, the findings include the following:

  • The survey was not based on a systematic sample of any kind. (It was, however, addressed to all age groups, including children under 11.)
  • All the questions were addressed to people receiving care: there were no questions that you could answer as a parent or carer.
  • There were three versions of the questionnaire (on-line, paper and easy-read): they were in material respects different from one another.
  • The questionnaires were not tried out among the general public, and a number of the most important questions were difficult to make sense of.
  • When the questionnaires were published no arrangements had yet been made for a proper analysis of the responses.

And while Council officers will doubtless express their satisfaction with the 2000 or so responses received to the survey, this will represent only 1 in 250 of Cornwall residents.

Who takes responsibility?

This incompetently designed and administered survey appears to have been largely the work of Cornwall Council officers, with NHS Kernow people playing a peripheral role, as witness the answer to a question from the public at the March 2016 meeting of NHS Kernow’s Governing Body:

NHS Kernow colleagues leading on engagement and communications commented on early drafts of the survey. It was then shaped by input from the Single Cornwall Plan steering group, this includes the CCG’s Director of Strategy as the representative for NHS Kernow. The final sign-off came from the Joint Strategic Executive Committee of which the Managing Director and Chair of the CCG are members.

It is not a straightforward matter to discover who among Cornwall Council’s officers was responsible for producing the survey. At the public events (daytime drop-in sessions and evening question-and-answer meetings) members of the Council’s communications team and the Communities and Organisational Development Directorate were seen. It is not apparent from the Council’s website how the latter might have been involved, but the domain of the communications team, according to its web page, includes not only public relations, brand management and leaflet design, but also ‘consultation and engagement’ with local people, to allow them to ‘give their views’ and ‘influence decision making’.

‘Engagement’ is a new name for public participation, which has been around since the 1970s. What seems to have happened here is that the leadership of the communications team successfully staked a claim over ‘engagement territory’, presumably on the basis that they possessed the requisite professional expertise. Evidently this claim went unchallenged: the survey’s many defects did not prevent it from being signed off by senior people in Cornwall Council and NHS Kernow. But practitioners of consultation and engagement need to have some grasp of and training in social research, especially social survey methods, where matters as personal and complex as health and social care are concerned: unfortunately the skills associated with public relations and brand management are very different.

Three different questionnaire formats
The questionnaire was available in three different formats: on-line (with completed questionnaires forwarded to a website in Sweden!), paper (as a pdf to be printed out and filled in by hand if downloaded), and ‘easy-read’. There were significant differences in the questions that they asked, so there were actually three different questionnaires, which will inevitably hinder analysis of the responses.

The survey was manifestly defective in other respects too. It was addressed to people who are receiving services (or who think they might be in future), but not to people looking after those who cannot express their needs for themselves. So the parents and guardians of young children, and carers – looking after people with disabilities, or who have dementia, or who are housebound and lack internet access – were effectively denied the opportunity to ‘have their say’.

And the language of the questionnaires presented problems. Members of a local Patient Participation Group in West Cornwall spent time in the waiting room of their GP practice encouraging patients to complete the questionnaire: they discovered that many patients found the questions difficult to understand and respond to. (See Appendix.) It is apparent that the questionnaire had not been tried out among members of the general public (although we are told that elsewhere in Cornwall members of a PPG, who are of course people with an active interest in health matters, had seen it and commented on it).

For the results of a survey to be of any use, the questionnaire must be designed with an eye to how the responses will be analysed. This one clearly has not been so designed. Several questions (such as ‘What are the three most important things to you when you experience health and social care services and support in Cornwall?’; ‘What would you like to achieve in terms of your own health and wellbeing?’; and the double (or treble) question ‘Do you have any suggestions that would help to improve your overall wellbeing and better meet your health and care needs at less cost?’) are open to being interpreted by different respondents in different ways, so it will be a formidable task to categorize and analyse the responses, especially for someone who was not involved in designing the survey. As of 3 April 2016 (a week after the closing date for responding to the survey) no arrangements had been made for a person with relevant experience to do this.

Comparison of this survey with the Council’s Residents’ Survey carried out in 2014 by the market research company Marketing Means is instructive: it demonstrates conclusively that this one has been formulated and the distribution of questionnaires organized by people with minimal understanding and experience of social research and survey methods. As a consequence there is no way that the findings can be sensibly analysed to guide priorities for the future of health and social care in Cornwall. At best this survey can serve no practical purpose other than as a ‘fishing expedition’ for ideas: a ‘list of mentions’ based on an entirely unsystematic sample is the most we can expect to emerge from it.

The questions that were asked
Q1. Your closest large town
The online and paper versions ask ‘Where is your closest large town?’ We can infer that they are not asking for latitude and longitude, since the paper version adds ‘For example: Truro, St Austell, Camborne, Bude or Penzance’ and the online version offers a drop-down menu with a list of 20 towns on it. Notably, Camborne/Pool/Redruth appears in this list as a single entity, and St Just does not appear at all, although its population is larger than those of Fowey, Lostwithiel and Padstow, which are on the list. So respondents have to scratch their heads and make a judgment as to the largeness of nearby settlements: they can’t simply give the first part of their home address’s postcode, which would of course be a perfectly straightforward thing to do.

The easy-read version asks ‘Which is your closest large town?’ and again gives the examples of ‘Truro, St Austell, Camborne, Bude or Penzance’. Although there is no ambiguity in the question (asking ‘which?’ instead of ‘where?’), this again calls for a judgment as to what counts as ‘large’.

Q2. Your age
All three versions ask ‘What is your age?’, and offer seven age ranges: Under 11 / 11-18 / 19-35 / 36‑50 / 51-65 / 66-80 / Over 80. Evidently the designers of the questionnaire wanted to cover the entire possible age range, but we may reasonably ask them: ‘What response did you expect to get from children under 11?’ While this is clearly a laughing matter, it raises a serious point which is not a laughing matter at all: Who speaks for the children? The questionnaire is addressed solely to individuals who receive services or might do so in the future: it entirely ignores people who care for them. Parents and other carers are treated as though they don’t exist. They will have experiences in looking after children, people with disabilities, people with chronic illnesses, dementia, etc., but no questions whatever are directed to them.

Q3. Ethnicity and ethnic origin

On this topic we find a great muddle. The paper and easy-read questionnaires ask: ‘What is your ethnicity?’ The paper version offers no alternatives or examples from which to choose. The easy-read version does offer some examples: ‘White British, White Cornish, Black British, White Asians, etc.’ (no ‘mixed’ category is offered as an example). But the very concept of ‘ethnicity’ is a challenging one (the author of the easy-read version seems to have been defeated by it!), and indeed the term is not in use in questions asked by the Office of National Statistics for census purposes. Only the online questionnaire asks the question as it should be asked – ‘How do you describe your ethnic origin?’ – and it offers a range on a drop-down menu: ‘White (for example, British, Scottish); Mixed (for example, White and Asian); Asian or Asian British; Black or Black British; Cornish; Other.’ Even here, however, the category ‘Cornish’ seems to have been added as an afterthought: it is scarcely an alternative to ‘White’.

We may wonder why there is an ethnicity/ethnic origin question at all. What is its relevance? (Did the compilers of the survey feel they were expected to include such a question?)

Q4. Long standing health condition

The online and paper versions of the questionnaire ask: ‘Do you have a long standing health condition? i.e. a physical or mental health condition or illness that is lasting, or expected to last, for 12 months or more.’ Both versions ask for a ‘Yes’, ‘No’ or Don’t know’. The easy-read version uses slightly simpler language – ‘Do you have a long term health condition? … that is lasting, or could last for 12 months or more.’ – and one of the choices is ‘I don’t know’. Again, in all three versions this question is directed only towards people who receive services or, one presumes, who might do so in the future: it is not a question for carers.

Q5. Written care plan

The online and paper versions of the questionnaire both ask: ‘Do you have a written care plan? i.e. an agreement between you and your mental health professional or social services to help you manage your day to day health.’ And both offer three alternative answers: Yes / No / Don’t know. The easy-read version shows that an attempt has been made to use plainer language: ‘Do you have a written care plan? For example: Something between you and your mental health professional or social services to help you cope with your day to day health.’

Q6. Three most important things
Again, the online and paper versions of the questionnaire ask the same question: ‘What are the three most important things to you when you experience health and social care services and support in Cornwall?’ Contrast this with the easy-read version: ‘What are the three most important things that you need when you have health and social care services and support in Cornwall?’ In asking about ‘needs’ the easy-read version is taking a less abstract, more down-to-earth, approach.

Significantly, no suggestions are made for what these ‘important things’ might be. Given the emphasis in the publicity on ‘hear[ing] about your priorities’, one would expect the questionnaire to have been designed to elicit these and place them in order, but this is clearly not the case. Moreover, if respondents had been offered some alternatives, such as suggestions emerging from a trial of the questionnaire, they would have gained some sense of ‘what sort of things’ were wanted. A good survey goes at least some way towards creating a dialogue between surveyors and respondents: questions like these do not.

Q7. Health and wellbeing
Under this heading there are some striking differences in language between the online and paper versions and the easy-read version, as we see in the table below. Contrast ‘What would you like to achieve in terms of your own health and wellbeing?’ with ‘What would you like to do that could make your health and wellbeing better?’ The very language ‘to achieve in terms of’ is abstract, ‘in-group speak’, doubtless commonplace in county halls and commissioning bodies but not out on the street and in people’s homes. And the very term ‘wellbeing’ is used nowadays to describe a wide variety of states: notably economic, social and psychological. How were respondents to know which was meant?

And it is a basic error in questionnaire design to shoehorn two (or more) separate questions together, as in asking for ideas to improve wellbeing but at less cost. By all means ask for ideas about improving wellbeing, and ask for ideas for saving money, but these are separate issues and conflating them will confuse rather than assist respondents.

What was the real purpose of this survey?
Surveys are designed and run for different purposes. It is extraordinarily difficult to pinpoint the purpose of this one.

The publicity coming from Cornwall Council stressed that Cornwall Council and NHS Kernow want to ‘know your health and social care priorities’, but the word ‘priorities’ does not appear anywhere in any of the questionnaires. And as we have seen, no attempt was made to get respondents to place the ‘three most important things’ in order of their importance.

Likewise Cornwall Council’s newsroom news release on February 29th said ‘Public views [are] sought on 5 year health and social care plan for Cornwall …’. but a draft plan had not been published and did not accompany the questionnaires.

Again, although the questionnaire is entitled (on the paper version) ‘Health and Social Care integration questionnaire’, it did not actually contain any question about people’s experiences of health and social care services operating alongside each other, and how well-integrated those services were found to be.

One possible purpose might have been to analyse patterns of need, for example to examine whether there is a correlation between need and particular age/sex groups or geographical location, but the questionnaires included no question about the sex of the respondents, and the question about geographical location (the ‘closest large town’!) was so imprecise as to be useless. Importantly, for such a survey to have value it would have to be comprehensive. If it were felt to be prohibitively expensive to send a paper copy to every Cornwall resident (aged from 0 to 80+), some form of systematic sampling should have been employed, as was done with the Council’s 2014 Residents’ Survey. Or target groups could have been identified, perhaps from the lists of patients held by general practices or lists of clients held by social services. None of these was done.

A lack of forethought must also be responsible for the survey’s failure to incorporate questions for parents/carers. This is simply inexcusable. It means that the needs of those who are cared for but cannot express their needs – such as young children, people with certain disabilities, those who are housebound and don’t have access to the internet, those who have dementia – are likely to be overlooked.

Lacking a clear purpose, the only function we can find that this survey performs is that of trawling for ideas: a ‘fishing expedition’. This is perhaps the kindest interpretation that can be put upon this exercise. But fishing expeditions can provide no basis for making policy in the complex fields of health and social care.

A questionnaire incompetently administered

Whatever the purpose of a survey, it needs to be set up and run competently. As the above detailed critique shows, this one has not been.

(1) There should have been no need for a separate ‘easy read’ version of the questionnaire, using plainer language: it was necessitated only by the existence of ‘difficult to read’ versions in the first place. And the person tasked with making it easy to read clearly threw in the towel when he or she came to the question ‘What is your ethnicity?’

(2) A closing date for receipt of responses was published, March 25th. The closing date was then brought forward with no announcement to March 14th, which actually fell before the end of the series of public meetings. At the beginning of March the original closing date was restored, again with no public announcement. But at the Penzance drop-in session on March 8th paper copies of the easy-read version were still showing March 14th as the closing date.

(3) There can be no excuse for not ‘piloting’ a questionnaire among the general public before disseminating it, rather than simply showing it to a small group of health activists.

(4) It is unfortunate that a team from the University of Exeter who have been asked to take part in analysing and interpreting the results were not involved in designing the questionnaire: questionnaires should always be composed with ‘What will the answers tell us?’ in mind.

(5) Finally, it is not clear what advantage has been gained by employing a firm whose headquarters are in Sweden. Who had that idea? It may be that had a firm closer to home been employed some of the errors listed here could have been avoided.

Learning from the public events
Following a series of drop-in displays and question-and-answer sessions around Cornwall, a report presented to Cornwall Council’s Health and Adult Social Care Scrutiny Committee on 5 April 2016 listed a number of ‘topics … building on the emerging themes from the survey’:

  • What’s good about health and care in Cornwall now?
  • What’s not good about health and care in Cornwall now?
  • How can we join up services to better meet your needs?
  • How can we best use the resources and local assets we have got?
  • How can we help you start well, live well and age well – away from hospitalised care?
  • How can we help you access the right care in the right place at the right time? i.e. GPs/doctor’s surgeries, minor injuries and urgent care, operations/surgery, social care, mental health, community hospitals

These questions are manifestly much easier to understand than many of those in the actual questionnaires. Talking to people is often a very good way of clarifying one’s own thoughts and expressing them in plain language.

Conclusions: Lessons for future surveys
(1) It is important to be clear about the purpose of the survey. What do you want from it? As we have seen here, in the present case the aims were variously described as ‘knowing your priorities’, seeking views on a ‘health and social care plan’, and finding out about ‘integration’ of services, but there were no questions about priorities, there was no draft plan to comment on, and there were no questions about people’s experiences of service integration or lack of it.

(2) It is crucial to decide how to sample your population. Do you want a sample covering the whole population that you can generalize from with confidence? Are there distinct groups you want to cover, e.g. carers, parents, women/men, people with experience of hospitalization, people with long-term health conditions, people living in care homes, people who are housebound?

(3) The people who are going to analyse the results should be involved in designing the questionnaire. Among other things, they will always be asking: What will we do with the answers? What will they tell us?

(4) Draft questionnaire(s) should be tried out – piloted – on the general public, not on people who are already well-informed about the subject.

In conclusion, a point needs to be made about the language used by officers and other professionals. Their current preference for ‘engagement’ over ‘public participation’ and ‘public involvement’ not only writes the public out of the term, literally: it places the initiative with the body concerned. The Council and the Clinical Commissioning Group are implicitly seen as active, with officers doing the engaging: the public is passive, on the receiving end, being engaged with. The effect is to reinforce an official mindset that already finds it difficult to cope with initiatives and criticism that come from watchdog groups and others in the public realm. Can we get back to ‘public participation’, please?

I do not doubt the goodwill and sincerity of those who organized the present survey. But the official mindset has not served them well. And they have to realize that undertaking a social survey is not a job for the inexperienced, however well-meaning and enthusiastic they may be.

Peter Levin

© Peter Levin 2016. All rights reserved.


Appendix: Notes from a general practice waiting room in West Cornwall

The following notes have been supplied by members of a Patient Participation Group in West Cornwall, and are reproduced here without any editing or alteration.

HEALTH AND SOCIAL CARE INTEGRATION QUESTIONNAIRE

Several members of our Patient Participation Group took it in turns to spend time in the waiting room of our practice, encouraging patients to complete the Health and Social Care survey.

Whilst we enjoyed spending the time talking to and listening to what patients had to say to us, we found that the survey was neither clear nor user-friendly in language, and patients found it difficult to answer. In particular, we noted the following:

Q1 Where is your closest large town?
It would have helped to have a list to choose from, rather than just five examples. Were they meant to choose from those?

Q3 What is your ethnicity?

Patients were unsure what to write here – a list of options would have made this easier to answer

Q5 Do you have a written care plan?

Patients felt that the survey was not relevant to them if they did not fit in this category

Q6 What are the three most important things to you when you experience health and social care services and support in Cornwall?

As we were in a GP practice, patients instinctively commented mostly about their experiences of general practice, and couldn’t think much further than that

Q7a What would you like to achieve in terms of your own health and wellbeing?

Patients found that a strange question – they just wanted to be fit and well! They also used these questions to comment on their own experiences, though how that will be used in analyzing the survey remains to be seen

Q7b What are you already doing towards that?

Patients found that difficult to answer, and weren’t clear what they were supposed to say

Q7c What additional help might you need?

Patients found that difficult to answer too. They weren’t sure whether they were meant to say “more money”, or “more disability aids”, or something else they didn’t know

Q8 Do you have any suggestions that would help to improve your overall wellbeing and better meet your health and care needs at less cost?
Patients found this a baffling question as it asked three things in one sentence. One lady responded: “What the hell is that supposed to mean?”

Other comments

a) Through whose eyes?
Some people wanted to reflect on health and social care from a carer perspective, and there was little opportunity to do that unless they pretended to be the person they cared for. The same applied to parents of young children.

b) What is ‘well being’?
Generally, the term ‘well being’ is not one used by the average lay person; neither does s/he find it easy to identify their ‘health and care needs’, and certainly not ‘at less cost’, because they don’t know what everything costs anyway

c) A lost opportunity?

I think we as members of the PPG found this an interesting exercise but also perhaps a lost opportunity. A few people managed to express ideas for the way forward whether or not this answered a particular question. How the analysts will use this also remains to be seen. Had the survey been better written and presented, you might have gained really helpful insights into how the community feels about health and social care. As it is, we think you will find it very difficult to use this data effectively or gain meaningful findings from analyzing the data you receive. So it is rather disappointing.

FC, CG and MB 21.03.16

 

Involving the Public in NHS Commissioning: A Culture Clash in Cornwall

This post can be downloaded as a pdf here. The original KCCG document, which appears to be no longer available on the KCCG website, can be downloaded here

Summary
This post takes a close look at how Kernow Clinical Commissioning Group (KCCG) carries out its duty to involve the public in its activities. It focuses on a document entitled Procurement framework for managing Commissioning changes which, although not intended for public consumption, was tabled at a meeting of the Governing Body of KCCG in November 2015. Having been revealed to the public, the document was then announced as being the subject of a ‘piece of engagement’, with comments required within five days, which included a weekend. Following complaints, the deadline was subsequently put back by a month.

West Cornwall HealthWatch (WCHW), an independent voluntary watchdog body, and six other local groups submitted comments on the document. In a detailed and reasoned criticism, WCHW pointed out that it contained no clear statement of the purpose of the document or its intended audience, and that in many places it was difficult to discern what the language used actually meant. Of particular concern was a very confused description of the process of managing changes in commissioning, a description that completely omitted any reference to involving the public in the process, although it is a process that has major implications for patients and potential patients.

Of even greater concern is the reaction of KCCG to the comments submitted and what that reaction reveals about the organization and its culture. The comments were not circulated to all the members of KCCG’s Governing Body, and at a subsequent meeting the document was approved with the wording unchanged. We have learned since that WCHW’s comments were regarded as hostile and aggressive.

KCCG’s reaction is indicative of wide differences between the culture of the organization and that of the public world within which it is situated. Differences in attitudes, expectations and language are so great – and there is so little understanding of them – as to constitute major obstacles to public involvement in KCCG’s commissioning activities. It seems likely that this will be the case in localities across the country, and indeed at national level too.

This paper consists of seven parts:

1. Introduction
2. Detailed comments on the ‘Procurement framework’ document
3. Discussion and conclusions on the ‘Procurement framework’ document (including questions about the way in which KCCG is run)
4. The sequel: how KCCG reacted to criticism
5. KCCG’s constitution and legal duties
6. Is KCCG abiding by its constitution and legal duties?
7. The ‘culture clash’ between KCCG and the public: is there a way forward?

1. Introduction
On November 10th, 2015, a document entitled Procurement framework for managing Commissioning changes [1] was tabled at a meeting of the Governing Body of the Kernow Clinical Commissioning Group (KCCG) [2]. Two weeks later, the co-ordinator of West Cornwall HealthWatch (WCHW) received an email drawing her attention to a ‘piece of engagement … around … a draft procurement framework document’ that had been placed on the KCCG website and informing her that the engagement would close five days later (the five days included a weekend). Following complaints, the deadline was subsequently deferred to December 31st. It is the document on the website, ‘Draft 6’, that is dealt with here. The full version is appended to this paper: there are extracts from it in the text, below, shown in italics.

An earlier version of this paper, under the title ‘The Procurement Framework: a cautionary tale from Cornwall’ was submitted to KCCG on December 29th, 2015 and subsequently published on WCHW’s website. That earlier version comprised an Introduction and two further sections which are reproduced without modification as Parts 2 and 3 of this paper.

2. Detailed comments on the ‘Procurement framework’ document
Section 1, ‘Introduction’
This framework sets out NHS Kernow’s management of major changes to the commissioning of services. Applying a consist (sic) approach for the gathering of evidence, exploration of options and governance, ensuring compliance to all relevant laws and guidance is key to the successful commissioning of services. NHS Kernow is committed to providing high quality clinical services that meet the needs of local communities as set out in its Corporate Objectives and Annual Delivery Plan agreed by the Governing Body. NHS Kernow will look at all options and engage as appropriate to secure the best services for the local population.

What we see here, in the title and the very first sentence of the document, is confusion between structure and process. ‘Frameworks’ are commonly understood to denote structure, something relatively fixed over time, such as an organization or a contractual agreement or – more loosely – a set of arrangements; ‘management’ clearly refers to process, a series of steps, something that proceeds over time. A ‘framework [that] sets out … management’ is a nonsense. The opening words ‘This framework’ suggest that the document itself is a structure, and this is a nonsense too.

Since the term ‘procurement framework’ is clearly a problematic one, capable of being interpreted in more than one way, it would have been helpful to a reader if the document had opened with a definition of it. In mitigation, the treatment of the term and its component parts on various NHS websites is nothing short of chaotic. On one, for example, we find that the answer to a frequently asked question ‘What is a framework?’ begins ‘A framework agreement is …’. [3] On another, belonging to NHS England, we find CCGs advised to ‘establish a procurement framework for “lead providers” …’, although this merely refers the reader to a diagram listing the services that a ‘lead provider’ is required to offer. [4] (The KCCG document makes no reference anywhere to ‘lead providers’.)

The opening paragraph does not state the purpose of the document, nor does it say for  whom it is intended. If the document is intended to be – or to develop into – a manual or handbook, to guide the process, it should say so. But as it stands this paragraph reads as merely a piece of self-praise.

Section 3, ‘Legislation’
Where NHS Kernow intends to work collaboratively with an existing provider to effect  a major change, it will evidence how due process has been followed to ensure all risks and benefits have been appropriately evaluated adhering to all relevant national and regional guidelines.

When NHS Kernow intends to procure a new contract by testing the market for competition, it will ensure compliance with EU Procurement Directives as implemented by UK Law and national guidance from NHS England, NHS Improvement and Crown Commercial Services …

All this seems to amount to nothing more than saying KCCG will follow the rules and show that it has done so. As for the list of pieces of legislation etc. that follows (see the Appendix), the document fails to conform to the convention that pieces of legislation should be given their correct titles and, in the case of statutory instruments, their S.I. number. Presumably, by ‘2015 Public Procurement Regulations’ the authors mean ‘The Public Contracts Regulations 2015’, SI 2015 No. 102. Accuracy and precision are crucial if a document of this nature to be taken seriously.

Furthermore, it is customary in such documents to distinguish measures according to the amount of discretion – latitude – that they afford to decision takers. Thus Acts of Parliament will typically set out requirements that must be adhered to (although a requirement that an office-holder must be ‘satisfied’ about a proposed course of action does of course allow discretion to that person). Codes of Practice, notes of guidance, briefings, advice, recommendations arising out of research: these allow different amounts of discretion to decision takers. This document fails to note these distinctions or even list the measures in a systematic and recognisable order.

Section 4, ‘Financial controls’
NHS Kernow’s constitution sets out the financial limits for the management best value on any purchases carried out on the CCGs behalf. … NHS Kernow will ensure all commissioned services aim to deliver value for money ensuring best quality and price for the service supplied.

This extremely brief section says nothing whatever about how financial controls are to be exercised. Quoting the ‘motherhood and apple pie’ aspiration of delivering ‘value for money ensuring best quality and price’ is hardly likely to assist someone engaged in the commissioning process.

Section 5, ‘Governance’
We see from the diagram in this section of the document (but not from the text) that the oddly-named Project Steering Group (Task and Finish) – ‘Specific job roles from various departments will be a core group …’ (sic) – will report to the Procurement Committee (and through that Committee to the Governing Body). Three of the boxes in this diagram correspond to ‘chunks’ of organizational structure, and have links that presumably correspond to lines of instruction and reporting, but there is a further box, containing the words ‘Finance, Performance & Quality’, that does not connect to any of the other three. This is more than somewhat unusual for a box diagram that purports to represent the structure of an organization. It suggests that the authors do not have a grasp of the very concept of ‘organizational structure’.
ProcurementFrameworkForManagingMajorCommissioningChanges_000005

Section 6, ‘Procurement Policy’
Interestingly, there is nothing in this section about the process of forming policy: the section is devoted entirely to the principles – high aspirations: motherhood and apple pie, again – to be followed. There is one oddity, however: this section gratuitously includes the completely irrelevant information that the provisions of the Bribery Act 2010 came into force on July 1, 2011.

Section 7, Major commissioning change process
The process for managing a major commissioning change within NHS Kernow covers 4 key stages:

1. Pre- procurement phase (review and plan for change)
2. Collaboration or Competition
3. Mobilisation of the change
4. Contract and performance management

[In the pre-procurement phase,] each commissioning change will commence with a Project Initiation Document outlining the findings following a review of the current state and where a commissioning view is that change is required. The Procurement Committee will review and endorse further work-up to a full business case for allowing a major commissioning change or advise on what further information is required to re consider the project at a later date. Development of a full business case will utilise a range of skills of individuals within the organisation and the development of a Project Steering Group as a Task and Finish Group will be established to oversee its delivery.

Of these four stages, it is Stage 1 – and only Stage 1 – that actually comprises the decision-making part of the process. Disentangling the wording of this section – necessary because the elements are not presented in chronological order – we see that the decision-making process runs as follows:

(1) ‘a review of the current state’;

(2) the forming of a ‘commissioning view … that change is required’;

(3) the writing of a ‘Project Initiation Document’;

(4) the ‘Procurement Committee will review and endorse further work-up to a full business case’;

(5) a full business case is developed: this includes an outline business case (see the ‘Procurement framework’ document), but we are not told whether or not this precedes the Procurement Committee’s decision to endorse ‘further work-up’.

(6)  At some point during the development of the full business case – we are not told when – ‘the development of a Project Steering Group … will be established (sic) to oversee its delivery’.

Conspicuous in this description of the process is the complete absence of any reference whatever to involvement of patients or the public in this process. KCCG makes great play of its commitment to ‘engagement’, but when we look for some description of how this is to be incorporated in the decision-making process we find nothing at all. Worryingly, what we see here is effectively a prescription for a process of building up such a strong momentum within the KCCG organization that by the point at which the public gain entry to the process it is extremely difficult to change direction: the result, inevitably, will be confrontation and conflict.

If we take KCCG’s protestations of its support for engagement at face value, this document has to be seen as a classic case of what goes wrong when drafting is carried out by people who do not appreciate the wider context within which the organization is situated and who do not fully comprehend the process and how it works. They may be fluent, to some extent, in jargon, but their words require interpreting and disentangling, as the above discussion demonstrates.

‘Collaboration or Competition’ is presented here as a ‘key stage’ in a process, but of course it is an issue, not a stage. As set out in the document, the issue is one for the Procurement Committee to decide, and to decide on the basis of the full business case. This seems bizarre, to put it mildly: collaboration and competition must surely each require their own business case. It is also an issue on which patients, the public and health service staff all have experiences and views that deserve to be taken into account. This is another reason for opening the process up and engaging them at an early stage.

‘Mobilisation of the change’ and ‘Contract and performance’ are essentially to do with the implementation of decisions, and accordingly fall beyond the crucial steps in the decision-making process. They will need to be thought about in advance, of course, and one might have expected to see the testing of feasibility as a (recurring) stage in the process, but such a stage does not feature in the KCCG document.

3. Discussion and conclusions on the ‘Procurement framework’ document
From its content, it is very hard to discern the purpose and intended audience of the ‘Procurement framework’ document. It may be that these were not actually identified, and that KCCG was simply following an instruction to produce a ‘framework’. Or that the leading lights in KCCG saw an opportunity for self-advertisement. It is certainly strange that the document makes no reference to any of the literature on commissioning already published by NHS bodies, which suggests that the authors have started from scratch and in effect embarked on ‘reinventing the wheel’.

Moreover, the document contains very, very little in the way of practical guidance: instead we find high aspirations, couched in gobbledygook, such as this: ‘NHS Kernow will ensure all commissioned services aim to deliver value for money ensuring best quality and price for the service supplied.’

What is really worrying is the seeming inability of the authors of the document to think in an analytical way. They have not defined the term ‘procurement framework’. They have not grasped the distinction between structure and process, and are demonstrably unable to lay out the elements of the process and see how they fit together. Their jargon-laden writing style too is suggestive of an inability to think clearly. If this is an indication of the calibre of those who oversee and budget for the National Health Service in Cornwall and the Isles of Scilly, then patients, the public and health service staff have every reason to be very concerned indeed.

In appraising the ‘Procurement framework’ document one word comes repeatedly to mind: ‘amateurish’. Judged by both its content and its presentation, it simply does not reach a professional standard. In part this may be attributable to the make-up of the CCG’s Governing Body, the 15 members of which include seven General Practitioners, another Doctor member and a Nurse member. General practices are essentially small businesses, and it is hard to see how running one can be a qualification for  running an organization with an annual budget of more than £700 million. It follows that the Governing Body must be heavily dependent on KCCG’s paid staff for advice and guidance, and accordingly serious questions must be asked about the calibre of those staff. The recent news that, 8½ months into the current financial year, KCCG is heading for a year-end deficit of £14m, having previously forecast a surplus of £500,000, and is to have a so-called ‘turnaround director’ appointed, adds weight to this concern. [5]

What lessons can be learned from the situation in Cornwall? Kernow CCG is just one of 209 clinical commissioning groups in England, and clearly we have no grounds for generalizing from this one CCG to others. But this study does show what can happen when a CCG gets the bit between its teeth. KCCG seems to have gone off on an eccentric foray of its own, and it is only a matter of chance that this latest escapade has come to light. The current financial situation, although it is now receiving attention from NHS England, seems to have been detected very late in the day, and while the diversion of resources into producing a grandstanding but ultimately useless document may not have been massive, it does raise questions about the judgment of those in charge.

So we are left with some questions. Are other CCGs grappling with the problem of having to change contracts that they have commissioned: if so, what guidance have they received from NHS England or other bodies? Is there scope for CCGs to work together on the problems that they face? Are arrangements for overseeing the work of CCGs satisfactory, or is an inspection regime of some kind called for?

And as for Cornwall itself, an improvement to the governance of KCCG is clearly urgently needed. What can be done to provide it?

Part 4. The sequel: how KCCG reacted to criticism
The KCCG Governing Body met in public on Tuesday, January 13th, 2016. The papers for the meeting included a document headed ‘Procurement framework …’: this turned out to be precisely the same document as had been offered for ‘consultation’, apart from the addition to the ‘Governance Structure’ diagram of a line connecting the ‘Finance, Performance and Quality’ Box to the ‘Governing Body’ box. No other changes at all had been made. The Governing Body was asked ‘to approve the final version of the Procurement framework’. [7]

It was reported to the Governing Body that a total of seven organizations had submitted comments, West Cornwall HealthWatch having been one of them. Questions from the public elicited the information that these comments had not been circulated to all members of the Governing Body.

The officers also said that ‘Procurement is not a policy/strategy that CCGs have to consult on’ and that ‘The framework that was discussed in November 2015 is the same one that was previously published on the CCG website, for the period 1st April 2013 to 31st October 2014 and is only updated with changes in the CCG structure and or legislative change’.

The officers reported as follows:

There are two consistent themes in the comments/responses received from the consultation:

1. The documentation is not public facing and therefore contains a number of NHS acronyms and administrative language.

The framework document that went for public consultation was written for an internal NHS audience. It is not necessary for CCGs to consult on frameworks of this nature and initially it had not been NHS Kernow’s intention to hold a public consultation on this updated and amended document from 2013.

To address this issue NHS Kernow is in the process of preparing a subsequent document that will support the framework and addresses the specific procurement regulations and the NHS requirements on competition and choice raised within this consultation. It is anticipated this document will be ready for public consumption in February and the document will be launched at an invitation event from contributors to the consultation held in December.

2. Public and Service User engagement is paramount throughout any service change/procurement.

NHS Kernow has received consistent feedback regarding engagement and as a result are in the process of creating a specific engagement group that deal specifically with service changes and procurement requirements. This group is an addition to the current engagement and service user group already established.

Having been asked ‘to approve the final version of the Procurement framework’, the Governing Body duly did so. [6]

5. KCCG’s constitution and legal duties
Involving the public: what KCCG’s constitution says
Kernow CCG is required to act in accordance with its constitution,7 in which there are numerous sections which refer to involving local people.

  • S. 5.4.1  ‘The governance arrangements … detail the way in which Kernow CCG will demonstrate principles of probity, accountability and transparency to allow the organisation to serve patients and the local population effectively.
  • S. 5.5.1  ‘The Group will demonstrate its accountability to … local people in a number of ways, including by:

 ◊   appointing independent lay members … to its Governing Body;

◊   holding meetings of its Governing Body in public (except where the Group considers that it would not be in the public interest in relation to all or part of a meeting);

◊   meeting annually in public to publish and present its annual report.

  • S. 6.2.1  ‘[T]he Group will make arrangements to secure public involvement in the planning, development and consideration of proposals for changes and decisions affecting the operation of commissioning arrangements by …

◊   working in partnership with patients and the local community to secure the best care for them;

◊   involving patients and the public, encouraging and taking account of feedback in the planning of commissioning services and in developing, considering and making decisions on any proposals for changes in commissioning arrangements that would have an impact on service delivery or the range of services available to demonstrate transparency, inclusiveness, fairness and accountability in decision making;

◊   making decisions in an open and transparent way so that people can understand how services are planned and delivered;

◊   making it possible for patients, the public and other stakeholders to be involved in decisions about services for the local population;

◊   consulting with people who are affected by service change …’

  • S. 7.5.4  The Governing Body may delegate authority to the People’s Commissioning Board and make decisions on any aspect of its work deemed appropriate by the Governing Body. The Lay Member for patient and public involvement is the Chair of the People’s Commissioning Board and will bring reports, recommendations and requests from it to the Governing Body. The Governing Body is required to ratify any recommendations or decisions made by the People’s Commissioning Board.
  • S. 8.7.1  The Chair of the Governing Body is responsible for …

◊   overseeing governance and particularly ensuring that the Governing Body and the wider Group behave with the utmost transparency and responsiveness at all times;

◊   ensuring that public and patients’ views are heard and their expectations understood and, as far as possible, met;

◊   ensuring that the organisation is able to account to its local patients, stakeholders and the NHS Commissioning Board …

  • S. 8.11.1&2  There are two lay members appointed to the Governing Body, one to lead on audit, remuneration and conflicts matters, and one to lead on patient and public participation matters. The lay members have a non-executive role within NHS Kernow CCG.

Involving the public: the law
Under Section 14Z2 of the Health and Social Care Act 2012 (Public involvement and consultation by clinical commissioning groups), KCCG ‘must make arrangements to secure that individuals to whom the services are being or may be provided are involved … in the planning of the commissioning arrangements by the group, [and] in the development and consideration of proposals by the group for changes in the commissioning arrangements where the implementation of the proposals would have an impact on the manner in which the services are delivered to the individuals or the range of health services available to them’. [8] In other words, KCCG has a legal duty to involve the public in planning its ‘procurement framework’ and in the actual process of developing and considering changes in commissioning.

The law has also something to say about the role of the Lay Member on the Governing Body. There are statutory Regulations, published by the NHS Commissioning Board, which cover this. [9] The Lay Member has ‘a lead role in championing patient involvement’. ‘Their focus will be strategic and impartial, providing an independent view of the work of the CCG that is external to the day-to-day running of the organisation. … This person will help to ensure that, in all aspects of the CCG’s business, the public voice of the local population is heard … In particular, they will ensure that the CCG … responds in an effective and timely way to feedback and recommendations from patients, carers and the public.’

6. Is KCCG abiding by its constitution and legal duties?
‘Accountability’, ‘transparency’ and ‘involvement’ are fine words but how can they be translated into practice? All of them imply a relationship with the public, so to put them into practice necessitates asking ‘Accountable to whom?’, ‘Transparent to whom?’, ‘Involving whom?’

To answer these questions, within an organization that deals with the public, there have to be some people on the staff and Governing Body who are able to imagine how that organization and its behaviour look to people outside. They have to be able to put themselves in the shoes of members of the public. Sadly, despite KCCG’s constitution committing it to transparency, it evidently did not occur to anyone that choosing the Isles of Scilly as the venue for its 2015 annual meeting, to publish and present its annual report, as its constitution requires, was guaranteed to prevent the attendance of any member of the public from mainland Cornwall, a plane ride away.

Likewise, it seems not to have occurred to the authors of the ‘Procurement framework’ document that it should as a matter of course be made public, even though the Health and Social Care Act 2012 requires CCGs to involve the public in the planning of its commissioning arrangements and in the development and consideration of proposals for changes in commissioning arrangements (see above). This rather contradicts the view expressed in the report to the Governing Body at its January 2016 meeting in the case of the ‘Procurement frameworks’ document that ‘[it] is not necessary for CCGs to consult on frameworks of this nature’. And clearly, to make no provision for involving the public in the ‘Major commissioning change process’ is also cocking a snook at the law.

In this context, the position of the Lay Member for patient and public involvement is of particular interest. The person appointed worked for 31 years in the NHS. At the point when he retired he was Chief Executive of two primary care trusts. His long experience and seniority will have given him considerable insight into the complexities of managing health services and a sympathetic understanding of the viewpoints of NHS staff. Within KCCG he has been appointed Chair of the Procurement Committee, although under KCCG’s constitution the lay members have a non-executive role within NHS Kernow CCG (S. 8.11.1&2) and the statutory Regulations explicitly say (see above) that he must provide ‘an independent view of the work of the CCG that is external to the day-to-day running of the organisation’. Perhaps it was his managerial experience rather than a grassroots affinity with patients and public that secured his appointment to the KCCG Governing Body. (Interestingly, although the KCCG constitution specifies that ‘The Lay Member for patient and public involvement is the Chair of the People’s Commissioning Board’, there is no indication on the KCCG website that such a body actually exists.)

7. The ‘culture clash’ between KCCG and the public: is there a way forward?
From the point of view of an organization such as West Cornwall HealthWatch, a voluntary, independent health watchdog that monitors developments and campaign to safeguard and improve existing services provided in West Cornwall by the National Health Service, KCCG’s constitution says many of the right things, with its references to public involvement, partnership and so on, and especially transparency and accountability. But the experience with the ‘Procurement framework’ document is a case-study of the gulf between a statutory body and a local watchdog group. It is a gulf that arises from a clash between two very different cultures.

Aspects of KCCG’s culture are readily apparent from the saga of the ‘Procurement framework’ document. Those within the organization are evidently very conscious of the boundary between the organization and those outside it. They have their own  specialized language – ‘administrative language’ – which members of the public do not comprehend. They work in a disciplined hierarchical world, where it is very clear who is above you and who is below, and in which the higher you are the more deference you expect and are accorded. Decisions are taken in committees, where there is pressure – which may be overt or subtle – to come to agreement, to a consensus: the phenomenon of ‘groupthink’. Disagreement is embarrassing, especially if it becomes public, and is consequently to be avoided if at all possible. As we have seen, the Governing Body approved the ‘Procurement framework’ document despite members not having read the critical comments from the public, a step that is hard to explain in terms of rational and independent consideration.

The links that KCCG people have beyond the immediate organization are with health service  ‘professionals’, who are accustomed to possessing a domain in which they have a good deal of autonomy, scope for exercising their ‘professional judgment’. (And in the healthcare world there is a pecking order of professions, to which everyone is very sensitive.) Asserting professional judgment amounts, of course, to claiming that members of the public have little or no right or competence to contest the decisions arrived at.

The culture of a watchdog group such as West Cornwall HealthWatch is very, very different from that of KCCG. There is no ingrained deference towards those high up in the hierarchy: indeed, there is hardly any hierarchy. There is a genuine desire to know what is being planned, and a desire to have matters explained, but the face that KCCG presents is seen as a smooth, hard, expressionless one, with no cracks that would allow outsiders to gain some purchase on what is going on inside. At times there will – understandably – be frustration when it appears that answers to questions are designed to fob off the questioner, and suspicion when it appears that full and accurate information and clear and convincing explanation are not being freely given. At such times some sensitivity to being treated with condescension may be evinced. Some members of the WCHW committee have worked within the NHS, so may be particularly aware of ‘staff side’ views. Some members are opposed on principle to policies such as ‘contracting out’ and other forms of privatization. (WCHW is strictly a non-political party organization, however.) But all are members of the public.

So what happens when a clinical commissioning group meets a watchdog group? One piece of evidence is KCCG’s treatment of the views on the ‘Procurement framework’ document submitted by the seven responding groups. These views were not circulated to members of the Governing Body, and that body approved the document rather than asking for more information or referring it back for further work. We have also learned that the Lay Member for patient and public involvement (who also chairs the Procurement Committee) found WCHW’s paper aggressive and hostile, and he has queried whether WCHW thinks it would be effective in changing people’s minds.

What we see here is a classic case of non-meeting of mindsets. Generalizing for a moment, we have the KCCG mindset, which frames WCHW’s critique as aggressive and hostile: but we can envisage the possibility of a very different mindset, which would see the WCHW critique as forthright and challenging. In the first mindset the critique is an attack, and accordingly to be repelled; the second admits of the possibility that the challenge can be constructively engaged with. One wonders, however, whether people within KCCG are actually aware of having a mindset. Organizational cultures are liable to be so dominant and all-pervading that there is no awareness of mindsets other than the one that is prevailing, no capacity to imagine that there are other ways of looking at the world.

To WCHW committee members, the point at issue over the ‘Procurement framework’ document is different: less subtle, more straightforward. ‘You asked us what we think. This is what we think. These are the questions that we have. Oh, you don’t like them! Are we meant to apologize?’ Arguably, shorn though it may be of tact and diplomacy, theirs is a more honest and direct mode of discourse.

When it comes to resolving issues, there will be a similar divergence of expectations and approaches. In the present case, we wait to see what will happen next. KCCG is in the process of preparing a new document ‘that will support the framework and addresses the specific procurement regulations and the NHS requirements on competition and choice raised within this consultation’. This document is due to be published in February and ‘launched at an invitation event from contributors to the consultation held in December’ (sic). So we wait to see what is in this document, whether there will be an opportunity to read it before the ‘event’ and ask questions at the event itself, whether the ‘Procurement framework’ itself is modified, whether there is any limit on the number of people from the contributing organizations attending – and indeed whether the event is held in mainland Cornwall or on the Isles of Scilly!

The forum of public involvement is an unruly one. Away from statutory bodies and big corporations, it is not hierarchical. Issues are not resolved by top-level negotiations between a select few senior people, to whom others defer, in private round a board-room table. Anyone can ask a question, in public, and all questions come with an entitlement to be given an answer, also in public.

For a constructive debate to take place, certainly the members of watchdog groups need to appreciate the difficulties that managers and professionals face in allocating resources and in planning and developing services.

For their part, the managers and professionals need to understand that transparency and accountability require them to be open, speak in a language that ordinary people can understand, forswear the habit of expecting deference, accept that there are valid mindsets besides their own, allow for and answer honestly questions that they might find uncomfortable, be prepared to learn from outsiders, and indeed take with a good grace some occasional mockery at times when the gulf between the cultures opens up. We trust that this is not too much to hope for.

© Peter Levin 2016

29 December 2015 & 25 January 2016

Notes

1. The document is attached to the pdf version of this post as an Appendix. It can be downloaded here. It was formerly to be found on the web at:
http://policies.kernowccg.nhs.uk/DocumentsLibrary/KernowCCG/WebDocuments/Internet/Engagement/ProcurementFrameworkForManagingMajorCommissioningChanges.pdf  (Accessed 27/12/2015)

2. Kernow Clinical Commissioning Group, KCCG, which is the CCG for Cornwall and the Isles of Scilly, also styles itself NHS Kernow. (Kernow is the Cornish language name for Cornwall.)

3. NHS London Procurement Partnership  http://www.lpp.nhs.uk/about-lpp/frequently-asked-questions/ (Accessed 27/12/2015)

4. NHS England, Questionnaire on supporting clinical commissioning groups to buy commissioning support
https://www.england.nhs.uk/wp-content/uploads/2013/06/buy-commis-supp.pdf (Accessed 27/12/2015)

5. BBC News, ‘NHS Kernow ordered to deal with financial problems’
http://www.bbc.co.uk/news/uk-england-cornwall-35113666   (Accessed 16/12/2015)

6. http://policies.kernowccg.nhs.uk/DocumentsLibrary/KernowCCG/OurOrganisation/GoverningBodyMeetings/1516/201601/2015062ProcurementFrameworkAndProcurementCommitteeTermsOfReference.pdf (Accessed 16/01/2016)

7. There appears to be no link to KCCG’s constitution on its own website, but that document can be found here: http://www.rcht.nhs.uk/GET/d10316833 (Accessed 16/01/2016)

8. Health and Social Care Act 2012, S.14Z2 Public involvement and consultation by clinical commissioning groups http://www.legislation.gov.uk/ukpga/2012/7/part/1/crossheading/further-provision-about-clinical-commissioning-groups/enacted (Accessed 21/01/2016)

9. Clinical commissioning group governing body members: Roles outlines, attributes and skills, published by the NHS Commissioning Board, October 2012, pp17-18. This version incorporates The National Health Service (Clinical Commissioning Groups) Regulations 2012S.I. 2012/1631. https://www.england.nhs.uk/wp-content/uploads/2012/09/ccg-members-roles.pdf  (Accessed 21/01/2016)