Assessment for Improvement - Our Approach
CHAPTER 1: INTRODUCTION
Ensuring that assessments are relevant to those who use, and work in, healthcare:
1. Will our proposals ensure that we engage effectively with patients, the public and healthcare professionals? Are there other or different steps that we should be taking?
We consider the Healthcare Commission's proposals to be a good starting point but believe that they will need further development over time as lessons are learnt about what does and does not work well.
2a. Are we measuring and assessing what really matters for patients and the public?
We believe that the proposed five-point assessment process has more breadth than the current star rating system and so should prove a useful tool to support patients' decision making.
2b. Are we measuring and assessing what really matters for clinicians?
We consider it to be important that clinicians are involved in developing the elements by which the Healthcare Commission assesses services, carries out assessments and identifies the priority areas of care. This involvement will ensure that what the Healthcare Commission measures and assesses will be what matters most to clinicians.
2c. Are we measuring and assessing what really matters for different types of healthcare organisations?
We consider that the proposed assessment process will provide information that will support decision making in both PCTs and SHAs.
3. Is there anything else that should be included in our proposals?
We consider that there needs to be further clarity around the approach that the Healthcare Commission will take when there is concern that a standard is not being met.
4. How often should we present our findings and what format would you find most useful?
We would suggest that the Healthcare Commission's findings are presented at the organisation's annual public meeting.
CHAPTER 2: GUIDING PRINCIPLES FOR ASSESSING PERFORMANCE
Will our proposed approach lead to improvement, in particular:
5. Will our proposals identify failings in the provision of healthcare and lead to appropriate steps to address these?
We consider that, in theory, the proposals should identify failings in the provision of healthcare and lead to appropriate steps to address these. Until the proposals are put into practice, however, it is difficult to say how well they will work.
We note that the proposals rely on the Healthcare Commission receiving both consistent quality information and good local intelligence, but are concerned that these may not always be available.
6. Will our proposals offer sufficient support to healthcare organisations' continuous efforts to improve their services?
It is difficult to answer this question without more clarity on how the Healthcare Commission plans to use the information it collects and without knowing the approach that the Healthcare Commission will take when it carries out an inspection. We would first need to know how intensive the inspection process will be, whether the intensity of inspection would be dependent on the seriousness of the problem identified and the likely administrative burden on NHS organisations. Until these questions are answered it is not possible to comment on whether the proposals will offer sufficient support to healthcare organisations' continuous efforts to improve services.
In the longer term we understand that the Healthcare Commission plans to move to a fee based structure; we are concerned that the need for additional inspections might increase the financial pressure on poorly performing organisations.
7. Do you believe that the assessments that we make will be fair?
We consider that the assessments will be seen as fair so long as the Healthcare Commission:
- puts in place a demonstrably robust quality assurance process; and
- ensures that all its assessors are adequately trained.
8. Do you believe that we will make assessments transparently?
We consider that there needs to be further clarity around the assessment process before it can be described as transparent.
In particular there needs to be more information on:
• which third party assurances the Healthcare Commission will be using; and
• how the Healthcare Commission plans to calculate the summary scores that are then used to calculate the overall annual performance rating. (Annex 5 describes how the final score is obtained but not the summary score.)
CHAPTER 3: OVERVIEW OF THE NEW APPROACH TO ASSESSMENT
We propose to phase in the new methods of assessment rather than introduce them all in 2005/2006.
9. Do you have any concerns about this approach?
We consider that it is important that the approach to assessment is consistent. There is a risk that consistency will not be maintained, however, if, as proposed, the focus of assessment shifts from �getting the basics right' towards assessment that �promotes development and improvement'.
We are also concerned that, due to the uniqueness of trusts, it will be difficult to develop an assessment process based on developmental standards that allows meaningful comparison of one organisation against another.
10. Can you suggest better ways that we can use information? How can we help to assure and improve the quality of information available to us?
We consider that the Healthcare Commission should work with Internal Audit, External Audit and other independent assessors to help assure and improve the quality of information available.
11a. Does our proposed approach live up to the Government's principles for better regulation? In particular, will they achieve the right balance between effective assessment without undue burden on those assessed?
In theory, we consider that the proposed approach will live up to the Government's principles for better regulation and that it will achieve the right balance between effective assessment and placing undue burden on those being assessed. There is, however, still much developmental work to be done on the proposed approach and this work needs to be undertaken and tested before the question can be answered with certainty.
11b. Does our proposed approach live up to the Government's principles for better regulation? In particular, will they achieve the right balance between healthcare organisations taking responsibility for their own performance and effective independent assessment?
In theory, we consider that the Healthcare Commission's proposed approach will achieve the right balance between healthcare organisations taking responsibility for their own performance and effective independent assessment. The key test, however, will be the processes adopted by the Healthcare Commission for �follow up of concerns' and for �spot checks'. It is important that, when developed, these processes are seen to be robust and transparent.
CHAPTER 4: GETTING THE BASICS RIGHT
What comments do you have on:
12. The processes by which we are proposing to assess compliance with the core standards, in particular, the intended use of a trust's declaration that incorporates the views of other organisations in the local healthcare community?
In general we agree with the Healthcare Commission's proposed �light touch', self-assessment approach.
We are concerned, however, that production of the proposed public declarations may prove to be an administrative burden to organisations - though it is difficult to know how much of a burden this might be until the guidance on what needs to be declared is issued. We consider that the burden might be reduced if the public declaration were to be included as part of the annual report.
The proposed approach does not make reference to an organisation's Statement of Internal Control and the Assurance Framework; we believe that both of these might contain information that would be of interest to the Healthcare Commission.
13. The draft guidance that we have published on what trusts might want to take into account in satisfying themselves on compliance with the core standards?
Although we consider it helpful that the Healthcare Commission has provided guidance on what trusts might want to take account of in satisfying themselves on compliance with the core standards, we are concerned that the guidance may encourage a tick box mentality. There is a danger that the guidance, as published, will become just another checklist for organisations to complete, thereby increasing rather than decreasing the burden.
14. The information that we are proposing to use to consider outcomes relating to the core standards?
Overall we agree with the Healthcare Commission's proposed approach for classifying a trust's compliance with the core standards. Without further detail, however, on the information that the Healthcare Commission plans to use to consider outcomes relating to the core standards it is difficult for us to respond to this question.
We note that in some cases the Healthcare Commission will use results from patient surveys to test compliance. We believe that the patients who respond to these surveys are likely to be the most vocal and therefore consider that patient surveys may not always give a fair and representative view of a trust's performance.
This potential problem is illustrated by the scenario on page 25 where a patient survey identified problems of cleanliness at St Somewhere's Hospital Trust which triggered a meeting between the Healthcare Commission and Chief Executive. We would like assurance that the Healthcare Commission will not rely on just one source of information, e.g. patient surveys, when it makes a decision about a trust's performance but that it will look at the wider picture.
15. Our proposed approach to the measurement of existing targets?
We agree with the Healthcare Commission's proposed approach to the measurement of existing targets. In particular we were pleased to note that trusts will be assessed on both performance and in-year improvements.
16. The proposed approach to our assessment of a healthcare organisation's use of resources?
We agree with the Healthcare Commission's proposed approach to assessment of an organisation's use of resources using the results of work carried out by the Audit Commission's appointed auditors (for non-foundation trusts) and by Monitor (for foundation trusts).
We are not clear, however, how this will enable the Healthcare Commission �to present a more comprehensive picture of a trust's performance than has previously been possible' or how it will �avoid the possibility of a trust being criticised by one organisation, while another gives it a high performance rating'. The rating system being adopted by Monitor, for example, is very different to that being proposed by the Healthcare Commission and so there appears to be more potential for confusing the public rather than informing them.
17. Our proposed approach to the use of other regulatory findings?
We agree with the Healthcare Commission's proposed approach of using the findings of other regulatory bodies.
We note that a number of regulatory bodies have been set up for the review, inspection and regulation of NHS organisations. We are concerned that each of these bodies has developed a unique rating system based on different criteria which, if published, would be more likely to confuse rather than inform the public.
We consider, therefore, that all these bodies should work towards developing a consistent approach to the rating of NHS organisations.
CHAPTER 5: MAKING AND SUSTAINING PROGRESS
What comments do you have on our proposed approach to the assessment of:
18. New national targets?
We agree with the Healthcare Commission's proposed approach to assessment of the national targets.
We consider that the development of local targets will be challenging. Organisations are likely to set very different targets depending on local needs. There is a danger that this will introduce an element of inconsistency into the ratings.
19. Developmental standards generally?
It is not yet clear how the Healthcare Commission plans to assess the developmental standards. The consultation document says �because of the complexity of the task, we do not think there should be a single approach to how we assess performance'. We are concerned, however, that this will make it difficult to ensure a consistent approach.
20. The element of the domain of governance concerned with leadership?
We consider that the Healthcare Commission's plans to assess leadership should be linked to the integrated governance initiative and should start by considering board effectiveness.
21. The improvement reviews of particular aspects of healthcare across healthcare organisations, from the perspective of patients?
We consider it important that the Healthcare Commission seeks the views of a representative sample of patients, not just the vocal minority.
We believe that the improvement reviews should focus on the population as a whole, not a token group. The scenario on page 35, for example, focuses on treating a very small section of the total PCT population; in our view this does not constitute a major improvement to service.
CHAPTER 6: ANNUAL PERFORMANCE RATING
What comments do you have on our proposals for:
22. Making information publicly available, in particular, the possibility of publishing results as they become available within an annual cycle of review?
We agree with the Healthcare Commission's proposal to publish results as soon as they become available but deem it to be important that a publication date appears alongside each organisation's results.
We consider that, as the published results rely heavily on self assessment, they should be issued with a caveat stating �results based on assurances from the organisation and in the absence of conflicting information'.
23. The categories that we will use for the annual rating of an organisation's performance?
We do not consider that the proposed five point scale is adequate for describing an organisation's overall performance as there appear to be no detailed criteria underpinning the headings. What is the difference, for example, between an organisation that is rated �unsatisfactory' and one that is rated �serious concerns'?
We are also concerned about the use of the ratings �satisfactory' and �unsatisfactory'. These two ratings could be used to describe the performance of all organisations and so the remaining categories could be viewed as superfluous.
24. Do you have a view on the approach to aggregating the different components of the framework of assessment in calculating the annual rating?
We consider Option 1 in Annex 5 to be the better approach to aggregating the different components as it is transparent and appears less subjective than Option 2.
25. Do you have a view on how we incorporate assessment of leadership and organisational capacity in the annual rating? Should it be part of a single overall rating or a separate rating on the organisation's prospects?
We consider that the assessment of leadership and organisational capacity should be reported on separately as, unlike the other ratings, it relates to forecast performance.
In the future, when a methodology for assessing leadership and organisational capacity has been designed and properly tested, then it may be appropriate for it to be amalgamated into the overall rating.
CHAPTER 7: INDEPENDENT HEALTHCARE
26. Do you agree with our proposals for independent healthcare to reduce the burden of regulation through proportionate inspection that is effective in targeting risk?
We agree with the Healthcare Commission's proposal to undertake a risk based approach to assessment and to move towards an alignment of standards across the independent healthcare sector and the NHS.
27. What should be the essential parts of our approach to assessments of independent facilities where, subject to legislation, there is an alignment of standards across the independent sector and the National Health Service through the Standards for better health ?
We consider that, over time, the Healthcare Commission should work towards developing a single assessment process that can be applied to the NHS and independent sector.
28. What are the priorities in improving the collection and use of clinical and performance information from independent providers, and who should be involved in this work?
We consider that, subject to legislation, it is appropriate for the Healthcare Commission to assess independent healthcare providers by reference to the same core and developmental standards as applied to the NHS.
This change in regulation might mean that independent healthcare providers will require support in identifying appropriate information systems for producing descriptions of their performance in meeting the standards for the Healthcare Commission.
29. Would you like us to keep you informed of the Healthcare Commission's work?
ACCA would like to be kept informed of the Healthcare Commission's work.