AIMS Journal, 2007, Vol 19 No 4
Debbie Chippington Derrick questions the Healthcare Commission's Review of Maternity Services 2007
As an exercise in obfuscation the Healthcare Commission's Review of Maternity Care is a classic example. Instead of evaluating the quality of maternity care in a meaningful way the Healthcare Commission has used a similar scoring system to that used by the Eurovision Song Contest where there are a set number of points to be assigned however bad all of the songs are, with the result that the 'less worse' songs still obtain a high score.
The Review of Maternity Services 2007 was published by the Healthcare Commission, England's healthcarewatchdog, on 25th January 2008. It claims 'Our review explored how organisations in the NHS are improving the way in which they deliver maternity services.' As my local unit was rated as a 'best performing' Trust I decided to have a look at what this actually meant, and how helpful the Review is in practice at illustrating how a Trust is performing.
There are 148 NHS Trusts detailed in the Review; 38 Trusts receive the classification 'best performing' (26%), 42 as 'better performing' (31%), 37 as 'fair performing' (22%) and 31 as 'least well performing' (21%). Interestingly the majority of the 'least well performing' Trusts were in London, where there were no 'best performing' and only one 'better performing' Trusts.
The Review emphasised that:'An assessment of "least well performing" does not mean that a trust is providing care which is unsafe. If we have concerns that a trust is unsafe, we do not hesitate to use our powers of enforcement by, for example, carrying out investigations.' The Review did not acknowledge that Trusts categorised as 'best performing' may also have significant failings that need to be addressed. I am also left with concerns; if the standard of this Review is an indication of how effective the investigations are, how is it going to be able to identify when a Trust is unsafe?
Trusts were rated on 5 indicators in 25 different areas. My local Trust, which ranked as a 'best performing' Trust, achieved an average score of just under 3.5 across all these indicators. The classification of performance was based on each Trust's averages of all 25 scores with
| best performing | greater than 3.28 average score | |
| better performing | less than 3.28 | |
| but greater than 3.00 average score | ||
| fair performing | less than 3.00 | |
| but greater than 2.76 average score | ||
| least well performing | less than 2.72 average score | |
| (worst score was 1.96) |
It quickly becomes apparent that the measures used are not assessing any standards of care that are considered desirable or appropriate, but that the performance of each Trust is measured solely against the performances of all other Trusts. So the outcome of the study's finding that some Trusts are performing well and others less well is a design feature of the study, not a reflection of the quality of our maternity services.
Since the study does not attempt to measure the performance of Trusts against standards that should be expected from such a ser vice or might be considered desirable, were it to be repeated in the future it would give absolutely no indication of whether our maternity services had improved or declined; it would simply show how each Trust is performing against the others, at that time, on the chosen set of questions.
For most of the indicators the scoring system was set up so that the worst 6.25% of Trusts would receive One point, then next worst 25% of Trusts receive two points, the middle 37.5% of Trusts three points, then next best 25% four points and the best 6.25% five points; as illustrated in Figure 1 below. There were some slight adjustments of this method for some of the indicators, as will be seen in those described later where more than one aspect was included in the indicator.
The 25 indicators are listed below and although there are some important aspects that are clearly missing such as place of birth, normal birth rates and infant morbidity, on the surface this looks as though this addresses important issues.
Below are details of four of the indicators, and how my Trust performed on each; other performance indicators are similarly structured around how well the Trust did in meeting a particular aspect of care in that area.
Indicator 1 -Women not receiving NICE recommended number of antenatal appointments
This is not based directly on whether a Trust is succeeding in meeting the recommendation of the NICE Guideline,1 but how it ranks against other Trusts in meeting this target.
For each Trust the results from the women's survey carried out in November 20071are used. It considers the percentage of women who made contact with the service prior to week 16 and had their baby after 37 weeks, who clearly did not get the number of appointments they should have. The Review acknowledges this is an underestimate due to lack of detail in the way the data was collected from the women.
The performance of all the Trust were used to generate a scoring system for this indicator based on what percentage of women clearly had not had the recommended number of antenatal appointments:
My Trust scored 4 points for this, as the survey showed that just under 16% of women were clearly not receiving the recommended number of appointments. This is a high score despite the fact that the Trust is failing to provide the appropriate number of antenatal appointments for about 1 in 6 women.
Indicator 2 - Appropriate use of caesarean section
The scoring system for this was as follows
The 24.38% CS rate, 31.53% VBAC rate and 27.09% ECV rate are the median (middle rate, when the rates for all the trust are put in order) of the rates of the Trusts that could provide this data.
My local unit scored three, one automatic point, one point for a very slightly lower primip CS rate of 23.84%, and one for knowing some of these rates. It was unable to provide the ECV rate, and the VBAC rate is an unbelievable low of 10.32%. The report documents that 92 Trusts of the 148 were unable to provide full data for this indicator ; it really seems quite shocking that only 56 Trusts were even able to provide answers to these questions.
The World Health Organisation recommends that caesarean section rates should not rise above 10-15%, with the evidence being that rates higher than 10% confer no benefit; so scoring a point for a rate of under 24.38% for first time mothers seems a little meaningless in terms of good performance.
Indicator 4 - Maternal morbidity
This indicator focused solely on blood loss and the time scale in which women had any perineal stitching carried out; no other aspects of maternal morbidity were considered.
One point was given regardless of performance with an additional point for each of:
The percentages used for the scoring system were obtained as follows: 25% of Trusts carried out 91.11% or more of stitching with in an hour, and 75% of Trusts carried out 84.43% of stitching within an hour. The median rate for haemorrhage in excess of 2500ml was 0.1939%.
My Trust only scored 2 showing there was room for improvement, as the suture rate was under 70% within an hour, one of the worse performances of all the Trusts, and the haemorrhage rate was over 0.37%, nearly twice the average rate.
However, I am left wondering what this really tells us about morbidity. These very specific aspects seem to fail to capture the concerns of most women. What about the actual rates of women needing stitching for example?
Indicator 17 - Staffing levels
One point was given regardless of performance and one point for each of:
The rates used for the scoring were obtained as follows: 25% of Trusts had less than 28.13 midwives per 1000 births, and 75% of Trusts had less than 34.90 midwives per 1000 births. The consultant obstetric and anaesthetist requirements were one of the few based on guidelines for numbers required, because this is one of the few areas where recommended staffing levels seem to be being met. It seems amazing that funding can be obtained when it comes to recommendations for these high level staff, but almost nothing else.
My local trust scored 4 on this indicator, losing one point as it only managed 31.87 full time equivalent midwives per 1000 births. One of the most important factors for women during pregnancy, birth and the postnatal period is midwifery support, so to see the number of midwives available brought down to just 2 points in only one of 25 indicators (less than 0.08 of the total score between 1 and 5) is very depressing.
Indicator 18 - Integration of support workers
One point was awarded regardless of performance and then:
The rates used for the scoring where obtained were as follows: 12.5% of Trusts had maternity support workers at less than 5.12/1000 births, half had them at more than 7.534/1000 and 12.5% had them at more than 10.68/1000. 12 was the median value across all the Trusts of the number roles maternity support workers were carrying out.
My Trust scored only 2, so should I be concerned or relieved about this? It would actually seem that its low score may be an indication that it has not been led into the erosion of midwifery in the same way that many other Trusts have.
Surely before we start making these aspects into targets we need to know whether they confer any advantage to bir thing women, or whether they are a marker of less midwifery support? How many roles is it appropriate to delegate to a maternity support worker? Should maternity support workers really be under taking tests, giving health advice, carrying out vital sign observation, carrying out antenatal classes, supporting women in labour, being theatre assistants, being the second person at home birth, etc? Are they simply a cheaper, less well trained version of a midwife? Is this indicator evidence of a Trust doing better or one that has hit crisis point with midwife numbers?
There seem to be three main failings of this Review
These shortcomings mean that the Review fails to assess whether the Trusts are providing what mothers and babies actually need from a maternity ser vice. It fails to provide a method that can be used in the future to measure whether our maternity services have improved or not. Nor does it provide information about how different service providers within a Trust are performing, so data from a midwifery unit or a group of community midwives providing a good home birth service will be swallowed up within the data for the whole Trust.
This Review seems to be another wasted opportunity to address the changes that are really needed to improve care for mothers and babies, and we are left to ask how much of tax payers' money has been wasted on this empty exercise. We have to hope that the data that has been generated and is to be made available on CD next month will be put to better use.
AIMS comment
To find out how your Trust did and what questions were asked to assess the other area of competency of Trusts, go to
http://www.nhssurveys.org/Filestore/CQC/2007_Maternity_services_survey_report.pdf
Or look at www.birthchoiceuk.com/HealthCareCommissionSurvey/T130.htm.
Where you choose to have your baby and whom you choose to look after you in labour will affect the type of birth you have and the care you receive. The BirthChoiceUK.com (www.birthchoiceuk.com) explains your options and gives information to help you make those choices.
You have the right to choose where to have your baby.
"All women should be involved in planning their own care with information, advice and support from professionals, including choosing the place they would like to give birth....." National Service Framework for Children,Young People and Maternity Services, pg 5. Department of Health 14/9/2004
References
Glossary
The AIMS Journal spearheads discussions about change and development in the maternity services..
AIMS Journal articles on the website go back to 1960, offering an important historical record of maternity issues over the past 60 years. Please check the date of the article because the situation that it discusses may have changed since it was published. We are also very aware that the language used in many articles may not be the language that AIMS would use today.
To contact the editors, please email: journal@aims.org.uk
We make the AIMS Journal freely available so that as many people as possible can benefit from the articles. If you found this article interesting please consider supporting us by becoming an AIMS member or making a donation. We are a small charity that accepts no commercial sponsorship, in order to preserve our reputation for providing impartial, evidence-based information.
AIMS supports all maternity service users to navigate the system as it exists, and campaigns for a system which truly meets the needs of all.