Monday, 25 April 2022 — Dr Malcolm Kendrick
[Until it died]
Once upon a time I was a member of the General Practitioners Committee. A sub-committee of the British Medical Association that represents GPs. This was during a time when the Quality and Outcomes Framework (QOF) system was being rolled out. There is hardly anyone working in the NHS, including almost all hospital doctors, who has any idea what QOF is. But GPs [Family physicians] sure as hell do.
I donned my armour and battled against it, in a purely Don Quixote style. I was aware that I was tilting at windmills, but I felt the need to do something, however unlikely I was to succeed. This stance did offer the advantage that I could then say two things that really irritate other people. First ‘It wasn’t my fault.’ Even worse ‘I told you so.’
Ah yes, what on earth is he talking about this time? What is this QOF thingy, you ask? And what has it to do with evidence-based medicine? Well, you could say that QOF represents the inevitable end-point of evidence-based medicine. The crowning glory of a system designed to remove uncertainty from clinical practice. Replace it with carefully crafted treatment algorithms, based on the best possible evidence.
To explain in a little more detail. QOF itself is a system whereby GPs can earn points for reaching various targets. They are then paid money for each point gained. How much money? You can skip the next bit, but it makes me laugh. It is but the tip of a mighty iceberg of complexity. A system that makes filling in a tax return look like light-hearted fun.
‘To work out your actual QOF value for your practice, you need to divide your population by 8,479 to derive a factor and multiply this to the QOF point value to derive the actual QOF value for your practice.
For example, if your practice has 4000 patients.
4000/8479 = 0.4717537
0.4717537 x £187.74 = £88.57 per QOF point.’
At present it is possible to achieve a maximum of 567 points (last time I looked). This equates to an income of roughly fifty thousand pounds, for a practice of four thousand patients. If, that is, you achieve all the points on offer. Which is tricky.
What sort of activity earns points? Well, take diabetes as an example. You start by establishing, and maintaining, a register of all patients, aged seventeen or over, with diabetes. The register must also specify the type of diabetes – where a diagnosis has been confirmed.
You may think this all sounds perfectly reasonable, but then ask yourself why does it need to be done? In the UK, all GPs use computer systems. If someone has diabetes, this will be known. It will be on screen. It’s not as if the GP is going to be taken by surprise to learn that the patient has diabetes when they carry out an audit.
In short, an up-to-date list makes no difference to their management. Nor are you going to suddenly stumble across more patients with diabetes simply by the magical act of creating a list.
No, the reason why a list must be created is that you can gain points for such things as lowering the blood pressure to a ‘target’ level in the approved percentage of patients. Or driving the cholesterol level down below the ‘target level’, or getting the blood sugar (HbA1c) level below the ‘target’ level in the approved percentage of patients.
In short, for QOF to work, the GP needs to create database after database of different diseases. Then carry out audit … after audit. What a great use of clinical time it all is. Appointment after appointment filled with patients called in to have their annual blood pressure check, which just sneaks in just below target level – every single time.
For the pharmaceutical companies this is manna from heaven. Every patient with diabetes logged and audited. Every one driven to reach a ‘target’. A target that will inevitably require medication. Medication that the pharmaceutical company just, ahem, happens to have developed. Medication where they just, ahem, happen to have done all the clinical trials.
In addition to QOF, you also need to link everything into NICE guidelines. NICE stands for ‘The National Institute for Health and Care Excellence’. They produce magnificent ‘evidence based’ medical guidelines on such matters as the management of low back pain, or treatment of high blood pressure. Amongst a multitude of other things.
Some of these guideline documents are, literally, hundreds of pages long. But if you do not follow them then you are in trouble. You could find yourself struck off the medical register.
If you add NICE to QOF, what do you get?
What you get are extraordinarily rigid pathways, and algorithms, for treating patients. Soviet style central planning at its finest. Everything commanded from on high, everything measured, everything inspected. All five-year plans in place …comrade.
At this point you may well ask, why the need for highly trained clinicians? Disease X requires treatment Y, at dose Z, to achieve the desired outcome. Anyone sitting in front of a computer can do this. It requires no knowledge of why you are doing any of it.
Equally, it requires zero understanding of the complex relationship between various physiological systems, or the specific medical needs of a patient either. What if the patient has three different diseases, where you must balance one system against another? What if no-one has ever studied the use, benefit, or harms, of four different drugs given at the same time? How do you balance one set of guidelines against another?
Leaving such issues to one side, depending on your philosophy of life, you may believe this is all a fantastic idea. Repeatable and reliable treatment protocols replacing potentially flawed clinical judgement. Factory worker vs. skilled artisan. Ford vs Rolls Royce. In general, we know who usually wins this one. Command and control vs. individual decision making. No contest.
However, if the medical authorities decide, as they have done, to go down the bureaucratic ‘command and control’ model – based on the best evidence available – then there is a critical thing. It is the absolute requirement to be certain that the evidence you use is of the highest quality. Untouched by bias … if not, your house of algorithms simply collapses.
So, how reliable is the evidence base? Here is what Richard Horton (editor of The Lancet) stated a few years ago in an article ‘Has science “taken a turn towards darkness”?’
“The case against science,” wrote Richard Horton, editor of the medical journal the Lancet, “is straightforward: much of the scientific literature, perhaps half, may simply be untrue.”1
A while back I wrote a book called Doctoring Data, in which I tried to help people understand the many, many, ways in which the data from major clinical trials are manipulated and biased. How they are carefully designed to obtain only the desired results. I also attempted to clarify the endless data manipulations used to report the results themselves.
If I had to sum up the overall message of the book, it is that we are all, essentially, bunny rabbits caught in the headlights of an onrushing car. The onrushing car, in this case, being pharmaceutical company profits.
More recently the BMJ published an article entitled ‘The illusion of evidence-based medicine.’ 2
It begins, thus:
‘The advent of evidence-based medicine was a paradigm shift intended to provide a solid scientific foundation for medicine. The validity of this new paradigm, however, depends on reliable data from clinical trials, most of which are conducted by the pharmaceutical industry and reported in the names of senior academics. The release into the public domain of previously confidential pharmaceutical industry documents has given the medical community valuable insight into the degree to which industry sponsored clinical trials are misrepresented. Until this problem is corrected, evidence-based medicine will remain an illusion.’
It goes on to say:
‘Regulators receive funding from industry and use industry funded and performed trials to approve drugs, without in most cases seeing the raw data. What confidence do we have in a system in which drug companies are permitted to “mark their own homework” rather than having their products tested by independent experts as part of a public regulatory system? Unconcerned governments and captured regulators are unlikely to initiate necessary change to remove research from industry altogether and clean up publishing models that depend on reprint revenue, advertising, and sponsorship revenue.’
I have been saying this, or something pretty much like this, for years. As have many other voices … howling in the wilderness. Has anything changed? Well, yes, it has changed. It has all got considerably worse.
For example, much of the recent research done during the COVID19 pandemic was almost laughably biased and dreadful. Anything that could make a pharmaceutical company money was promoted ruthlessly – did someone say remdesivir. Anything where no little money could be made was slammed though the floor. Did someone say hydroxychloroquine?
As for the vaccine trials themselves. Let us draw a discrete veil over those …vague approximations to science.
What we currently have is a crisis in evidence-based medicine. The evidence that we use is, at best flawed and incomplete. At worst, just plain wrong. Yet, this is this evidence used to create the NICE guidelines and drive the QOF targets.
Any wonder so many GPs are completely fed up. It is not the only reason, but it is a major reason. ‘You trained me for ten years, now I cannot even make a bloody clinical decision. What is the point?’ A GP colleague calls it ‘monkey medicine.’ In that a well-trained monkey could do it.
When QOF was first being heavily promoted as the glorious future of primary care, I made a prediction. I predicted that life expectancy of the elderly (where most of the QOF points aggregate) would gently start to fall. This would happen because everyone was going to be monitored and measured. Then treated with drug after flawed ‘evidence-based ‘drug.
Two problems. First, this would inevitably drive polypharmacy [many different drugs prescribed simultaneously], and the evidence for this is overwhelming, and clear. Here is a short section from a paper examining the increasing use of multiple medications. ‘Medication usage change in older people (65+) in England over 20 years: findings from CFAS I and CFAS II.’
‘The number of people taking five or more items quadrupled from 12 to 49%, while the proportion of people who did not take any medication has decreased from around 1 in 5 to 1 in 13.’3
Polypharmacy is, in of itself, potentially dangerous, in that all the different drugs can start interacting with each other in unexpected and, often, damaging ways. Many studies have demonstrated this unequivocally. 4
These inherent problems with polypharmacy are, of course, made far worse by being driven by biased evidence. It does not take a genius to add two and two in order to predict that, in this situation, life expectancy may well go down, rather than up.
Biased evidence base + polypharmacy = increased morbidity and mortality
In support of this, here is an analysis from Imperial College London entitled ‘Life expectancy declining in many English communities even before pandemic.’
‘A substantial number of English communities experienced a decline in life expectancy from 2010-2019, Imperial College London researchers have found … For such declines to be seen in ‘normal times’ before the pandemic is alarming.’’ 5
Cause and effect? This cannot be said for certain – rather too many variables flying about. I know what I think.
However, one thing you can certainly argue is the following. If the evidence we now use to audit and treat everyone, using QOF, was of unbiased high quality, then you should expect to see some improvement in life expectancy.
But that is not what happened. What happened, was a fall. Not a huge, oh my God fall, but a fall, nonetheless. Has anyone pointed to QOF, and NICE, and the endless proliferation of guidelines as potential factors? You already know the answer to that one. Not a chance.
Whilst other countries do not have QOF, or NICE, the relentless march of evidence-based guidelines, and the subsequent clinical algorithms that they are based on, has become a world-wide phenomenon. The US, too, is seeing a fall in life expectancy.
At one time, long ago, I was a great believer in evidence-based medicine. It seemed like a good idea at the time. I now recognise that I was hopelessly naïve. First, as a student of history, I should have known that centralised command and control systems always end in disaster.
This happens, no matter how well intentioned it may have been to start with, and QOF was well intentioned. A crushing and inflexible bureaucracy will inexorably grow, and suffocate, and drain enthusiasm and energy from the workplace. The guidelines themselves would also, inevitably, end up as a Procrustean bed, upon which no patient can ever fit. So, you have to chop bits off, or stretch, as required.
Procrustes “the stretcher [who hammers out the metal]”, was a rogue smith and bandit from Attica who attacked people by stretching them or cutting off their legs, so as to force them to fit the size of an iron bed. [The process was always fatal].
In this case, the Procrustean bed has been further distorted by the fact that the evidence base itself rests of quicksand. It is a horribly biased mess. So, yes, evidence-based medicine was a good idea (sort of). It died long ago. R.I.P.
As an end-note, the impact of QOF was reviewed a few years ago. In 2017, to be precise. Nothing since that I am aware of. As the study concluded:
‘The lack of effect of the QOF on mortality is surprising, given that the indicators are based on high-quality evidence of effectiveness of interventions. Why this is the case is not clear… ‘6
Not clear… There are none so blind as those who have not eyes to see.
1: https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(15)60696-1/fulltext
2: https://www.bmj.com/content/376/bmj.o702
3: https://academic.oup.com/ageing/article/47/2/220/4237359
5: https://www.imperial.ac.uk/news/231119/life-expectancy-declining-many-english-communities/
The New Dark Age 