Professor Malcolm Hooper’s Detailed Response to Professor Peter White’s letter to Dr Richard Horton about his complaint re: the PACE Trial articles published in The Lancet
28th May 2011
(Note that the complaint submitted by Professor Hooper to which Professor White responded can be accessed at http://www.meactionuk.org.uk/COMPLAINT-to-Lancet-re-PACE.htm ).
Professor Hooper believes that there are two related issues: (A) the inaccuracies in the undated letter sent by Professor Peter White on behalf of the co-authors of the PACE Trial article to Dr Richard Horton, Editor-in-Chief of The Lancet, refuting his complaint and (B) what Professor Hooper continues to believe are failures by The Lancet to fulfil its ethical duties with regard to the preparation and publication of the PACE Trial articles. Both are addressed in this document.
As Professor White’s letter is written in the third person, Professor Hooper has done likewise.
Part A: Reply to Professor Peter White’s response to the complaint submitted to The Lancet
Professor White’s letter, which can be read here, that was forwarded to Professor Hooper by Zoe Mullan, Senior Editor at The Lancet, appears to contain factual inaccuracies and errors; these are here addressed in order of presentation in Professor White’s letter.
In the interests of transparency, Professor White’s letter itself, together with evidence supporting an important aspect of the complaint to The Lancet, will be placed in the public domain together with this response.
· in his response to comments on “Protocol for the PACE trial” published in BMC Neurology on 28th July 2008 (http://www.biomedcentral.com/1471-2377/7/6/comments#306608) Professor Peter White wrote: “ME is not an exclusion for the trial….We did consider using the Canadian criteria for ME but….the London criteria for ME were chosen instead. The London criteria were based on the original description of ME, given by Melvin Ramsay”
· in her letter of 21st February 2011 Dr Rawle from the MRC referred to the disorder as “CFS/ME”, writing: “In your view the entry criteria used mean that the PACE trial is studying patients who have chronic fatigue and not CFS/ME, and you are concerned that the results might be applied to different patient groups. The clinical and scientific experts who reviewed the protocol were satisfied with the entry criteria, which are clear in the published protocol (which refers to “CFS/ME”) and are also set out clearly in the published results (which do not refer to “CFS/ME” but to “CFS”, and the Chief PI has now confirmed that they did not purport to be studying CFS/ME), so that clinicians and others using the results of the trial to inform their practice will know which patient groups the treatments should be applied to” (given the discrepancies, this cannot be so)
· by letter dated 16th June 2005, Dr Sarah Perkins, Programme Manager, MRC Neurosciences and Mental Health Board, confirmed that the use of the Oxford criteria as entry to the PACE Trial “will not be used to exclude patients with a diagnosis of ME”
· that the PACE Trial was studying those with CFS/ME was confirmed by one of the Principal Investigators (PIs), Professor Michael Sharpe, at the Press Conference held on 17th February 2011 at the Science Media Centre to launch the PACE Trial results, when he said: “CFS/ME, as it has come to be called, is a condition that is relatively common….There is an issue with some people regard(ing) CFS and ME as separate conditions….I think the majority view is that people regard them as the same….”
http://download.thelancet.com/flatcontentassets/audio/lancet/2011/18february.mp3
Given the substantial evidence that the PACE Trial did purport to include people with ME and was indeed studying “CFS/ME”, it remains to be clarified by Professor White on what evidence he relied in his letter to Richard Horton when he stated that the PACE Trial “does not purport to be studying CFS/ME”.
If the PACE Trial Investigators did not purport to be studying those with “CFS/ME”, why were participants “also assessed by international criteria for chronic fatigue syndrome…and the London criteria for myalgic encephalomyelitis” as described in The Lancet article methods section?
Furthermore, why did the Investigators use a version of the (unpublished) “London” criteria for ME in which cardinal symptoms of ME were deemed by the PIs not to be necessary?
Professor White states in his letter: “We were provided with the second revised version of the London ME criteria; we did not invent our own”. The original version of the proposed “London” criteria bears little resemblance to the version that was used in the PACE Trial (now understood to have been provided by the charity Action for ME as Dr Ellen Goudsmit PhD – who claims co-authorship of the proposed “London” criteria -- has confirmed that the version used was not the one provided by her directly to Professor White). If it was not provided by AfME, and if the PIs did not invent their own version, who were the authors of the version of the “London” criteria used in the PACE Trial? Dr Ramsay’s description requires neurological problems to be present for a diagnosis of ME to be made, but the PACE Trial version specifically states (on page 188 of the Full Protocol) that neurological disturbances “are not necessary to make the diagnosis” and further states about fluctuation of symptoms: “the usual precipitation by ‘physical or mental exercise’ should be recorded but is not necessary to meet criteria”.
Put another way, the version of the “London” criteria as used by the Investigators does not require the cardinal features of ME to be present in a subgroup of patients in a trial that purported to be studying “CFS/ME”.
Against this background, Professor Hooper notes with concern that the text of The Lancet article states that participants were also assessed by “the London criteria for myalgic encephalomyelitis (version 2) requiring postexertional fatigue”.
As for some time Professor White has been theorising as to whether or not CFS is different from or similar to other Functional Somatic Syndromes (J Psychosom Res 2010:68(5):455-459) it is strange that, when given the opportunity to settle this question once and for all, he simply states: “We studied the results for differently defined subgroups and they were similar to those in the entire group”. Given that the clear distinction between Ramsay-defined ME and somatisation disorder has been removed by the PIs by their use of version 2 of the “London” criteria, the remarkable similarity of results between the “differently defined subgroups” and the full cohort is unsurprising. Indeed, it was almost inevitable, given that the version of the “London” criteria used in the PACE Trial was virtually identical to the Investigators’ own Oxford criteria minus the presence of psychiatric illness.
In view of Professor White’s assertion that the PACE Trial “does not purport to be studying CFS/ME”, there has been much comment on the internet about possible fraud and deceiving of participants as well as the research ethics and steering committees. As Chief PI, Professor White had convinced the funding bodies that the trial would be studying those with CFS/ME and ethical approval and funding from government bodies including the Department for Work and Pensions, the Department of Health and the Scottish Chief Scientist’s Office, as well as the MRC, were granted on the basis that the PACE Trial did purport to be studying those with “CFS/ME”, which Professor White has now denied in writing.
It is notable that The Lancet has made no attempt to address this point in response to Professor Hooper’s complaint and has provided no explanation for agreeing to fast-tack the article, other than confirming that the executive editor took Peter White on trust.
The Lancet’s “Information for Authors” states: “For research papers that are judged to warrant fast dissemination, which will usually be randomised controlled trials (despite its title in the documents, the PACE Trial was not a controlled trial), The Lancet will publish a peer-reviewed manuscript within four weeks of receipt”.
Given that the PACE Trial article was fast-tracked, the question arose as to whether or not there had been sufficient time for scrupulous checking of the data by The Lancet’s own statisticians. For the avoidance of doubt, when on 29th March 2011 the executive editor responsible for the publication of the PACE Trial articles was asked how The Lancet’s statisticians could have let such conflicting interpretation of the data be published in a journal of its reputation, he confirmed that he had taken Peter White on trust, saying (verbatim): “We can only do what we can do. We have to take things on trust. We don’t get the statisticians to go round and check every calculation that’s been done. It’s not up to the statisticians to advise on all the adding up”.
However, when on 31st March 2011 this issue was raised with a different executive editor, he was astounded to hear that the executive editor responsible for the publication of the PACE Trial article had acknowledged that it had not been rigorously checked by The Lancet’s own statisticians before publication; he said that all studies to be published go for scrutiny by the journal’s own statisticians and that he himself had set up this process in 1990. It is thus not clear how meticulously The Lancet’s statisticians checked the data before publication.
They certainly did not pick up (alternatively they were not concerned about) the fact that on the SF-36 physical function score, it was possible for a participant to have a fatigue rating that was both “normal” and “abnormal” depending on which of the Investigators’ various definitions was applied. Indeed, identical responses could both qualify a person as sufficiently “fatigued” for entry to the PACE trial and later allow them to be deemed to have “normal” levels of fatigue (“normal” meaning as defined by the Investigators themselves, which does not equate to “recovered”). What’s more, as with physical function, it would be possible for a person to record a poorer score on the CFQ (Chalder Fatigue Questionnaire) on completion of the trial than at the outset, yet still be deemed to have attained “normality” on this primary outcome measure. It cannot be acceptable to describe PACE participants as having “normal” levels of fatigue and physical function when they could simultaneously be sufficiently disabled -- as judged by their levels of fatigue and physical function -- to have qualified for entry into the PACE Trial in the first place.
It remains Professor Hooper’s view that it is astonishing that such a manifest contradiction survived The Lancet’s supposedly rigorous peer review process, about which Richard Horton stated on 18th April 20011 on Australia’s ABC Radio National: “the paper went through peer review very successfully, it’s been through endless rounds of peer review and ethical review so it was a very easy paper for us to publish” (see below).
Quite certainly, those appointed by The Lancet to conduct this supposedly rigorous peer-review process failed to be concerned about the highly misleading associated Comment by Bleijenberg and Knoop published in The Lancet. Despite the absence of any reference to “recovery” in the PACE Trial article itself, Bleijenberg and Knoop lean heavily on this concept, stating: "PACE used a strict criterion for recovery: a score on both fatigue and physical function within the range of the mean plus (or minus) one standard deviation of a healthy person’s score. In accordance with this criterion, the recovery rate (sic) of cognitive behaviour therapy and graded exercise therapy was about 30%". This is plainly wrong. Despite their stated intention to do so, White et al did not report the number of participants who “recovered”, yet The Lancet’s peer-reviewers permitted such a blatant error to be published and hence to be cited unchallenged in the literature in the future. Indeed, in 2000 the UK’s leading medical statistician, Martin Bland, (then at St George’s Hospital Medical School, London, but now Professor of Health Statistics, University of York) pointed out significant statistical errors in a paper by Simon Wessely and Trudie Chalder published in the BMJ; Wessely attempted to absolve himself from any blame but Bland was robust: “Potentially incorrect conclusions, based on faulty analysis, should not be allowed to remain in the literature to be cited uncritically by others” (Fatigue and psychological distress. BMJ: 19th February 2000:320:515-516). This is exactly the situation that now pertains at The Lancet, about which its senior editors appear entirely unconcerned.
Patient reports of adverse reactions to these interventions are legion, so it is all the more remarkable that the PACE Trial Investigators did not implement the rigorous and systematic investigation of this area that the Trial Protocol led one to expect.
The cavalier dismissal of the PACE Trial participant feedback contained in Professor Hooper’s complaint – indicating the impact of deterioration following participation in the interventions involved in the PACE Trial -- is disturbing.
The recognition that this change in recruitment criteria “may affect generalisibility” is highly significant. A fundamental problem relating to generalisability is that the findings of a study on the “Oxford” fatigue criteria recruits are being applied to other patient populations. Taken together with the assertion that PACE Trial “does not purport to be studying CFS/ME but CFS defined simply as a principal complaint of fatigue that is disabling, having lasted six months, with no alternative medical explanation”, the consequences of these statements for clinical practice may be disastrous for patients with classic ME/CFS.
In the Investigators’ published reply to critical correspondence (The PACE trial in chronic fatigue syndrome – Authors’ reply; Lancet 17th May 2011: doi:10.1016/S0140-6736(11)60651-X), they state: “However we defined CFS and myalgic encephalomyelitis, we found that cognitive behaviour therapy and graded exercise therapy provided a significant and clinically useful advantage of moderate size over adaptive pacing therapy and specialist medical care, but were no less safe”.
This message has been picked up by the “NHS Evidence” website (“Eyes on Evidence”: http://www.evidence.nhs.uk/) which has translated it into the statement that: “CBT and GET are the most effective form of treatment. The study also showed that they work however CFS or ME is defined”.
It is alarming that practitioners in the NHS are being advised that the findings of the PACE Trial can apply no matter what definition is used (be it the Canadian criteria, the Ramsay definition, the Fukuda criteria or the Oxford criteria for “fatigue”), as such an assertion is potentially damaging to patients and requires to be addressed as a matter of urgency, since that message cannot be sustained by the published findings (which relate only to the Oxford criteria for “fatigue”).
He further notes that this statement goes hand in hand with a failure to address the relevant points made in his complaint. The critique of the reporting by the authors on “outcome results” in the Lancet paper included, as noted in point 6 above, the observation that a rating which qualified a person as sufficiently impaired to enter the PACE Trial could also be considered as indicating that the person had attained “normal” function if scored at the end of the Trial. This applied to both of the primary outcome measures used to gauge the efficacy of the Trial interventions. Professor Hooper finds it incomprehensible that Professor White claims to have difficulty in understanding his comment on this important issue.
Given that the data fed into the trial arms results may or may not have been influenced by the “treatment” (according to the Chief PI), then there is no objective evidence of outcome measures at all (merely participants’ subjective answers to questionnaires).
It is a matter of logic, not statistics: how can feeding in admittedly ambiguous data possibly lead to an unambiguous conclusion?
In support of his use of the six minute walking test, Professor White cites a paper by Butland RJ et al without disclosing that the test validated by Butland et al was not used in the PACE Trial because the PIs used their own version of it, which was in fact in contradiction to the recommendations of Butland et al (BMJ 1982:284:1607-1608). Professor White has not adequately addressed this issue. Butland et al cite a further paper (McGavin CR et al; BMJ 1976:I:822-823) which draws attention to the difficulty of achieving reproducible results with such a test, stating that it needs to be carried out twice to achieve reproducible results. Unless this protocol was followed (which is not the case in the PACE Trial), the test is invalid according to the reference cited. Moreover, the six minute walking distance test has been shown to be influenced by test familarisation so is potentially unreliable (Gibbons WJ et al; Cardiopulm Rehabil 2001:21:(2):87-93). Furthermore, the six minute walking test has low test/re-test reliability (especially as the assessors knew to which of the intervention groups the participants had been allocated in the trial, masking of the assessors being deemed “impractical” by the PIs).
In any clinical trial the important thing is the designated primary outcome measure and how it differs between groups. It is remarkable that the primary outcome measure relied on participants’ subjective reporting, given that they suffer from what the Wessely School refer to as “perceived disability” (BMJ 2003:326:595-597). This means that on the one hand, the Wessely School believe that people with “CFS/ME” are unreliable in their own assessment of their disability (asserting that people with ME/CFS only “perceive” themselves to be ill and that they hold “aberrant illness beliefs”), yet on the other hand the Wessely School have based the outcome of a £5 million study on such patients’ personal assessment of their disability (ie. PACE Trial participants were deemed capable of accurately reporting their symptoms/disability). In other words, the PIs are satisfied that the primary requirement to prove that CBT and GET are effective was for participants (whose judgment the PIs regard as suspect) to say that they are effective.
“We did not ask participants to undertake a practice walking test for the reason mentioned in the complaint; post-exertional fatigue being a characteristic feature of CFS”: post-exertional fatigue is documented in the CMO’s Working Group 2002 Report on CFS/ME and in the 2007 NICE Guidelines on CFS/ME (CG53), where it is noted that post-exertional exacerbation of symptoms can last for days or weeks. If he accepts this is the case, why did Professor White choose a single, one-off test that cannot measure the ability of a participant to maintain a particular level of activity? Did participants in the GET arm who, on average, walked the furthest, relapse the following day? Conversely, might the APT group (judged to have performed relatively poorly in this walking test) have been able to sustain higher levels of activity in the longer term because they had learned not to push themselves to the point of relapse?
“The objective walking test favoured GET over CBT”: the walking test as carried out in the PACE Trial is not an objective test as most scientists understand “objective”. The test appears to provide reproducible measures only when properly conducted (ie. repeated as required for validity, which was not the case in the PACE Trial).
If participants’ cardio-respiratory and immunological profiles had been documented before and after the walking test, these would have constituted objective measures of increase or decrease in post-intervention physical capacity (especially the measurement of TNFa, which Professor White knows remains elevated for three days post-exercise in “CFS/ME”: “The pro-inflammatory cytokine TNF-a is known to be a cause of acute sickness behaviour, characterised by reduced activity related to ‘weakness, malaise, listlessness and inability to concentrate’, symptoms also notable in CFS” : White PD et al; JCFS 2004:12 (2):51-66).
It is remarkable that Professor White states that post-exertional fatigue is a characteristic feature of “CFS”, given that the PACE Trial entry criteria (the Oxford definition of CFS) does not even mention it, let alone require it, and that Professor White has stated the PACE Trial was not studying “CFS/ME”.
In the PACE Trial, the walking test was a secondary outcome measure in which the increased distance walked in the GET group was only 45 metres compared with those who received standardised “specialist” medical care (which amounted to little more than being given a leaflet, advice on balancing exercise and rest and antidepressants if required), which is hardly a ringing endorsement of the intervention.
It is interesting to note that complete data on the 6-min walking test was available for only 69% to 76% of participants, a completion figure roughly 20% lower than those for the other secondary outcome measures. Does this imply that 25-30% of the participants were unable or unwilling to cope with the walking challenge, or might there be other explanations?
The walking test results show that the average increase for the GET group was the "best" result of the 4 arms, at an average increase of 24.5%. However, the absolute distances walked in 6 minutes were only 312 metres (baseline) and 379 metres (after 52 weeks), a change of 67 metres in a year. As a comparison, Butland et al (one of the references given by White et al) tested patients with 'stable chronic respiratory disability owing to various diseases' and found that participants were able to walk between 200 and 550 metres.
In comparison with these patients, the 379 metres walked in 6 minutes by the GET group seems a very poor result indeed.
Of further note is that the Department for Work and Pensions Training & Development ESA Handbook (2009) states at section 3.1.9: “Variable and fluctuating conditions…If a claimant cannot repeat an activity with a reasonable degree of regularity, and certainly if they can perform that activity only once, then they should be considered unable to perform that activity”. Moreover, Lord Freud, Parliamentary Under Secretary of State for Work and Pensions, is on record thus: “I can respond to the noble Countess, Lady Mar, on fluctuating conditions. It must be possible for all the descriptors to be completed reliably, repeatedly, and safely, otherwise the individual is considered unable to complete the activity” (Hansard; Lords: 16th March 2011).
This means that according to English Regulations regarding entitlement to incapacity benefits, the version of the six minute walking test relied upon in the PACE Trial does not provide meaningful data about health or disability for patients with ME/CFS (or even for those with CFS/ME or CFS).
One of the many things lacking from the trial is a true control group and it seems that the unexpected effectiveness of “specialised” medical care (ie. the group that received no intervention) has made it harder for the PIs to demonstrate an unambiguous benefit from CBT or GET.
A significant failure of the PACE Trial is that it does not provide a reliable answer to the question most relevant to patients and health-care providers: which of the interventions, if any, best enables a sustainable improvement in the physical capacity of a patient with ME/CFS? In the light of this, Professor White’s justification for dropping post-intervention actigraphy appears specious.
What was actually said by Professors Sharpe and Chalder features in The Lancet’s podcast:
http://www.meactionuk.org.uk/pacepressconf.html
Having emphasised how tightly the PACE Trial had conformed to the MRC’s rigorous guidelines, Professor Sharpe handed over to Professor Trudie Chalder, who unambiguously conveyed to the journalists that the PACE Trial had been resoundingly successful, saying: “OK, so we had two primary outcomes, they were fatigue…and physical functioning and I’m going to talk about both these outcomes…. If you take these two outcomes together, that is fatigue and physical functioning, again you see the same pattern of results with graded exercise therapy and cognitive behaviour therapy doing better than specialist medical care and adaptive pacing therapy and if you think of the number of people who got back to normal levels of functioning and fatigue then you see twice as many people in the graded exercise therapy and cognitive behaviour therapy group improving and getting back to normal compared to the other two groups”. Professor Chalder went on to tell the journalists: “The effect that we see in terms of the improvement was similar across all the outcomes we measured. The effect was the same or very similar if we looked at people who were operationally defined as having chronic fatigue syndrome but of these there was a percentage of those who fulfilled operational criteria for ME (the version of the “London” criteria for ME used in the PACE Trial had not been operationalised) and again we saw exactly the same patterns of results so we can be quite confident that the pattern of results is fairly robust across different definitions or different ways of defining the illness”.
In defence of published criticisms by David Tuller of his chosen entry criteria in the New York Times (ie. that the problem of definition had limited the interpretation of the trial treatments), on 14th March 2011 Peter White replied: “We also assessed trial participants to see if they met two other definitions of the illness that are favored by some scientists”, to which David Tuller responded: “The article asked whether findings among a population defined by one set of criteria would apply to populations defined by ‘very different criteria’. In this study, all participants were first defined, identified and selected not by different criteria but by the same criteria, the so-called Oxford criteria used in Britain. Subgroups within that already screened population who also meet secondary criteria are not easily compared to patients who have not been screened, since an unknown number who met the secondary criteria might not have met the study’s criteria for inclusion. The gold standard for making comparisons across groups of patients identified by three varying case definitions would be a study with three completely separate cohorts, not one large sample with two embedded subgroups”.
At the press conference Professor Chalder went on to acknowledge the funders, mentioning that the primary funder was the MRC; she named the Department of Health and the Scottish Chief Scientist’s Office but made no mention of the Department for Work and Pensions, a surprising omission given that the PACE Trial is the only clinical trial that the DWP has ever funded. It is known that this was on the recommendation of Professor Sir Mansel Aylward, then Chief Medical Officer at the DWP, now at UNUMProvident Centre for Psychosocial and Disability Research at Cardiff, who attended meetings of the PACE Trial Steering Committee as an observer.
Mindful of the PIs’ comments and of the plainly erroneous assertion made by Belijenberg and Knoop in their Lancet Comment, it seems that there has been a co-ordinated attempt to promote the PACE Trial as a successful study, as no attempt was made to correct the wholly inaccurate reporting in the press of the PACE Trial outcome, for example:
on 18th February 2011 The Guardian carried an article by Health Editor Sarah Bosely entitled “Study finds therapy and exercise best for ME”. The article continued: “The biggest-ever study of treatments for ME, known as chronic fatigue syndrome, has found that more people recover if they are helped to try to do more than they think they can – rather than adapting to a life of limited activity….It found that patients showed more improvement – and a small minority recovered completely – after cognitive behaviour therapy (CBT), one of the so-called ‘talking therapies’, or graded exercise therapy (GET)….Trudie Chalder, professor of cognitive behavioural psychotherapy at King’s College, London, said that ‘twice as many people on graded exercise therapy and cognitive behaviour therapy got back to normal’ compared with those in the other two treatment groups…Professor Peter White from Barts and the London school of medicine and dentistry, said: ‘this is a real step forward in informing patients with CFS/ME which treatments can help to improve their health and ability to lead a more normal life’ ”.
It is noted that the PACE Trial page on The Lancet’s website specifically directs people to the article by Sarah Bosely:
http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(11)60096-2/abstract
Netdoctor proclaimed: “Exercise and talking therapies may help up to 60 per cent of patients with chronic fatigue syndrome (CFS) or myalgic encephalomyelitis (ME), a major new trial suggests….Study co-author Professor Trudie Chalder, from King’s College, London, said: ‘It is very encouraging that we have found not one but two treatments that are similarly helpful to patients’ ”.
Sky News claimed: “Talking And Exercise Could Cure ME – The chronic fatigue syndrome ME could be reversed with counselling and exercise, according to new research”.
The Mail Online was no less clear: “Got ME? Fatigued patients who go out and exercise have best hope of recovery, finds study….scientists have found encouraging people with ME to push themselves to their limits gives best hope of recovery”.
The Independent reported: “Got ME? Just get out and exercise, say scientists”.
Boots WebMD said: “ME/CFS: Pacing yourself isn’t the answer”.
Nursing Times headlines said: “CBT and exercise challenge ‘no cure’ for ME”.
BBC News said: “Brain and body training treats ME, UK study says”.
The Edinburgh Evening News reported: “It was heralded as the greatest ever study into a condition that has always defied medics….The conclusion…seemed remarkably simple. ME patients should increase their exercise through a scheme called Graded Exercise Therapy (GET) and go through Cognitive Behaviour Therapy (CBT), a procedure that would teach them to take a fresh mental approach to their lives….The magnitude of the research seems enough to influence health policy in Scotland. If implemented, it means patients would be placed on recovery schemes, with thousands benefiting from a new regime of exercise and a ‘positive mental attitude’….The fact the Department of Work and Pensions helped fund the work is also raising suspicion about what could happen to those unwilling to participate….Could they be stripped of their benefits?”.
For the Investigators not to have challenged – indeed to have been responsible for -- such wide mis-reporting of the PACE Trial’s actual findings must surely border on scientific misconduct.
The PACE Trial took 8 years to complete; the Investigators were granted extra time and extra funding, and even though the published data appear to have been selectively reported, the best that the Investigators can demonstrate is a modest improvement in a proportion of participants.
It seems that, before the data analysis was performed, Professor White was aware that SF-36 scores do not correlate with objective improvement in fatigue and physical functioning as measured by actigraphy monitors and that the Investigators were cognisant of the existing evidence that subjective reports of improvement do not necessarily match objective measures, yet the one truly objective measure of intervention effect was abandoned.
It is notable that in their article and in all their post-publication interviews, the PACE Trial Investigators made no comment about their behavioural model of “CFS/ME” that failed to respond to behavioural interventions, whereas it may be thought that an open-minded scientist would at least have considered the possibility that the model was wrong and would have been concerned to advance the field for the benefit of patients, which should be the primary aim of all clinical trials.
It cannot be disputed that, from the data that has been published, the extremely expensive PACE Trial failed to result in recovery and apparently did not succeed in returning participants to gainful employment (the data documenting those who returned to employment is yet to be published).
Given the current economic climate, from the perspective of any health economist not blindly wedded to the notion of the psychosocial model, it is clear that standard medical care is better value for money and almost as successful as the PIs’ favoured interventions of CBT and GET for people with “CFS/ME” (and much less costly to the nation’s purse).
Part B: Concerns about alleged failure of The Lancet’s editorial process to fulfil ethical duties with regard to the preparation and publication of the PACE Trial articles
On 17th May 2011 in an email to Professor Hooper, Zoe Mullan, Senior Editor at The Lancet, stated “Your complaint and (the authors’) response were discussed at the highest management level and this group of executive editors was fully satisfied that there were no grounds whatsoever on which to take further action…From an editorial perspective, the case is now closed. If you believe the editors have acted inappropriately, you could approach our independent ombudsman”.
The Lancet’s ombudsman is Professor Charles Warlow who, as confirmed by a Lancet executive editor, is conflicted because he has previously co-authored on “symptoms unexplained by organic disease” with Professor Michael Sharpe, one of the co-authors of the PACE Trial article (Brain 2009:132:2878-2888: Pt 10), and another member of The Lancet staff has stated that they have no-one else who could investigate the matter, so serious concerns remain unaddressed.
A Lancet Editorial on 17th May 2011 implied that there had been an ad hominem attack on the Principal Investigators (PIs), stating: “White and colleagues have been accused of having ‘formed their opinion about the intended outcome’ before the trial began. This view is unjustified and unfair”. It is not an ad hominem attack but a statement of fact to quote the published evidence of the PACE Trial Investigators themselves about the expected outcome of the PACE Trial.
For example, participants in the CBT group were informed on five separate occasions in their own CBT Manual that they can “overcome their CFS/ME” -- ie. they can expect to be cured -- by the application of CBT; Professor Michael Sharpe has claimed that “There is evidence that psychiatric treatment can be curative” (BMB 1991:47:4:989-1005) and during the life of the PACE Trial Professor Peter White unambiguously asserted: “recovery from CFS is possible following CBT….Significant improvement following CBT is probable and a full recovery is possible” (Psychother Psychosom 2007:76(3):171-176). Also, referring to the draft Full NICE Guideline 188 6.3.6.16, Professor White was categoric: “These goals should include recovery, not just exercise and activity goals”.
Equally, there is evidence that the PIs dislike pacing and do not regard it as helpful; from his published record, Professor White was unlikely to support pacing: all three PIs (Professors Peter White, Trudie Chalder and Michael Sharpe) are known to be strongly opposed to pacing (BMJ 5th January 2002:324:7; BMJ 19th January 2002:324:131). Peter White has further stated his outright opposition to pacing: “The theoretical risk of pacing is that the patient remains trapped by their symptoms in the envelope of ill-health”; that he resigned from the Chief Medical Officer’s Working Group on CFS/ME because he had a conflict of interest about pacing, and he acknowledged the support of Professor Sharpe (Postgraduate Medical Journal 2002:78:445-446).
On the basis of this published evidence, it cannot credibly be denied that the expectations of the PIs were that CBT and GET would be effective and that pacing (APT, or Adaptive Pacing Therapy) would not be successful. It is thus ill-informed to ascribe the reporting of the Investigators’ published views as an ad hominem attack on the Investigators themselves.
It is important to be aware that APT as used in the PACE Trial is a vehicle for incremental aerobic exercise and involves planning, achieving and sustaining targets. The CBT Therapists’ Manual states about APT: “Activity is therefore planned”, which indicates a structured activity regime, and the APT Therapists’ Manual lists other requirements for APT including “plan set activity in advance” (so activity must be “set activity”, not simply what the patient may be capable of doing at the time); there must be “activity analysis”; APT participants must “constantly review model, diaries and activity” and there is the requirement to “involve relatives”, which is nothing like pacing, ie. “doing what you can when you can”.
Professor Hooper does believe that The Lancet editors, including the Editor-in-Chief Richard Horton, have acted inappropriately in not addressing his and many other peoples’ evidence-based concerns.
Given what Richard Horton wrote in his editorial about the much-criticised 1996 Joint Royal Colleges’ Report on CFS (CR54) fully supporting people with ME/CFS, namely: “Psychiatry has won the day for now …. The sixteen-strong committee was top-heavy with psychiatric experts, so the emphasis on psychological causes and management is no surprise …. We believe that the report was haphazardly set-up, biased, and inconclusive, and is of little help to patients or their physicians” (“Frustrating survey of chronic fatigue”, Volume 348, Issue 9033, Page 971, 12 October 1996), Professor Hooper is troubled at Horton’s scathing editorial and immoderate comments about the same patient population on Australia’s ABC Radio National programme on 18th April 2011; at the way Horton has failed to address the very genuine concerns submitted to The Lancet and how he has so publicly aligned himself with the scientifically insupportable stance of Professor Peter White. What evidence can have persuaded Horton to adopt such a diametrically different position 15 years later?
In the broadcast on Australian radio in which Professor Michael Sharpe also participated, Richard Horton described Professor Hooper’s complaint as a “43 page diatribe” and as “this 43 page attack”
(http://www.abc.net.au/rn/healthreport/stories/2011/3192571.htm).
Amongst other things, Horton said: “We were delighted to get this trial, it was eagerly awaited….The investigators stepped back and were willing to do an experiment comparing conventional treatment for chronic fatigue (sic), cognitive behavioural therapy for example, against a treatment which was very much endorsed by parts of the patients community but very sceptically received by the more scientific community, and that was the adaptive pacing therapy….Yeah, I mean adaptive pacing therapy essentially believes that chronic fatigue (sic) is an organic disease which is not reversible by changes in behaviour, whereas cognitive behaviour therapy obviously believes that chronic fatigue (sic) is entirely reversible”.
Horton went on to say: “We have been deluged with dozens of letters raising serious objections to the conduct and interpretation of the study…Pretty much every aspect of the study you can think of has been impugned….The issue here which I still fail to understand is that nobody is claiming that chronic fatigue syndrome (sic) is an invented illness. It’s taken just as seriously as any other condition”.
Clearly Richard Horton seems to be unaware of the 25 year history of the Wessely School’s dismissal of ME/CFS as a neurological disorder and their published insistence that it is nothing but a myth and that it is a disease with no pathology and is merely an aberrant illness belief, with sufferers who “refuse to be placed into and accept the stigma of mental illness” being “the undeserving sick of our society and our health service” (these and other illustrations of the Wessely School’s published views can be seen at http://www.meactionuk.org.uk/Quotable_Quotes_Updated.pdf ).
In the broadcast, Horton then asserted that “the freedom of information requests and the legal fees that have been wracked up over the years because of these vexatious claims has added another £750,000 of taxpayers’ money to the conduct of this study”. Horton seems unable to grasp that the interventions used in the PACE Trial (CBT and GET) had already been shown to be of very limited value and in at least 50% of cases to be actively harmful, or to grasp that if those interventions were of any help, no-one – including the patients’ charities which opposed the trial -- would have objected to it.
When Norman Swan, the Australian radio interviewer, pointed out to Horton that patients’ groups were involved with the study all the way through, Horton replied: “Indeed, and I think this is where one sees a very real fracture in the patient community. One is seeing a very substantial number of patients very willing to engage in this study”. That was not the case: the Investigators had great difficulty in recruiting enough participants, so they were given more time and were granted extra funding to open another referral centre for that reason.
Out of the 3,158 patients screened for eligibility, 1,874 (59%) were excluded because they did not meet the trial’s primary eligibility criteria, including over a thousand patients (1,078, or 34%) who did not fit the Oxford entry criteria. That is a very substantial proportion of the referred patients.
Among the 1,284 potential participants remaining, 554 declined to participate.
This means that 43% of potentially eligible participants declined to be assessed any further or to go forward for randomisation to one of the PACE Trial participant groups.
In the broadcast, Professor Michael Sharpe said: “we recruited 640 patients into this trial and there wasn’t a high rate of refusal of taking part in the trial”, yet 554 people had declined to participate.
Put simply, this means that for every 100 people who were recruited to the PACE Trial, 86 people refused.
On what basis did Michael Sharpe state that there was not a high refusal rate?
Apparently oblivious of these facts, Horton continued: “one sees a fairly small, but highly organised, very vocal and very damaging group of individuals who have, I would say, actually hijacked this agenda and distorted the debate so that it actually harms the overwhelming majority of patients”.
The less than robust results of the PACE Trial support those who have been campaigning for ME to be recognised as a biomedical disorder which, as such, would be intractable to directive (as distinct from supportive) cognitive restructuring and incremental aerobic exercise, but Richard Horton has apparently overlooked the extensive biomedical evidence and seems to have backed himself into a position where he is no longer able to recognise it without loss of face.
Horton’s stance is particularly notable when compared with what he wrote in The Guardian on 7th July 2010: referring to the report by Sir Muir Russell looking into the allegations against scientists at the University of East Anglia, Horton wrote: “The allegations against climate scientists were that they perverted and corrupted the tried and tested processes and procedures of science. They supposedly manipulated data, suppressed research they didn’t like….threatened editors of scientific journals, and let their political views trump their scientific instincts. The Russell review has rejected all claims of serious scientific misconduct. But he does identify failures, evasions, misleading actions, unjustifiable delays…all of which amounts to severely sub-optimal academic practice….(Russell’s) report deserves scrutiny by all scientists, scientific organisations (such as the Royal Society), and universities that support scientific research. What Russell has identified is the beginning of a revolution in the way science is being done. If scientists don’t adapt to this revolution soon, the trust that the public and politicians put in science will be jeopardised. The credibility of science itself is at stake….Simply accepting a scientist’s assurance that data are accurate and reliable is no longer enough. Scientists will have to make their data available for independent audit….But with the advent of new critical public voices in science – the birth of the blogosphere, for example – scientists must redefine who is a legitimate critic and who isn’t. It is easy to brand the blogosphere as universally damaging and defamatory. But …while some critics do enjoy abusing scientists, others ask tough and illuminating questions, exposing important errors and elisions. These critics have an important part to play in shaping scientific debate and dialogue….Scientists need to take peer review off its pedestal. As an editor, I know that rigorous peer review is indispensable. But…peer review is not the absolute or final arbiter of scientific quality. It does not test the validity of a piece of research. It does not guarantee truth….If we treat peer review as a sacred academic cow, we will continue to let the public down again and again….Scientists should be educated to embrace this new culture of science, not fear or resist it” (http://www.guardian.co.uk/commentisfree/cif-green/2010/jul/07/climate-email-inquiry-revolution).
How does this accord with Horton’s demeaning dismissal and contemptuous “attack” on those who have made valid criticisms of the PACE Trial?
For examples of such valid criticisms, see:
Because he remains deeply concerned at what he believes is an abuse of the scientific process and its effect on people with ME/CFS, Professor Hooper here reproduces his letter of 28th March 2011 to the executive editor responsible for publishing the PACE Trial article and the accompanying Comment by Bleijenberg and Knoop:
Dear Dr XX
COMPLAINT WITH REFERENCE TO FAILURE TO FULFIL ETHICAL DUTIES WITH REGARD TO THE PREPARATION AND PUBLICATION OF ARTICLES ON “PACE” TRIAL PUBLISHED ONLINE ON 18TH FEBURARY 2011
I am writing to draw to your attention to a number of apparent failures to observe the relevant professional ethical codes in connection with the production and publication of the above article, and to seek an investigation of the matters set out in my attached report.
The report raises a number of questions concerning possible breaches of research and publication ethics in relation to the conduct, reporting, and publication of the above trial and the presentation and publication of the associated Comment.
In my opinion, the material presented in the attached report demonstrates that an in-depth, independent, evidence-based review by The Lancet editorial staff is indicated.
I ask that, in order to prevent iatrogenic harm to people with myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS), the article itself and the Comment be urgently retracted pending the outcome of such a review.
With specific reference to Elsevier Ethical Guidance policy, there would appear to be breaches of the Ethical Duties of Authors in relation to:
· Reporting Standards: Authors of reports of original research should present an accurate account of the work performed as well as an objective discussion of its significance. Underlying data should be presented accurately in the paper. Fraudulent or knowingly inaccurate statements constitute unethical behaviour and are unacceptable. Review and professional publication articles should also be accurate and objective
· Fundamental Errors in Published Works: If the editor or the publisher learn from a third party that a published work contains a significant error, it is the obligation of the author to promptly retract or correct the paper or provide evidence to the editor of the correctness of the original paper
· Disclosure and Conflicts of Interest: All authors should disclose in their manuscript any financial or other substantive conflicts of interest that might be construed to influence the result or interpretation of their manuscript
· Ethics in Publishing: Instructions to Authors: Ethics and Procedures (General): Fundamental Principles: the paper should….be placed in the context of prior and existing research
With specific regard to the review article Ethical Duties of Reviewers, I ask The Lancet to consider whether the relevant criteria have been met:
· Standards of Objectivity: Reviews should be conducted objectively
· Disclosure and Conflict of Interest: Reviewers should not consider manuscripts in which they have conflicts of interest resulting from competitive, collaborative, or other relationships or connections with any of the authors, companies or institutions connected to the paper.
Several of the requisite criteria set out in the Elsevier Publishing Ethics Resource Kit (PERK) may have been violated. In particular, there would appear to be issues with regard to:
· Research Error and Fraud (PERK 5): Fraud is publishing data or conclusions that were not generated by experiments or observations, but by data manipulation or invention. Changing the data measurements to conveniently fit the desired end result is fraud, but excluding inconvenient results is deliberate research error, which, in effect, is the same result – fraud
· Research Standards Violations (PERK 6): Research standards violations normally come to light when a referee sees that there was no informed consent on human subjects
· Undisclosed Conflicts of Interest (PERK 7): Financial relationships (such as employment, consultancies, stock ownership, honoraria, paid expert testimony) are the most easily identifiable conflicts of interest and the most likely to undermine the credibility of the journal, the authors, and of science itself. However, conflicts can occur for other reasons, such as personal relationships, academic competition, and intellectual passion
· Reviewer Bias (PERK 8): Editors should avoid selecting external peer reviewers with obvious potential conflicts of interest … Reviewers must disclose to editors any conflicts of interest that could bias their opinions of the manuscript, and they should disqualify themselves from reviewing specific manuscripts if they believe it to be appropriate. As in the case of authors, silence on the part of reviewers concerning potential conflicts may mean either that such conflicts exist that they have failed to disclose, or that conflicts do not exist. Reviewers must therefore also be asked to state explicitly whether conflicts do or do not exist.
My appraisal of these documents in relation to the published PACE Trial article strongly suggests that the Elsevier policy on retraction/replacement of published articles is pertinent:
· Article Replacement: Identification of false or inaccurate data that, if acted upon, would pose a serious health risk.
I note that the Elsevier journals group, to which the Lancet belongs, appears to set high store by publishing ethics, acknowledging that: The publication of an article in a peer-reviewed journal is an essential building block in the development of a coherent and respected network of knowledge. It is a direct reflection of the quality of the work of the authors and the institutions that support them.
I further note that all Elsevier journals are members of the Committee on Publication Ethics (COPE) which provides resources to support the investigation of and response to possible breaches in research and publication ethics.
In so far as investigation of any failure of the editorial process may fall within the remit of The Lancet Ombudsman, I bring to your attention the fact that the present incumbent, Professor Charles Warlow, is conflicted since he has previously collaborated with one of the authors of the PACE article, Professor Michael Sharpe. This has been confirmed by a Lancet editor with whom contact has been made.
I am aware that this complaint cannot be investigated unless the corresponding author and the funding institutions are informed, with which I concur.
I request that your prompt response be sent to my home address, but in the meantime, given the over-riding public interest, I reserve the right to place this full complaint in the public domain within a reasonable time.
Yours sincerely,
Malcolm Hooper
Emeritus Professor of Medicinal Chemistry
University of Sunderland
cc. Erik Engstrom, CEO, Reed Elsevier, 1-3 Strand, London WC2 5JR
For the avoidance of doubt, despite a follow-up telephone call made on 14th April 2011 to Patrick Kerr, deputy director, corporate relations and communication for the Reed Elsevier group (a £6 billion company) and Mr Kerr’s promise that he would contact Erik Engstrom’s office about the complaint, Professor Hooper has received neither an acknowledgement nor a response to his letter from Mr Engstrom.
In a compelling article responding to The Lancet’s Editorial (Patients’ Power and PACE, 17th May 2011), Susanna Agardy from Australia admirably captured the journal’s disturbing lack of impartial reporting: “It is a pity that the Lancet editorial is being used as a vehicle to express rancour and contempt for ME/CFS patients and other critics of PACE in an attempt to expose them to ridicule….The biomedical issue has been completely excluded from the authors’ world….Its authors ignore the real world experience of ME/CFS patients and the entire project is a pretence that the physical problems do not exist. The whole condition is misrepresented, with a heterogeneous sample, misleading definitions, inflated claims and the side-stepping of major symptoms. It is bizarre to claim that the critics ‘formed their opinions first, ignoring the findings of this rigorously conducted work’, and that we were recruited into a co-ordinated attack. I certainly did not need to be co-ordinated into criticism of PACE….The disconnect between my and others’ experience following exertion and the results of the CBT/GET studies instigated my research into the subject and I found the various sleights-of-hand that are involved in these studies….The whole CBT/GET issue has been manufactured and serves to distract from the real biomedical issues, at great cost to patients. In its vehement defence of the unscientific PACE study and its rejection of patient concerns The Lancet is perpetuating the distraction from the real serious issues of ME/CFS” (Co-Cure ACT: 17th May 2011).
It is worth noting that in her testimony
to the US CFS Advisory Committee meeting in May 2011, Dr Lily Chu said about
the PACE Trial: “The CDC should further review patient, researcher, and
clinician concerns about the effects of graded exercise and cognitive
behavioural therapy (GET; CBT)….The current CDC website has some precautions
about GET and CBT but there is more to the story. For example, surveys of
thousands of patients internationally over the last decade have shown that many
patients felt that GET and/or CBT significantly worsened their health and a
less publicized January 2011 Spanish GET/CBT showed deterioration in physical
function and increased pain with these treatments. (4, 5) Indeed, 89-96% of
participants in the UK PACE trial suffered “non-serious” adverse events (6,
Table 4 in the paper) but details of these events are not given and a high bar
(to my clinical eye) for “serious” deterioration was set whereby subjects had
to be continuously
functionally disabled for 4 weeks to qualify as a “serious” deterioration (7).
Past GET and CBT trials have several methodological flaws including differing
subject selection criteria, short-term follow-ups (less than one year) for a
chronic illness, poor assessment of treatment safety, and lack of objective
outcome measures that should be considered when assessing their conclusions.
(8, 9) Finally, CDC need to be cautious regarding which advisors they utilize:
some UK advisors have ties to disability insurance companies which may
influence their views. In the UK, patients have had their disability benefit
claims rejected or discontinued due to not participating in these treatments …
or are accused of medical non-compliance when they do not improve. At
the very least, advisors should be asked about any possible conflicts of
interest before serving on panels, committees, etc”
The Lancet editorial staff have made no attempt to address the points made by Professor Hooper which fall directly within their remit, such as why the plainly erroneous Comment by Bleijenberg and Knoop has not been retracted or at least corrected, nor have they attempted to assess the relative merits of the points made by Professor Hooper against the reply from Professor White.
It is profoundly disquieting that when so many significant errors and short-comings in the PACE Trial itself and in the published articles were brought to the attention of The Lancet’s senior editorial staff (including what Professor Hooper believes is a failure of the Elsevier publishing protocol), rather than address the issues, they chose to disregard them and instead attacked the motives of their correspondents.
This has been widely noted; indeed, having read a report in Lancet Oncology, one scientist is on record as stating that, in view of The Lancet’s handling of the PACE Trial fiasco, for the first time ever, he wondered if he could trust what he had read in Lancet Oncology.
By its lamentable action over the PACE Trial article, The Lancet’s reliability and reputation as a high impact factor journal of international repute cannot but have been damaged.