Klick zur deutschen Fassung (externer Link zur Homöopedia)
Criticism of criticism of homeopathy: The NHMRC review
This article about how the Homeopathy Research Institute (HRI) tries to rebuke the critics of homeopathy is a little bit different than the previous ones, as the critic on the NHMRC-Review is not covered in the FAQ section of their website, but consists of a few pages of its own (link).
This following article is an excerpt of a longer piece that is published (in German) in our Wiki on homeopathy, called Homoeopedia (link). Articles there offer in-depth information on many topics concerned with homeopathy, together with full background literature and sources.
For this English version I dropped the long and detailed description of what the NHMRC review contains, the methods applied, and the results, in order to just focus on the criticism of the review. The omitted parts are only essential for German reader who may have some problems to understand the English source documents of the NHMRC review.
—
Reception
The results of the NHMRC review fall in line with the other seven available systematic reviews about homeopathy in general, that is the ones not restricted to specific conditions. Convincing evidence that homeopathy in general or for any single indication has any effect exceeding placebo could never be identified. All authors came to much the same conclusion: the quality of the available studies is too poor to draw meaningful and definite conclusions about efficacy.
Remark: The aforementioned reviews are Kleijnen (1991), Linde (1997), Linde (1998), Cucherat (2000), Shang (2005), Mathie (2014), Mathie (2017). Especially the two reviews by Mathie are interesting: Mathie works for the Homeopathic Research Institute and for sure is not biased against homeopathy. In both his reviews he only found two (2!) studies with low risk of bias and found no indication that homeopathy is of proven efficacy. You can consider this as a truly independent replication of the NHMRC review.
But NHMRC receives harsh criticism by organisations of homeopathy, by the HRI and other individuals to such an extent as only the review of Shang (2005) had received before. But all this criticism is very perfunctory and does not reach any depth. Potentially critical statements are made very often wrongly and unwarranted, but no effort is made to illustrate in what way the claimed shortcoming could have affected the results, if they would be for real. In that sense, this is hardly a fact-based discussion. More or less the whole of the criticism seems targeted only to give a negative impression to the public, that something was crooked in the preparation of this paper.
Many of the points of criticism are brought forward by several groups and individuals, and it is hard to tell who first came up with them and who only copied and reiterated the issues. In this article, I provide just a single source to every issue raised, which should be considered as a randomly selected source out of many.
The criticism shows a considerable variety, and the claims were taken and developed further by independent groups and individuals, with different grades of familiarity with the methods of clinical trials and reviews. So we are confronted with a multitude of claims that cannot be covered completely in an article like this one. I try however to address all the major points, especially those brought forward by associations and societies of homeopaths.
Pseudocriticism
Quite a considerable portion of the claims raised by homeopaths does not concern the content of the review but argue its context and how the work was allegedly performed. The claims are put forward, but there is no description as to how and where the claimed shortcomings could have affected the research work and could have biased the outcome. This criticism only serves to shed some bad light onto the NHMRC and the way they worked, but without showing any error in the review itself.
Claim: The review was performed twice and the first version is kept secret and unpublished
The Homeopathic Research Institute (HRI) claims that there had been a prior version of the published review, which is kept secret from the public and remains unpublished. HRI wonders whether this first version was refuted for quality issues – implying that the results were in favor of homeopathy and suppressed for this reason alone.
In fact, the Australian Homeopathic Association (AHA) gives some detail on this alleged first NHMRC review version on their website (Link http://www.nhmrchomeopathy.com/firstreview.html). However, no references to support their claims are available to the public. This first version, AHA claims, was of good quality because it was performed by a specialist who was involved in editing the NHMRC standards applicable to such work. Forty-eight indication-specific reviews relevant to the scope of the NHMRC review were included (not 57 as in the final published version), which were assigned to five levels of evidence. But any hints on what the results of this work was is lacking.
Even if this ominous first version really does exist – publicly available evidence is not yet presented – it is open to speculation what the outcome might have been, and whether it was different from the published version. And there is no indication whether the published version is off the track or the alleged first one was, let alone what any shortcoming of the published work might be.
Claim: The chairman of the working group had a conflict of interest
Prof. Brooks, the chairman of the working group, HRI claims, did not disclose his conflict of interest. He is a member of an anti-homeopathy group and thus biased against homeopathy.
First off, the statement of HRI is wrong: The Homeopathy Working Committee (HWC) was not chaired by Prof. Brooks but by Prof. Glasciou of Bond University. Brooks was member of an organisation named “Friends of Science in Medicine”, which is opposing pseudo-medicine. But his membership ended after only four months due to opposing views, long before the NHMRC review was completed. Brooks did resign form chairing HWC and was one of seven members of HWC (see question 6 in the FAQ section of the NHMRC documents).
Similar claims were brought forward about possible conflicts of interest by the contractors who did the evaluation. These are companies with a long-standing relationship to NHMRC and have a clause in their contracts, that if ever a researcher should have conflicting interests, they should be disclosed to NHMRC. In the beginning, there was no declaration, because there were no conflicting interests and thus no need to declare them. But declarations were added to the documents later (see question 4 of the FAQ).
Anyway, HRI does not show what could have been the consequences of the claimed conflicting interests. Every step NHMRC and their contractors took to get to the final result is exceptionally well documented, including criteria, guidelines and explanations of the decisions. The critics did not point to a single item which might have been influenced by any bias of the persons involved and how this could have affected the result of the review.
Claim: In contrast to NHMRC’s own regulations, there was no homeopath present in the HWC
This is a standard argument raised whenever scientific work reached an outcome not in favour of homeopathy. But it is true: there was no homeopath included. Scope and object of this review was to evaluate the available evidence and whether there was any proof of efficacy of homeopathy in any clinical condition. The Homeopathy Working Committee (HWC) controlled the work and authored the final report. Evaluation was performed by contractors commissioned to do the job (OPTUM, ARCH). It is unclear what would have been the difference if a homeopath would have joined the HWC or how the lack of special knowledge on homeopathy could have had any impact on the result.
NHMRC’s internal guidelines, according to which a specialist of the therapy considered would have to join the team, concern the preparation of guidelines where recommendations for therapists are being developed from the data. This clearly requires some expert knowledge of the therapy in question. But this is not necessary for a review with the only purpose to check and analyse the available evidence without a deduction of guidelines (see question 3 of the FAQ).
Claim: This is not a meta-analysis but a mere literature search
The German “Wissenschaftliche Gesellschaft zur Homöopathie” (“Scientific Society of Homeopathy”, WissHom) claims, that the NHMRC review is not a real meta-analysis but merely the result of a literature search alone. They are right of course, the NHMRC review is not a meta-analysis, because this would be the pooling of the results of studies using mathematical and statistical methods.
But still this claim completely misses the mark. As it is quite obvious to see in the very extensive documentation, the overview report and appendices, NHMRC not only performed a very extensive literature search but also reviewed the papers found – together with papers found by public consultations or submitted by stakeholders – very thoroughly, and they documented their approach and their findings in the sizeable paperwork of nearly 1,000 pages.
Claim: NHMRC did not consider more than 1,800 Studies
The German Society of Homeopathic Doctors (“Deutscher Zentralverein homöopathischer Ärzte”) claims that this review is not based on the findings of more than 1,800 studies, but only includes 176.
This claim is wrong in quite a lot of ways. In fact, NHMRC do state in their press release that their conclusions are based on the rigorous assessment of more than 1,800 publications, but right in the next sentence, they disclose that only 225 studies met the criteria for inclusion into the review on the efficacy of homeopathy by the NHMRC. These two statements are required by the CONSORT-statement to allow future analysis and replication – and they are absolutely correct.
- The three steps of their literature search resulted in 1,863 articles and papers that contained the search phrases of interest (“homeopathy” etc.). This included multiple occurrences, letters to editors, press releases of organizations, basic research, in-vitro research etc. (There were 1,367 papers from the original literature search, 343 nominations of interested parties and 153 nominations from the public consultation).
- 57 systematic reviews with extractable data included 176 studies, the nominations of stakeholders yielded 9 additional studies not yet included, and submissions from public consultation brought another 40 studies, 225 studies in total.
However, the media frequently cited in error that more than 1,800 studies were included in the final evaluation but this seems less a problem of NHMRC but more of the ignorance of some journalists to not cite the following sentence. It is open to discussion how NHMRC could have been even clearer in their statement.
Criticism of methods
This chapter concerns criticism in the method that would indeed have affected the outcome if it was true. But this criticism is based on an astonishing amount on wrong citations and claims.
Claim: Studies with less than 150 participants were excluded from the review
It is not only the German Society of Homeopathic Doctors that claims that studies with less than 150 participants were excluded from evaluation and thus the vast majority of available data was not included in the findings. This is one of the most frequently-heard arguments against the NHMRC review. But this claim is completely wrong as can easily be seen. NHMRC defined a threshold to consider studies and systematic reviews of less than 150 participants as “small” and less than 50 participants as “very small”. Small and very small studies were not considered reliable evidence, however they were not excluded from the review right away. Instead they were reviewed, and their contents were documented. Evidence for a given clinical indication that only consisted of small or very small studies was not considered reliable. If, however, there had been a systematic review that rated these studies as of good quality, they would have been included in the NHMRC review – but such cases were not found. For more on this, see the next point below.
Claim: The cut-off value of 150 participants was not explained
NHMRC does not give any reason why studies with less participants than 150 were considered unreliable, the German Association of Homeopathic Doctors claims. NHMRC referenced a paper published in BMJ, the former British Medical Journal, one of the leading journals in medicine, but this article is not applicable because it only discusses dichotomous data. However, most of the results in the studies are continuous data. In addition, NHMRC themselves conducts studies with less than 150 participants, which would be useless according to this standard. As such is the claim.
It is true, NHMRC does not explain or justify the cut-off at 150 participants for a reliable study, and it is very likely that it is hard to find any reason based on science. This is similar to the “paradox of the heap”: What is the threshold for a number of sand grains to become a heap of sand? Somewhere has to be such a limit, but for each and every cut-off it can be argued whether one grain more or less will change the situation between a heap and a mere number of grains. Same here. Does one participant less turn a reliable study to unreliable? This question can be raised at any threshold value.
The authors of the BMJ article examined the impact of the number of participants on effect size. They found that studies with less than 50 participants result in twice the effect size compared to studies with more than 50 participants. By interpolation of the data given in fig 2. of the BMJ article, one can estimate that effect size is increased by 30% when studies with less than 150 participants are compared to studies with more participants. This may be a sound reasoning, but it is true, this BMJ article only deals with dichotomous data, where all the patients were assigned one of (at least) two properties which are mutually exclusive to each other.
For continuous data – where the result for any patient can achieve any value within certain limits –, the GRADE handbook recommends that with less than 400 participants a downgrade of evidence to “not reliable” should be considered. The threshold of 150 for the mostly continuous data of homeopathic trials proves reasonable and obliging.
This claim may and will be discussed for a long time to come. But what the critics of the NHMRC review did not provide until now is the impact this limit has or hasn’t on the result. All the other systematic reviews of homeopathy came to pretty much the same result without limiting the sample size of the studies, just by considering quality: namely that there is no evidence from which a conclusion can be derived that homeopathy is more effective than placebo. In the NHMRC review this additional criterion of number of participants was introduced, but it seems to be of no real impact. In the documents of the NHMRC, the assessment of all data and the conclusions drawn thereof are given for all clinical indications in great detail. It should be easy for any critic to identify any indication where the rating of the evidence depended on the low rating of small studies alone. This would be the most convincing of all arguments about size of studies – but this has not been presented by any critic yet.
Claim: All studies published in a language other than English were excluded
The Austrian homeopath Friedrich Dellmour – by his own account the “science expert” of the Austrian Society for Homeopathic Medicine (“Österreichische Gesellschaft für homöopathische Medizin”, ÖGHM) – claims on his website that all studies were excluded that were not published in English. This claim is wrong, at least in this generality. Indeed, at first there were some studies excluded because their publication language was not English, but most of these studies were included indirectly.
Search of literature: At an early stage of the search of literature, 10 papers were excluded due to their language not being English, but only in one case a study is concerned that otherwise may have met the inclusion criteria (see overview report, chapter 7.1.3), namely the paper by Lüdtke and Wiesenauer about the treatment of hay fever with Galphimia glauca. But the individual studies contained in this very review were included in the review by Linde (1997), and this one was included by NHMRC.
Individual studies that could be analysed indirectly because they were discussed in a review published in English and identified in the search for literature were included regardless of language: More than 40 individual studies published in French, German, Italian, Portuguese, Spanish and Norwegian were taken into account.
Individual studies: In the literature identified in the public consultation or nominated by the stakeholders, 10 papers were not considered for their language of publication.
Five of these “omitted” studies were submitted in the public consultation: two of them (published in Spanish) contained clinical trials, a third was a pilot study and did not meet the inclusion criteria, the fourth did not report any benefit of homeopathy over placebo, and the last one (again) was one of the studies included via the review by Linde (1997).
During public consultation, 334 submitted papers were not considered, amongst them five that were excluced for their language. However, two of those were dealt with already.
This leaves a total of five studies that might have met all inclusion criteria except publication language and might contain reliable evidence for homeopathy. But compared to the number of 225 studies that were actually considered, five excluded ones look quite small, and the impact on the overall findings seems negligible.
Claim: Jadad Score 5 out of 5 is an unusually tough requirement
Another claim by Dellmour is that a quality rating of five points on the Jadad-score – or equivalent – was required to consider a study reliable, and thus too many studies were rated unreliable. But this claim is wrong as well.
The Jadad score is a somewhat outdated rating scale for study quality. It was replaced and extended by the currently applied assessment of the “risk of bias”, where there are three ratings of “low”, “unclear” or “high” risk of bias. The overall quality of a study is rated according to the worst rating in one of six different domains. Studies of good quality (that is low risk of bias) thus meet all the requirements and would have given five out of five points on the Jadad score. This method is applied in the HRI-initiated reviews Mathie 2014 and 2017.
That the quality of the trials of homeopathy does not meet the requirements of high standards is a feature that all systematic reviews of homeopathy complain about and thus conclude that no conclusions as to the efficacy of homeopathy can be drawn. So NHMRC does not use an uncommonly rough criterion for quality. That the evidence for homeopathy does not meet satisfying standards of quality is a problem of the evidence, not of the application of any rating scheme.
Claim: Homeopathic requirements of quality were not observed
The German Society of Homeopathic Doctors claims in their criticism that the homeopathic criteria for quality were not observed. However, they do not disclose the nature of these requirements. This would be needed, because there is no other systematic review about homeopathy as of yet where special requirements were introduced that should have been observed in study selection, assessment, or evaluation.
Claim: Not a single randomized controlled trial was assessed
The Austrian homeopath Friedrich Dellmour complains that not a single double-blinded, controlled study was considered, and that the team relied on second-hand data alone. But this complaint is wrong again.
Indeed, the method used by NHMRC to evaluate the evidence is somewhat unusual. The original studies included in the systematic reviews that were identified by the literature search, were assessed by way of the information contained in all the reviews where they were included. But again, the critics do not point out how this could have any negative impact on the findings. The research question was whether there is any clinical condition where the use of homeopathy could be of advantage, but not to derive new insights into this therapy. For the first question it is sensible to include all the findings of all the researchers that worked on this issue before, which in all probability gives a less-biased picture than if there had been a necessarily small group of individuals to redo this work. After all, there are reviews performed by persons of varying attitude towards homeopathy, from the alleged critic (Shang) to the supporter (Mathie). Advantages and shortcomings of this approach are discussed in the information paper pp 25f.
By the way: Of the individual studies submitted by the public and the stakeholders, 49 papers – more than a quarter of all studies –, were of course assessed directly by the commissioned contractors.
Criticism of quality and publication
Claim: There was no peer review
The German Scientific Society for Homeopathy claims that the NHMRC review was not published in a journal with peer review, which is supposed to imply that the steps to ensure quality were not sufficient and that the review is lacking quality.
Without any doubt, this review was not published in a journal. This would have been impossible because at almost 1,000 pages it would have exceeded the scope of any journal. However, the implication is wrong, because this review was subject to a more rigorous quality check than any article in any journal.
-
All decisions and ratings subjected to guidelines and instructions are documented in the various papers.
- The method was evaluated before the work begun by an independent organization (The Australasian Cochrane Centre)
- The review was itself reviewed by three experts of CAM, and their comments are included in the documentation.
- In April 2014, the work was presented for public consultation where additional evidence could be submitted, as well as comments on the method.
- Finally, the review was approved by the NHRMC Council.
All this can be found in the information paper.
Criticism on the result
Claim: There are five clinical conditions where reliable evidence exists
HRI claims that there are five clinical conditions where “good quality, well designed studies with enough participants for a meaningful result” exist, and that NHMRC has misled the public by dismissing this evidence. HRI names the conditions as (1) hay fever, (2) sinusitis, (3) upper respiratory tract infections, (4) diarrhea in children, and (5) lower back pain. If such evidence was for real, this would indeed indicate that the NHMRC review – for whatever reason – does not give a true picture of the current situation. So let’s briefly check the evidence for these five conditions.
(1) Hay fever (allergic rhinitis)
NHMRC extracted a total of 13 placebo-controlled trials (PCTs) in the systematic reviews identified by their literature search, together with two observational studies that had control groups. To do a detailed review of all these papers would exceed the limits of this article, but instead we can compare the findings of NHMRC to those of Linde (1997) and Mathie (2017).
Based on 13 PCTs with a total of 1,436 participants, and two observational studies with together 248 Patients, NHMRC concludes that there is but one good study of almost sufficient size (n=144) that resulted in a positive conclusion. But this study was never replicated, and all the other papers considered contradict the result. This one good study does not significantly change the overall picture, and NHMRC came to the conclusion that there is not enough evidence for the efficacy exceeding placebo.
Six of the papers considered by the NHMRC were included in the review by Linde et al. from 1997, and there was an additional paper not found by the NHMRC. Linde concluded that there is no clinical condition where any efficacy exceeding placebo could be recognized – which includes his findings on hay fever with a total of 899 patients.
Quite similar the review by Mathie et al. from 2017: This review includes eight of the 13 papers considered by NHMRC, including five that have already been reviewed by Linde before, and two additional controlled studies, with a total of 1,203 participants across all studies. In his review, Mathie performed indication-specific meta-analyses and could not identify any reliable evidence for effectiveness beyond placebo – including for hay fever.
Conclusion: The NHMRC results fall in line with the results of other systematic reviews – both held in high esteem by homeopaths – and the statement of HRI that there is good evidence for homeopathic treatment of hay fever is unproven.
(2) Sinusitis
The systematic reviews found in the literature search yielded three individual studies for sinusitis (Zabalotnyi (2007), Weiser (1994), Wiesenauer (1989)) which were considered as not reliable. The submissions yielded a fourth study (Friese (2007)) which was rated “high risk of bias” for the big difference in attrition in the two experimental groups (verum 2%, control 88%) which result in a biased result in any case.
NHMRC concludes that there is no reliable evidence for the treatment of this condition.
The German-based Carstens-Foundation (Karl und Veronika Carstens Stiftung, KVC) operates a comprehensive database of clinical trials of homeopathy, the COREHom database. For sinusitis there is only one additional paper listed (Ricciotta (2005)), which may have matched the inclusion criteria as indicated by the data in COREHom. This paper showed positive findings but had 38 participants only.
The four papers of Zabalotnyi, Friese, Weiser and Wiesenauer were included in Mathie’s 2017 review and were rated as not reliable as well. The Zabalotnyi paper was rated “unclear risk of bias”, all the others were rated “high risk of bias”. Mathie found not a single clinical condition where a meta-analysis yielded evidence for efficacy of homeopathy – which includes this condition.
Conclusion: NHMRC came to the same conclusion as Mathie did later on, with the usual methodology of systematic reviews, and as someone affiliated with the Homeopathic Research Institute and therefore certainly not disposed against homeopathy. The claim by HRI that there is reliable evidence for homeopathy in sinusitis is unwarranted.
(3) Upper respiratory tract infection (URTI)
For this indication, NHMRC extracted seven individual studies from the systematic reviews they identified by their literature search: four controlled studies (Steinsbekk (2007), Steinsbekk (2005), DeLange (1994), and Lecocq (1985)) and three observational studies (Haidvogl (2007), Rabe (2004), and Riley (2001)). None of the PCTS showed any significant benefit of homeopathy. The observational studies showed that homeopathy was not inferior to conventional therapy including antibiotics.
In the literature submitted by third parties there were another two observational studies (Zanasi (2014), Steinsbekk (2005_b)). The paper of Steinbekk was rated “high risk of bias” for a lack of blinding in combination with the subjective main outcome. The paper of Zanasi was rated “low risk of bias”, however it was downgraded due to the low number of participants.
Checking the data included in the NHMRC review for completeness with the COREHom database proves difficult. On one hand, studies dealing with influenza and influenza-like syndromes are listed for the same keywords as URTI, and on the other hand NHMRC included some studies not listed in COREHom. So it is hard to evaluate if NHMRC’s database is complete.
Only a few of these studies were considered by other reviews: in the one by Mathie (2014), the Steinsbekk (2005) paper is rated “high risk of bias” and Delange (2004) as “unclear risk of bias”. Linde (1997) rated the paper by Lacocq as poor (Jadad-score 2 of 5, internal validity 50 of 100).
In summary, it is difficult to check the facts here. There are six PCTs where only one came to positive results – though not in all the criteria by far –, while the great majority of studies did not, however. Observational studies alone, even with control group, cannot build solid evidence, as due to the lacking randomization co-founding factors are very hard to control. In addition, the impact of interventions for selflimiting conditions like viral infections is vastly dependent on the setting of the study, for instance the point in time when the observation takes place. If you check for the outcome two weeks after any intervention, the common cold is gone, independent of what the intervention may have achieved.
(4) Childhood diarrhoea
NHMRC concluded that there is one reliable study for complex homeopathy in childhood diarrhoea (Jacobs 2006), but this study did not show any effect of homeopathy. No other of the three studies of individualized homeopathy in childhood diarrhoea (Jacobs 1993, 1994 and 2000) was considered reliable.
The COREHom database contains only one additional study for this condition (Cadena 1991), but according to the database, this paper does not include any statistical evaluation.
In his 2014 review on individualised homeopathy, Mathie rated the three papers by Jacobs (1993, 1994, 2000) – like NHMRC – as B or C, that is with unclear or high risk of bias. In the 2017 review of non-individualized homeopathy, Mathie rated Jacobs (2008) much worse than NHMRC.
The review by Altunc on childhood and adolescent ailments, including childhood diarrhoea and the Jacobs papers, did not yield any hints for reliable evidence in any condition.
Conclusion: The result of NHMRC is in line with the available reviews. The claim of HRI, that there is positive evidence for homeopathy in childhood diarrhoea, is unjustified.
(5) Lower back pain
For this indication only sparse evidence is available. NHMRC found only one PCT (Stam 2001) which was rated of good quality and with 161 participants of sufficient size to yield reliable evidence. In this study, the efficacy of a preparation named “Spiroflor SRL” was tested against conventional therapy. The authors found the drug of same efficacy but of better tolerability as the control remedy (Cremor Capsict Compositus). The manufacturer advertises Spiroflor SRL as a homeopathic remedy. However, the content per 10 g is given as “Rhus toxidendron tincture 5 g, Ledum palustre tincture 5 g”. Except for the name on the label, there is nothing to indicate that this product is actually homeopathic. Even if this study came to positive findings, it is doubtful whether it could be considered evidence for homeopathy.
In the submissions there was another study by Pach (2011) where a complex remedy was tested against placebo and no treatment. The study was rated “low risk of bias”, but with 150 participants across three groups it was too small. There were significant differences to no treatment – but not to placebo.
The COREHom database lists another three papers (Gmünder (2002), Beer (2012), and Morris (2016)), the last one published after the NHMRC review. In the paper of Gmünder, homeopathy is compared to physiotherapy, though in a pilot study with a limited number of participants (n = 43). In this study, homeopathy showed significantly better results than the alternative treatment. Strange however, that this remarkable study, which would be of considerable improvement for patients – just swallow a few sugar pills instead of seeing a physiotherapist on a regular basis – is still not replicated after 18 years (as of March 2018). There is no indication that these formidable results can be reproduced.
In the paper of Beer, which was published in German and was therefore excluded from the NHMRC review, the evaluation of the randomised patients did not yield a significant benefit of homeopathy compared to placebo.
Conclusion: The available evidence does not indicate that a homeopathic treatment could be beneficial to lower back pain.
In a nutshell: The results of NHMRC concerning the five indications for which the HRO claims positive evidence for homeopathy exists, are completely in line with the available evidence as evaluated in other reviews, especially with the reviews of Mathie from 2014 and 2017.
Claim: The searches in the databases were selective and not sufficient for CAM
Dellmour, the science specialist of the Austrian Society of Homeopathy claims that the literature search method used for the medical databases was selective and not sufficient for Complementary and Alternative Medicine (CAM). This statement is wrong. As can be seen by the research question (“Is there any medical condition where homeopathy can be useful?”), the scope of this review was not CAM in total but just homeopathy. The literature search and the submissions combined yielded 225 individual studies, both controlled studies and observational studies with control groups. The British Faculty of Homeopathy indicates on their website that at the end of 2014, a total of 189 randomised controlled studies on homeopathy existed. Based on this figure, NHMRC included the available data quite completely.
Claim: The evaluation of efficacy must be performed systematically and separately for each condition
Dellmour complains that homeopathy was reviewed completely, that is including all conditions at once, which allegedly cannot yield any meaningful result. Instead, homeopathy should be reviewed by condition. However, this is exactly how NHMRC approached the question: The evidence was checked condition by condition, the conclusions drawn and only summarized at the very end. However, Dellmour often cites the reviews of Mathie 2014 and 2017 which were indeed conducted with a different approach, that is the one Dellmour complains about.
Summing up the criticism
The criticism of the NHMRC review is very extensive and concerns many features of the research environment, the methodology, the conduct of the work. and finally the – in homeopaths’ view – unpleasant result. But these very engaged public discussions are fruitless: NHMRC just reaches the same conclusions as Mathie as an affiliate to HRI did in his 2014 and 2017 reviews.
In both reviews, Mathie considered a total of 107 individual studies and only found two of high quality (“low risk of bias”), which could be considered as reliable evidence, and these two did not show any benefit for homeopathy. Mathie upgraded a further four studies to be of reliable evidence despite their unclear risk of bias, but this is in violation of the usual procedure, of Mathie’s own protocol, and of the recommendations of the Cochrane Collaboration.
The critics on the NHMRC review did not come up with any sound and well-warranted refutation. They could not produce any indication where there was reliable evidence that homeopathy is more effective than placebo. This is important because Mathie’s work apparently did not show the alleged shortcomings of the NHMRC:
-
As Mathie is an affiliate to HRI and worked with authors that mainly work for organizations that bear “homeopathy” in their names, any bias against homeopathy can be excluded.
- Mathie performed classic reviews and even distinguished between different flavours of homeopathy, namely individualized vs. non-individualized homeopathy.
- Mathie did not exclude any study for its sample size.
And still he came to the same conclusions as NHMRC did.
This can be considered as the best independent confirmation of the NHMRC review there can be. So we can conclude that it’s a proven fact, that as of today there is no evidence available that homeopathy is more than placebo in any clinical condition. The claim that homeopathy is a therapy comparable if not superior to conventional medicine is unwarranted.
