More information about text formats
Randomized controlled trials (RCTs) are considered to be the best method for evaluating the effectiveness of medical interventions.1 Despite their strengths, RCTs have substantial limitations.1 Although RCTs have strong internal validity, they occasionally lack external validity and generalizations of findings outside the study population may be invalid. More specifically in retinal surgery, there are many obstacles to conducting RCTs to address the specific questions asked, so the analysis using real-world data is useful.2 Drs Anguita and Charteris wrote an editorial in the British Journal of Ophthalmology (BJO) on the merits and limitations of studies using real-world data.3 They cited our papers that were recently published in BJO which used the data collected in the Japan Retinal Detachment Registry (J-RD registry), and I would like to comment on with a focus on the retinal surgery.4,5
As correctly stated by Drs Anguita and Charteris, studies using the propensity score matching method cannot be performed well if one is not familiar with the limitations of this technique. 3 However, this is also true for those who do not have a deep understanding of the disease and may make incorrect interpretations. This would be the case for our paper4 cited in the editorial. This study compared pars plana vitrectomy (PPV) and scleral buckling for superior RD without macula detachment using the data from the J-RD registry. The results which were analyzed using propensity score...
As correctly stated by Drs Anguita and Charteris, studies using the propensity score matching method cannot be performed well if one is not familiar with the limitations of this technique. 3 However, this is also true for those who do not have a deep understanding of the disease and may make incorrect interpretations. This would be the case for our paper4 cited in the editorial. This study compared pars plana vitrectomy (PPV) and scleral buckling for superior RD without macula detachment using the data from the J-RD registry. The results which were analyzed using propensity score matching showed that there was no significant difference in the best-corrected visual acuity at 6 months after surgery, but there were significantly fewer surgical failures with scleral buckling than with PPV. Thus, we concluded that, “Although the indications for PPV are becoming broader, PPV may not be the optimal approach for repairing all types of RDs. Therefore, careful considerations are needed when selecting the appropriate surgical technique in treating uncomplicated phakic macula-on RD case”, knowing the limitation of evidence level obtained from real-world data study.4 The editorial by Drs Anguita and Charteris indicated that a major problem with this study was the lack of adjustments for the presence or absence of a posterior vitreous detachment (PVD) which is the most important factor in selecting the surgical method.3 Traditionally, the presence of a PVD has been determined by echography, but its accuracy is inferior to that of optical coherence tomography, and above all, it was found to vary from operator to operator.6 On the other hand, it is widely accepted that almost all cases of retinal tears are caused by a PVD,7 so we decided it would be more objective to adjust the evaluations of PVD by using retinal tear or hole instead. We believed that this analysis, in which preoperative factors were adjusted for retinal tear and/or hole, adjusted for PVD to an acceptable level. Without understanding this background, the findings of this paper might be misinterpreted. On the other hand, we mention in the paper by Funatsu et al, which was also cited in the editorial, on the potential toxic effects of silicone.5 There was a misunderstanding of the intent of our study, however because of space limitation, we will not discuss it here.
We agree that the analysis of real-world data using propensity score matching has its limitations.3 However, there are major problems in implementing RCTs in retinal surgery. First, RCTs are very costly, and the overall cost of an RCT study has skyrocketed to a level that cannot be borne by the surgeons or researchers. In recent years, RCTs are no longer conducted unless they are sponsored by large pharmaceutical companies that can profit from the results of RCTs.8 Additionally, if the drug is not effective, it will not necessarily be published.8 Studies in which the company's profit is not clear, such as retinal surgical treatments, are less likely to be adopted as a topic of study. Second, RCT is time-consuming. It usually takes only a few weeks to complete a study using registry data, but it generally takes years to complete RCT studies from planning, implementation, and analyzation. If prospective RCTs were performed for comparing PPV and scleral buckling for superior RD as in our study, it would have taken several years to accomplish the project. Furthermore, it has been noted that RCTs are virtually impossible to perform to compare existing and new surgical methods because surgeons’ preferences already exist and enrollment does not work.9 In general, surgeons want to know how to save the patient in front of them, often an individual problem, as soon as possible. Not only is it extremely difficult to recruit patients for an RCT who meet the inclusion criteria of individual problem of retinal surgery, and it can take several years to obtain the results.9 Thus, it is not practical to use an RCT for this purpose.
The editorial by Drs Anguita and Charteris is very important and I congratulate that. As they stated, we do not believe that the results obtained from real-world data analysis can replace the evidence of RCTs, either. On the other hand, it is true that RCTs cannot answer all of the surgical questions. Most importantly, RCTs are essentially experimental trials of humans. It is unclear whether it will continue to be ethically acceptable to put a large number of subjects at risk even for medical purposes. In contrast, real-world data analyses are basically retrospective studies so it does not expose patients to any new risks. Until better analysis methods are developed, real-world data analysis will provide certain answers to many problems which surgeons have. Nevertheless, I fully agree with them that the researchers and the readers need to recognize the validity and limitations of propensity score matching studies as well as to know the background of the treatment.
1. Frieden TR. Evidence for Health Decision Making - Beyond Randomized, Controlled Trials. N Engl J Med 2017;377:465-75.
2. Ryan EH, Ryan CM, Forbes NJ, et al. Primary Retinal Detachment Outcomes Study Report Number 2: Phakic Retinal Detachment Outcomes. Ophthalmology 2020;127:1077-85.
3. Anguita R, Charteris D. Could real-world data replace evidence from clinical trials in surgical retinal conditions? Br J Ophthalmol 2022:bjophthalmol-2022-321759. doi: 10.1136/bjophthalmol-2022-321759. Epub ahead of print. PMID: 35580995.
4. Kawano S, Imai T, Sakamoto T; Japan-Retinal Detachment Registry Group. Scleral buckling versus pars plana vitrectomy in simple phakic macula-on retinal detachment: a propensity score-matched, registry-based study. Br J Ophthalmol 2022:857-62.
5. Funatsu R, Terasaki H, Koriyama C, et al. Silicone oil versus gas tamponade for primary rhegmatogenous retinal detachment treated successfully with a propensity score analysis: Japan Retinal Detachment Registry. Br J Ophthalmol 2021:bjophthalmol-2021-319876. doi: 10.1136/bjophthalmol-2021-319876. Epub ahead of print. PMID: 34373251.
6. Moon SY, Park SP, Kim YK. Evaluation of posterior vitreous detachment using ultrasonography and optical coherence tomography. Acta Ophthalmol 2020;98:e29-e35.
7. Michaels RG, Wilkinson CP, Rice TA. Vitreoretinal precursors of retinal detachment. In Retinal detachment, eds Michaels RG, Wilkinson CP, Rice TA, The CV Mosby Company, St Louis, 1990, pp 29-100.
8. Flacco ME, Manzoli L, Boccia S, et al. Head-to-head randomized trials are mostly industry sponsored and almost always favor the industry sponsor. J Clin Epidemiol 2015;68:811-20.
9. Lonjon G, Boutron I, Trinquart L, et al. Comparison of treatment effect estimates from prospective nonrandomized studies with propensity score analysis and randomized controlled trials of surgical procedures. Ann Surg 2014;259:18–25.