Re:Lack of scientific rationale for use of valproic acid for retinitis pigmentosa
Authors' Reply to Sandberg et al. letter entitled "Lack of scientific rationale for use of valproic acid for retinitis pigmentosa"
Radouil Tzekov 1, Carol Bigelow2, Christine Clemson1, Jenna Checchi1, Mark Krebs3, Shalesh Kaushal1
1 Department of Ophthalmology University of Massachusetts Medical School Worcester, MA, USA
2 Division of Biostatistics and Epidemiology/Department of Public Health University of Massachusetts School of Public Health and Health Sciences, Amherst, MA, USA
3 Department of Molecular Genetics and Microbiology, University of Florida Gainesville, FL, USA
Correspondence to: Dr. Shalesh Kaushal Shalesh.Kaushal@umassmemorial.org
Dear Editor, We appreciate the concerns raised by Sandberg et al. in their recent letter1 pertaining to the design and statistical methods of our analysis described in our recent article,2 and we welcome the opportunity to respond to each point raised.
Study Design. We agree that ideally the best, "gold standard" study design would involve the use of matched controls, with matching on eye function at baseline, as we plan to do in our upcoming clinical trial. For preliminary pilot studies, however, other study designs, including methodologies that do not have controls, can also be meaningful, especially when the objectives are to assess treatment potential and establish the equipoise necessary for further investigation in a randomized control study. An example that is relevant to our work is presented by an article from one of the authors of the letter (Rosner et al., 2006; Section 4.1),3 in which the investigators assessed the potential benefit of treatment and concluded that their findings were suggestive of a lack of effect due to treatment. These researchers did not have a control group, yet they felt comfortable with the conclusions based on this design. Our analytical design and findings are consistent with this example. Our focus was an exploration of potential therapeutic value, and our detailed description of visual function in 14 eyes establishes a possible treatment benefit that merits a randomized control trial.
Statistical analysis. We believe that the exploration of treatment potential in a sample size of seven is best addressed with detailed descriptions rather than formal tests of statistical significance. Given that significance levels were reported, however, we agree with Sandberg et al. that the unit of analysis should have taken into account the correlation structure of the data. We thank Dr. Rosner for pointing out the modification of the Wilcoxon Signed Rank Test for use with paired data. We expect to use this modification in future studies that have formal tests of significance as their goal. With respect to our pilot analysis, with this correction, calculated by Dr. Rosner, the statistical significance of the improvements in visual field and visual acuity (p= 0.14 and p=0.06, respectively) no longer meets widely used thresholds (such as p < 0.05), as the Sandberg et al. letter points out. We reason that these p-values, as they pertain to a small sample size setting, do not provide conclusive evidence of the null hypothesis, as the associated confidence intervals of the differences are wide. Accordingly, while we could calculate a corrected analysis of statistical significance, we believe the relevance of our findings to the advancement of treatment for retinitis pigmentosa would be the same. We would also like to note that the calculated p-values for improvement in visual acuity and visual field have been compared to a hypothesis for no change in function. We would expect a comparison to historical controls that experience on average some deterioration in function to have associated p-values that are closer to p <0.05. We observed a potential for treatment benefit that was accompanied by at most modest and tolerable side effects. This is of public health importance as our data contribute to a growing body of literature that collectively suggests the appropriateness of a larger scale study of valproic acid therapy that utilizes randomization and comparisons with controls.
Floor effects. We appreciate the issue of floor effects. In this regard, we wish to note that in our sample, only 1 of the 14 eyes studied exhibited visual acuity of less than 20/200 (logMAR = 1.0). Importantly, even in this case, the visual acuity is only questionably "floor". Visual acuity below 20/200 is routinely measured in clinical settings and when converted to logMAR values, it can be recorded reproducibly to a logMAR of 2.6. Additionally, only 1 out of 14 eyes had a visual field area at baseline, which was < 10% of normal. Thus, only this one eye could be considered for some presence of floor effect. However, even in this case, the increase in visual field area from baseline to follow-up was relatively large (from ~4% to ~10% of normal), so it is unlikely that such an increase happened by chance only. In summary, we feel that floor effect concerns are unjustified in this particular data set.
Comparison with historical data. We understand that one of the goals of relatively large observational studies like the ones cited in our work4,5 is to serve as benchmarks against which subsequent (often much smaller in size) studies can be compared. Of note, testing of the visual field in our work was done exactly the same way as in the two cited works. Also, the population can be regarded as very similar, as in all three cases the patients were referred to a large tertiary center. In this regard, the criticism that we used "different methods" is unjustified. Again, we wish to reiterate that ours was an exploratory analysis that included a comparison of our findings with those of other, independent studies. The utility of our analysis, as with other small-scale studies, is to inform the appropriateness and design of future, large-scale studies. Indeed, we fully support the concept that large-scale clinical trials that enroll a broad diversity of patients are the best tools for inference to a "whole" population.
Relevance to a clinical trial. Retinitis pigmentosa is a very serious disease, there is currently no treatment, and it is devastating for its sufferers. In our opinion, our detailed description of the seven patients' longitudinal data regarding their responses to valproic acid administration is essential to the understanding of its therapeutic potential. Eleven of the 14 eyes (79%) presented in our analysis experienced gains in retinal function, and the probability of this happening by chance is small, even when corrected for clustering, etc. Our conclusion is that these results, together with the findings of similarly conducted independent studies, make a clear case for further study using randomized control trial methodology. We would like to also point out that regarding the lack of a large sample size in our pilot analysis, we are in a similar situation as Dr. Berson and his group faced before the initiation of their clinical trial testing the effects of vitamin A in retinitis pigmentosa in 1984, when the preliminary clinical data were limited to the finding that ERG was lower in 2 out of 18 patients on Vitamin A (11%, no statistical significance presented),6 yet the trial proceeded and established no effect on visual field or visual acuity, but confirmed a small positive effect on ERG (which was the primary endpoint) due to treatment.
Valproic acid side effects. First, our article included the side effects that were reported to us by the patients whose charts we analyzed, as is essential in an exploratory pilot analysis. Second, the Sandberg et al. letter introduces isolated anecdotal evidence regarding valproic acid treatment side effects. Virtually every drug is associated with a myriad of mostly rare side effects that are bound to occur when administered to the general population. However, the correct approach would be to place potential side effects into the context of the drug's potential therapeutic benefits. With respect to valproic acid, this drug was in clinical use in Europe since the mid-1960s, was approved in the United States in 1978 and has met the test of decades of use in the general population. Thus, its side effects are well known. Specifically, it has been clearly established that the frequency and severity of most of the side effects of valproic acid are dose-dependent. The maximum recommended dose in the US is 60 mg/kg/day? (which translates to 4500 mg/day for a 75 kg patient), and this is the dose range where the most (and most severe) side effects appear. In the present analysis, the maximum dose used was ~10 mg/kg/day, which is six times less than the maximum recommended dose. It has been demonstrated that the lower dosing level used in our work is associated with very low frequency and low-grade severity of side effects. Examining the literature reveals that in a study summarizing results of 16 trials and 1140 patients treated with different doses of valproate, side effects were observed in 26% of the patients, but discontinuation of the therapy was required in only 2%.7 Furthermore, in 48 children dosed with 30 mg/kg/day or more of sodium valproate for 22 months, a change in therapy was required in five patients (average dose = ~47 mg/kg/day) and withdrawal was required in two patients (average dose = ~43 mg/kg /day).8 Similarly, in a study that followed up 118 patients for an average period of 18 months on valproate monotherapy on a mean dosage of ~19.4 mg/kg/day, only four patients (3.4%) elected to discontinue treatment.9 Side effects in this last study were observed in 16% of the patients at the time of the first follow-up visit, but only in 2% (excluding weight gain that had stabilized) by the last analyzed follow-up visit.9 Neither hearing loss nor deterioration in vision function was reported in any of the three studies cited above (a combined population of 1306 patients) and, therefore, if associated with the drug intake, should be a very rare occurrence. As mentioned in our article and in our recent letter to the journal,10 we saw very few side effects with the low dose employed in our analysis, and only one patient elected to discontinue treatment. We agree with a recent review that summarized the 30 years of clinical experience with the drug, which stated "Valproate also possesses an impressive safety profile, being well-tolerated in most patients".11
Summary. Although valproic acid has been used for several indications during the past 30+ years, its possible therapeutic effect in retinal degenerative diseases is currently unknown. We believe that the state of knowledge regarding the treatment of retinitis pigmentosa at the inception of our analysis warranted a "pilot analysis" approach to the exploration of valproic acid and a detailed description of individual eyes. We would like to emphasize that the goal of our pilot data analysis was to examine the potential benefit of valproic acid; this analysis provided preliminary data that we believe warrants further investigation in a randomized control study. The initial impression of a therapeutic benefit presented in our analysis can be best clarified in such a clinical trial.12
Sincerely, R. Tzekov C. Bigelow C. Clemson J. Checchi M. Krebs S. Kaushal
1. Sandberg MA, Rosner B, Weigel-DiFranco C, Berson EL. Lack of scientific rationale for use of valproic acid for retinitis pigmentosa. British Journal of Ophthalmology, 26 October 2010 (online). 2. Clemson CM, Tzekov, R, Krebs, M, Checchi, JM, Bigelow, C, Kaushal, S. Therapeutic potential of valproic acid for retinitis pigmentosa. British Journal of Ophthalmology. July 20, 2010 [epub ahead of print]. 3. Rosner B, Glynn RJ, Lee ML. The Wilcoxon signed rank test for paired comparisons of clustered data. Biometrics 2006;62(1):185-92. 4. Berson EL, Rosner B, Weigel-DiFranco C, Dryja TP, Sandberg MA. Disease progression in patients with dominant retinitis pigmentosa and rhodopsin mutations. Invest Ophthalmol Vis Sci 2002;43:3027-36. 5. Massof R, Dagnelie G, Benzschawel T, Palmer RW, Finkelstein D. First order dynamics of visual field loss in retinitis pigmentosa. Clin Vision Sci 1990;5:1-26. 6. Berson EL, Rosner B, Sandberg MA, Hayes KC, Nicholson BW, Weigel- DiFrano C, Willett W. Vitamin A supplementation for retinitis pigmentosa. Arch Ophthalmol 1993;111(11):1456-9. 7. Schmidt, D. Adverse effects of valproate. Epilepsia ZS (Supp1.1) 1984:S44-S49. 8. Herranz, JL, Arrnijo, JA, Arteaga, R. Effectiveness and toxicity of phenobarbital, primidone, and sodium valproate in the prevention of febrile convulsions, controlled by plasma levels. Epilepsia 1984;25(1):89- 95. 9. Bourgeois, B, Beaumanoir, A, Blajev, B, de la Cruz N, Despland PA, Egli M, Geudelin B, Kaspar U, Ketz E, Kronauer C, et al. Monotherapy with valproate in primary generalized epilepsies. Epilepsia (Suppl. 2) 1987:28:S8-SI I. 10. Kaushal, S, Clemson, C, Tzekov, R, Krebs, M, Checchi, J. Re: conclusions of Clemson et al. concerning valproic acid are premature. 20 September 2010 [epub ahead of print]. 11. Peterson, GM, Naunton, M. Valproate: a simple chemical with so much to offer. Journal of Clinical Pharmacy and Therapeutics 2005;30:417-421. 12. ClinicalTrials web site. Trial entry: NCT01233609. http://clinicaltrials.gov/ct2/results?term=NCT01233609
Licence for Publication The Corresponding Author has the right to grant on behalf of all authors and does grant on behalf of all authors, an exclusive licence (or non exclusive for government employees) on a worldwide basis to the BMJ Publishing Group Ltd and its Licensees to permit this article (if accepted) to be published in BJO editions and any other BMJPGL products to exploit all subsidiary rights, as set out in our licence(http://group.bmj.com/products/journals/instructions-for- authors/licence-forms/).
Competing Interest: None to declare.
Conflict of Interest: