Flaws in 2008 Pediatrics article

[Buster]

I posted this first on October 5th 2008 in response to a number of papers being released. http://www.latitudes.org/forums/index.php?...st=0#entry26571

 

Our pediatrician forwarded a set of papers to me from the June issue and October issue of Pediatrics saying "these say PANDAS doesn't exist". That is not what the papers say but it looks like people read abstracts or titles and not papers.

 

I thought I'd spend a moment here about the papers, the methodological flaws and the actual conclusions of the papers in case someone brings any of these up to you:

 

The papers are:

• Kurlan and Kaplan's June 2008 Pediatrics paper entitled "Streptococcal Infection and Exacerbations of Childhood Tics and Obsessive-Compulsive Symptoms: A Prospective Blinded Cohort Study"
  
  http://pediatrics.aappublications.org/content/121/6/1188.long
  
• Singer's article in the same journal titled "Serial immune markers do not correlate with clinical exacerbations in pediatric autoimmune neuropsychiatric disorders associated with streptococcal infections." http://www.ncbi.nlm.nih.gov/pubmed/18519490
  
• Lin, Kaplan and Leckman's October 2008 paper "Streptococcal upper respiratory tract infections and psychosocial stress predict future tic and obsessive-compulsive symptom severity in children and adolescents with Tourette syndrome and obsessive-compulsive disorder." http://www.ncbi.nlm.nih.gov/pubmed/19833320
  

The first thing to know is that all of these papers are studying the same set of subjects. There are basic recruitment flaws to all three papers.

 

Problems with the papers

 

The papers have 7 major methodological flaws:

1. The studies are about long-term tic disorders in older kids (11-12) and not about sudden onset PANDAS or symptoms in younger kids (mean age 7.5).
   This is study about children with long term chronic tics who had onset over 3 years prior. 75% of the subjects in the proported PANDAS group were diagnosed with TS (i.e., had symptoms for > 1 year and no remission for > 3 months). The sample is pulled from the longitudinal study by Kurlan reported in the same issue of Pediatrics.

   
    
   

2. The study did not indicate how many of the children had already hit puberty with the massive hormonal and immune changes that occur.

   
   

   Multiple studies indicate that progestrone and other hormones affect the T helper cell regulatory response. Kirvan and Swedo looked at sudden onset in pre-pubescent children and thus if this was not controlled, this brings even more question into whether the sample is valid.

   
    
   

3. The diagnostic criteria used by Kurlan for his proported PANDAS group is not the same as used by Swedo or Snider.
   
   Kurlan used a "clinical course characterized by the abrupt onset of symptoms
   or
   by a pattern of dramatic recurrent symptom exacerbations and remission". Swedo used an "Episodic course characterized by acute, severe onset
   and
   dramatic symptom exacerbations." (emphasis added) While these sound similar, they are not. Indeed the episodic nature is the key distinguishing element in Swedo's studies as is the severity of the symptoms. Swedo wrote in her May 2003 response to Kurlan, "The episodic, relapasing-remitting course of the PANDAS subgroup is distinctly different from the undulating, waxing-waning course seen in other patients with OCD or tic disorders." In addition, Singer discloses that the average onset was over 4 years prior to his study. Kurlan discloses that the onset was not from documentation, but rather obtained through interviews thus being very prone to recall bias. So it is unclear whether the proported PANDAS subjects met the Episodic course, the severe onset, and the dramatic symptom exacerbations of the Swedo critieria.

   
   

4. The subjects exhibited no OCD behavioral changes and are different from Swedo's/Snider's subjects
   
   The subjects attributed to be PANDAS subgroup in the Kurlan and Singer studies had a CY-BOCS score that changed only 1.6 [-0.4 to 3.6] (i.e., no change) with controls changing 1.0 [-1.1 to 3.1] (i.e., no change). This is hardly episodic given the baseline CY-BOCS scores and certainly does not indicate remission within the 2 year period but rather the small waxing and waning of OCD symptoms and the limited objective accuracy of the CY-BOCS measure. Swedo subjects often exhibited > 15 points of change in CY-BOCS score. Granted, I have some issues with these studies as well, but this 10 fold difference in CY-BOCS measured exacerbations definitely makes one wonder if these are the same subgroups.

   
   

5. The subjects have high tic exacerbations but not OCD exacerbations.
   
   It is true that the subjects in the Kurlan/Singer studies had YGTSS-tic exacerbations of 11 pts [4.2-17.9] and certainly this is significant, but most of the other papers on PANDAS focus on the OCD element and less on the tics. So this begs the question about whether the study was more about Tourette's and association with streptococcal pyogenes rather than PANDAS. It also begs the question whether Swedo's criteria should include tic-only exacerbations. It also begs the question whether chronic tics are fundamentally different than onset.

   
    
   

6. The subjects were all on numerous anti-psychotic, alpha-agonists, and mood stabilizers.
   
   It is totally unclear what these effects had on the subjects and how these variables were controlled. Did this suppress the OCD response?

   
    
   

7. The blind was broken by informing pediatricians if the children cultured positive for GABHS
   
   While this is understandable, it defeats the actual study. PANDAS is thought to be similar in pathogenesis (cause) to Sydenham Chorea where symptoms emerge after
   untreated
   streptococcal infection. Multiple studies show that treated GABHS infections lower the incidence of ARF and SC. Given the similar pathogenesis, is it a wonder that treated GABHS would not yield increased exacerbations?
   
   

This is a long way of saying that I don't see how they can reach any conclusion regarding PANDAS given that their kids don't seem to be PANDAS subjects but rather Tourette's subjects.

 

I have numerous other issues including sampling theory problems, but the key question is how many of the kids in Kurlan's and Singer's studies actually had PANDAS as opposed to being kids with severe Tourettes with the unfortunate waxing/waning of tics associated with Tourette symptoms.

 

Best regards,

 

Buster

Edited January 22, 2012 by Buster

[EAMom]

Note it is my understanding that these are the same kids used in the 2011 study http://www.sciencedirect.com/science/article/pii/S0890856710008166 which was mentioned in Neil Swidey's 2012 Boston Globe article on PANDAS.

 

"Dr. James Leckman, a professor of child psychiatry at Yale and specialist in Tourette’s syndrome, was the lead author on what is perhaps the most persuasive study challenging the PANDAS hypothesis. That long-term study, published in 2011, found no compelling evidence linking the exacerbation of tic and OCD symptoms to new strep infections.

 

Yet Leckman tells me that in late 2008, well after all the patients had been enrolled in the study, he came to an astonishing realization: He and his coauthors had been studying the wrong children. Most of the kids in the study resembled those he regularly sees in his clinic — children with “garden-variety” Tourette’s and OCD. But after working with more physicians treating PANDAS patients, he had come to see firsthand that there was a distinct group of kids who literally had changed overnight, with dramatic onslaughts of OCD and other symptoms. And these “true” PANDAS/PANS cases weren’t represented in his study in any meaningful way.

 

Leckman says he lobbied his coauthors, who included Harvey Singer, to admit to this failing in their paper. But they refused, insisting they had followed the published PANDAS criteria in selecting their subjects. Leckman had to concede they were right — the children all met the criteria Swedo’s team had established. It’s just that he now believed those criteria were far too broad. So Leckman’s name was listed first on an influential paper that he felt was technically accurate but missed the larger point."

 

In other words, EVEN THE LEAD AUTHOR ON THIS PAPER ADMITS THE STUDY DIDN'T HAVE "REAL" PANDAS KIDS IN IT!

Edited April 3, 2013 by EAMom

[T_Mom]

Thank you for reposting this excellent review.

[smartyjones]

eamom -- is there some reason that you posted this now? or did you just read or reread it and want to comment on it?

Edited April 5, 2013 by smartyjones

[rowingmom]

I think the important thing is that research on the same cohort of children, a cohort not specifically diagnosed with PANDAS, has been used and is still being used, in studies and reports (2011, 2012) challenging the PANDAS hypothesis. These may appear to be the result of new research, but are actually one and the same. Something to keep in mind when being challenged by oppositional doctors.

[EAMom]

Hi Smartyjones,

 

I happened to post it now because I realized Buster, myself, and many others had conjectured early on that the 2008 study (and also the 2011 study based on the same kids) by Kurlan/Singer/Leckman/Kaplan didn't actually have PANDAS kids in them. They were studying older kids with long standing TS pulled from a TS study group with minimal OCD changes etc etc. And, here we actually have (in the Boston Globe) the lead author (Dr. Leckman) confirming that "true PANDAS/PANS cases weren't represented in his study in any meaningful way."

 

This is a complete validation of what we have been saying all along. I thought it was terribly important to point out that it was not only PANDAS parents saying the study was flawed, but also one of the lead authors (Leckman) of the study. I should have posted it back in 2012 when the Globe article first came out, but didn't think to link it to this discussion at the time. (Although I'm sure it got posted elsewhere on this forum.)

[EAMom]

I think the important thing is that research on the same cohort of children, a cohort not specifically diagnosed with PANDAS, has been used and is still being used, in studies and reports (2011, 2012) challenging the PANDAS hypothesis. These may appear to be the result of new research, but are actually one and the same. Something to keep in mind when being challenged by oppositional doctors.

 

Good point rowingmom. Unfortunately these 2008/2011 studies "disputing PANDAS" seem to be quoted/referenced far and wide.

[T_Mom]

Yes, the 2011 study was quoted to us twice by Blcross and then again by the federal govt. Office of Personnel Management as we spent most of 2012 appealing for payment of a 2011IVIG....we lost even though we had multiple proofs across measures of academics, OCD, rage extinction...they pointed to that article.

[EAMom]

Yup...even though Singer's research (2011 paper, and also the others including 2008 Pediatrics papers--since they were all the same kids) SHOULD have been completely discredited by the Boston Globe article, the damage has already been done with too many other people/places quoting the faulty research. Grrrr. But at least in the future, parents will have some recourse (IF they are on the ball).

 

It's a maddening situation. I wonder if Singer/Kulran know (or care?) how many kids/and families they have harmed with their PANDAS naysaying and bad research? Judging by the Globe article, it sounds like Singer at least, doesn't care one iota.

[thenmama]

We really need a diagnostic code for PANDAS, so the "controversy" over its existence isn't used as cause to deny treatment. My approach with insurance is: "We didn't request an evaluation of the medical condition, we requested coverage for a treatment based on medical diagnostic codes relevant to our child's condition and current health status. The article you've produced has been discredited, it is only one article, where many more provide evidence for both the condition and its treatment. I'm sure you also are aware that the condition is recognized by the NIH, and its standard of care treatments are listed on the NIMH Pediatric Division's PANDAS website. It seems appropriate to remind you that you're not our child's physician, nor are you an expert in this condition or a relevant field. Your opinion about the condition's existence and our child's medical diagnosis is unqualified, unwarranted, and irrelevant in this case, where you are merely a claims adjudicator. So, as for the benefits request, let's get back to determining whether or not the treatment should be covered for the diagnostic code it was requested for, according to the specific terms of our health plan, and in a manner compliant with X (fed or state) regulations for claims and appeals and consistent with our member rights to a full and fair review under (PPACA/ERISA/XXX), now shall we?"

 

Make them stick exactly to the letter of the plan, their own established coverage guidelines, and state or federal law. This is how it is supposed to work: insurer or TPA gets pre-auth request; looks at case/medical need and evaluates coverage request against plan terms and coverage guidelines to determine if plan can cover the requested treatment; issues decision according to how it stacks up against plan provisions. It should basically be a checklist sort of exercise based on the plan documents and its relevant established policies and guidelines. But, insurers never reference their own guidelines or attempt to determine whether a treatment meets coverage criteria per the actual plan. Denials are systematic and the plan instruments are all a facade--never really implemented. They just throw out whatever as a rationale, figuring it'll stick. Hold them accountable--make them play exactly by the book. The condition's existence isn't up for debate in a claims review. Period.

 

Sorry that happened to your family, T.Mom.