Rebuttal Report of Barbara J. Burns, Ph.D

Rebuttal Report of Barbara J. Burns, Ph.D.

February 4, 2005

I.Introduction

I previously prepared a disclosure and an expert report (dated 10/25/04) for the Rose D. case which sets forth my professional experience and affiliations, and which includes my CV and a list of prior cases in which I have testified. That information will not be repeated here.

I was asked to prepare a further report to address the opinions presented by Dr. Leonard Bickman in his initial report (dated 10/28/04); the rebuttal reports of Dr. Bickman and Dr. Peter Metz (filed on 11/24/04); and the depositions of Dr. Bickman (01/06/05) and Dr. Metz (01/17/05).

II.Materials Reviewed

A list of supplemental documents reviewed is listed on the last page of this report.

III.Response to Dr. Bickman’s Initial Report (10/28/04) and Deposition (01/06/05)

1. Critical Terminology

Despite his insistence on precision and scientific methods, Dr. Bickman confuses concepts and uses critically different terminology interchangeably. Specifically, he refers in one sentence of his Rebuttal Report to “home and community-based services,” intensive home and community-based services,” “system of care,” “continuum of care,” and “wraparound approach.” These terms mean quite different things to the professional mental health and research community, and have quite different implications in this case. A continuum of care is simply a set of services that cover a range from the most restrictive to least restrictive. A system of care describes the operation of an entire network or array of children’s mental health services, and is generally understood to refer to a system that operates consistent with certain value-based principles that were initially developed by the federal government during the 1980s. Wraparound is a method for delivering children’s mental health services that operates on the individual child level, and that also implies or assumes certain principles. Home and community-based services, on the other hand, are distinct treatment interventions provided in an organized fashion through a coordinated clinical team that includes certain professionals and stakeholders (the child and family), and that respects certain treatment guidelines.

My understanding is that this case is about home and community-based services that are covered by Medicaid. It is with this understanding that I have prepared this report.

B.General conclusions

Dr. Bickman’s 7-page report is comprised mostly of his background and credentials. His report does not differentiate among systems of care, wraparound and home and community-based services, and does not address work by others in the field. The report does not indicate that he has ever visited Massachusetts, reviewed the Massachusetts behavioral health programs for children, examined data from any of these programs, and particularly its two demonstration home-based programs (MHSPY and CFFC). In general, the report does not indicate that Dr. Bickman has any knowledge of the way mental health services are provided to children in Massachusetts specifically. The report does not present any opinions or conclusions concerning the availability or lack of availability of home-based services in Massachusetts. The report does not present any opinions or conclusions concerning the effectiveness of home-based services in Massachusetts, including the MHSPY and CFFC programs. The report does not present any opinions or conclusions concerning the effectiveness or probable effectiveness of the home-based services that plaintiffs are attempting to secure in this case for Medicaid-eligible children in Massachusetts. The report simply refers the reader to books and papers which report the findings of three studies conducted by Dr. Bickman in other states, but no effort is made to determine the relevance or applicability of these findings to Massachusetts.

Dr. Bickman’s deposition confirms many of the points above and further demonstrates a fundamental problem. It is Dr. Bickman’s opinion that no mental health services for children are effective (p. 230, lines 14-19, Deposition, 01/06/05). His belief that nothing works may contribute to his notion of maintaining the status quo in Massachusetts (i.e., “It’s less expensive typically to keep going and less disruptive than to introduce something which you have no data about ...” [p. 233, lines 18-22, Deposition, 01/06/05]). Such conclusions logically lead to paralysis. Missing from Dr. Bickman’s viewpoint are (1) a considerable knowledge base which identifies numerous clinical interventions that could benefit youth with SED in Massachusetts; and (2) recognition, for example, of the cost (e.g., for 49,000 administrative hospital days) and disruption in the lives of youth, most of whom could function in the community with the support of evidence-based treatments and supports.

C.The Three Studies Cited in Dr. Bickman’s Report

Dr. Leonard Bickman conducted three studies of systems of care and wraparound in the 1990s. Most importantly, the studies do not attempt to measure the effectiveness of home-based services -- the very issue in this case. Nor could they even begin to reflect the variability in design and implementation necessary to answer the question of whether systems of care, as model for providing children’s mental health services are effective, even in well-conceived and well-developed systems of care. Even if the types of variability described below did not exist, a sample size or number (N) including the total of the three service systems studies would not begin to be sufficient to reach definitive conclusions about a system of care approach. To lay out the complexity of this type of research and evaluation, key factors that have to be measured and controlled because they directly influence the functioning and effectiveness of a system of care are:

• System Goals and Priorities: A comprehensive approach to prevention, treatment, and evaluation, versus a focus on youth with severe emotional disturbance and significant impairment; values and incentives driving the system, such a community-based versus institutional, or family-centered versus practitioner-directed care.

• Funding: Amount of resources; whether dollar are managed or not; the use of waivers; and the mix of private versus public (Medicaid, CHAMPUS, Medicare) insurance including the benefits, co-pays, and deductibles for each.

• Organization of Services: Ownership types (public, private not-for-profit, private for- profit); state versus county run; integrated versus separate service agencies; single-sector versus multiple-sector participation; and the culture and climate of service organization which promote innovation or traditional approaches to care.

• Geography: Clinically trained professionals are less available in rural areas and use of allied professionals more necessary; access in urban areas is influenced by acceptance of Medicaid and barriers among competing agencies.

• Target Population: Full range of clinical conditions in the general population with varying severity (acuity), chronicity, risk factors, and history of prior treatment in contrast to a more restricted subset of youth with severe clinical and functional impairment.

• Array and Availability of Services: Full array of community-based and residential clinical and support services, in contrast to a traditional array of clinic and hospital care; true availability is influenced by supply (sufficient versus a waitlist), access (such as, provision beyond the 9-5 day); location (influencing travel requirements); and cultural competence (whether the setting is perceived as user-friendly to different ethnic, cultural, and socioeconomic populations).

• Local Conditions: The extent of community involvement; treatment preferences may vary with the racial/ethnicity mix and socioeconomic characteristics of the community.

• Quality of Service Provision: Whether service components are implemented as intended (i.e., dose, duration, and therapeutic quality), possibly most critical to child outcomes.

• Adequacy of the Work Force: Availability of well-trained providers; the organizations’ ability to support retraining and supervision; and salary sufficient to encourage retention of trained personnel.

The likelihood of addressing the degree of variation in each of the factors above, through just a few studies, is obviously low. For “gold standard” research, multiple randomized clinical trials would be conducted across the country to represent variability in the above factors. This would be prohibitively expensive and too complex to design. At the next level of research sophistication, quasi-experimental designs, comparing communities with a system of care to those without one could be used to examine the differential benefit of a system of care. This is problematic because the system of care philosophy, values, and principles have spread across the country since this became the prevailing approach for children’s mental health services in the late 1980s, even though these systems of care may not be sufficient in all respects. At a third level, observational studies (pre-post designs without comparison sites) provide evidence of promising practices, but suffer from the possibility that such findings might occur in the absence of a system of care, although enough successful demonstrations would lend credibility to the model. The likelihood of meaningful change in these challenged youth without significant clinical intervention is unlikely.

The studies conducted by Dr. Bickman provided a starting place in a new area of research in the early 1990s. One needs to be exceedingly cautious with any suggestion that these findings can be generalized to other communities or states due both to the impossibility of incorporating the factors above, as well as to specific research limitations associated with the Fort Bragg, Stark County (Ohio), and the Midwest military wraparound studies. Apart from limitations in study design and general applicability, more recent research has demonstrated the weaknesses in his conclusions, and provided affirmative evidence of the effectiveness of home and community-based services.

I do not believe that the findings of Dr. Bickman’s three early studies are relevant to Massachusetts or applicable to this case, since they do not even attempt to evaluate the effectiveness of the home-based services sought by the plaintiffs. Moreover, it does not appear that Massachusetts state officials do either, since, despite Bickman’s conclusions, they developed a new home-based services initiative (CFFC), and then created an evaluation protocol for this initiative that in no way attempts to replicate the evaluation methodology employed by Bickman. Nor do I agree with the suggestion in Dr. Bickman’s initial report that the home-based services requested by the plaintiffs in this case are not effective, necessary, or useful. My conclusions are based upon three analyses: first, a review of Bickman’s three studies, including commentary about each; second, recent findings from a national multi-site system of care[BJB1] evaluation which includes the provision; and third, commentary on the State’s own plan for delivering and evaluating CFFC. The specific evidence for home and community-based services was addressed in my report of October 25, 2004.

1.Fort Bragg

The Fort Bragg Demonstration project, designed and implemented in the early 1990s, was intended to provide, without barriers (waiting time, cost), the full array and amount of mental health services needed for military dependents in the context of a responsive service system. The results of unlimited benefits without a co-payment were then to be compared to two other military installations in the South with services under the usual, but very generous, CHAMPUS benefits.

Demonstration Design Issues:

The Fort Bragg Demonstration was intended to test a system of care for a general population rather than one for children with serious emotional disturbance (SED). Dr. Bickman indicates that this was a test of managed care, albeit one with no limitations on benefits and no co-pays or deductibles. This was entirely inconsistent in the 1990s when managed care for mental health included the preceding criteria.

CHAMPUS historically provided generous benefits at little cost to an employed population (the best mental health benefit package in the country) -- entirely inconsistent with either private insurance or emerging managed care. Excessive and expensive services were provided to some children at Fort Bragg who did not require them (e.g., a team and case management for children with a diagnosis of ADHD). With the exception of case management, the full range of home and community-based services were not included in Fort Bragg’s system of care and were not considered in his findings.

Methodological Issues:

The measures used were state-of-the-art at the time, but lacking specific indicators of functioning now in use in current research. The analysis was sufficiently sophisticated, although the impact of services on youth with SED was not analyzed until later, and the interpretation of findings was problematic.

Comparability of the Children:

Perhaps the greatest criticism of this study is the preferred socioeconomic status of most children in this study. A military community, although claimed by Dr. Bickman to have children similar to those in the general population seeking services, differs significantly from low income families eligible for Medicaid and children with SED who are likely to come from broken or challenged families. These military families were employed, usually with both parents living in the same household. At least the member in the military was likely to have been screened for severe mental illness and not accepted into the service if s/he had a major behavioral health condition. Further, due to the necessity of military readiness, vigilance about child behavioral problems is common. Early identification, and a wealth of other services, were available -- not just mental health – with strong support to use them. This contrasts to families who struggle to obtain public sector services with little knowledge about psychiatric conditions or services, or support to obtain them.

Comparability of Services:

As indicated above, home and community-based services, except for case management, were not specifically included or evaluated in the demonstration. Recognition of the need for a full range of home and community-based services occurred in the field after this demonstration was concluded.

Professional Criticism of the Study:

The unexpected finding of no clinical difference between Fort Bragg and the comparison military forts elicited a debate which filled several special issues of scientific journals and many additional manuscripts (see particularly American Psychologist, Volume 31, 1996) and the Journal of Mental Health Administration (Volume 23, 1996). For the most part, there was agreement about the professional conduct of the evaluation, but disagreement with Dr. Bickman’s interpretation of the findings. A key -- and sometimes forgotten issue – was that the children improved clinically at both Fort Bragg and the other sites. Both child groups moved out of the clinical range on clinical research measures (84% were better and 50% very much so). Plausible explanations include:

• Clinical treatment at the comparison forts was of sufficient professional quality (private practitioners located around military bases, not public services) to yield similar clinical benefit; and

• Based on research conducted by Heflinger (1996), the comparison site service systems met criteria for a system of care based on evidence that service coordination was of similar quality at the comparison sites.

Another important, but ignored, finding was the dramatic reduction in the use of institutional care at the system of care site (Fort Bragg). The rate of hospital admissions (15.7%) at Fort Bragg before the demonstration declined to 8.3%, averaged across demonstration years. For the comparison military forts, the starting hospital admission rate was comparable to Fort Bragg, but 14.2% over the demonstration years (although declining somewhat in later years). Similar and equally dramatic findings occurred in admissions to residential treatment centers (RTC). Days spent in an RTC were more than double for the comparison sites: 63.5 days at Fort Bragg versus 140.8 days in comparison sites (see Tables 5.4 and 5.8 in Bickman et al., 1995). These are meaningful findings in the context of valuing less use of restrictive care to avoid the risks of poor child outcomes and excessive costs. These reductions in hospital and RTC admissions and length of stay are, in fact, proxies for child clinical outcomes. If children are not deteriorating to the extent that they require inpatient care or rehabilitative RTC stays, by definition, the Fort Bragg services improved clinical status.

A further relevant finding relates to better outcomes for Fort Bragg versus comparison site youth with SED. Significant differences were found by Lambert and Guthrie (1996) on 3 out of 12 clinical measures (Youth Self Report, Global Level of Functioning Assessment Scale) in favor of the youth treated at Fort Bragg.

Other viewpoints could be documented here, but the three stated above argue against Dr. Bickman’s interpretation of no clinical benefit of from the Fort Bragg demonstration of a system of care program. In summary:

• Comparison youth at other sites received high quality clinical care also consistent with a system of care approach, negating, to some extent, any useful conclusions concerning “usual care”;

• There were significantly greater reductions in institutional care at Fort Bragg; and

• There were several relevant indicators of greater clinical benefit for youth with SED at Fort Bragg.

Relevance of the Findings to Massachusetts: