Author(s): Katherine P. Strater, Melissa Strompolis, Ryan P. Kilmer, James R. Cook

resumen:

This study sought to explore caregiver-reported needs of youth and families receiving services within a System of Care (SOC). Data were collected to augment the in-depth, standardized interview protocols that are part of the National Evaluation of SOCs in order to identify gaps in the local delivery of services and supports to youth with severe emotional and/or behavioral disorders and their families. Primary caregivers of youth receiving services within the SOC were asked to report on the current need(s) of the youth and the current need(s) of the family. These open-ended responses were coded and examined for themes.  The most common needs of the youth were Services and Life Management, while the most common needs of the family were Services and Financial.  Focus groups conducted with youth and caregivers confirmed these themes.  Findings from this study shed light on the diverse needs of these families, indicate that families believe they are not receiving adequate supports and services, underscore that youth have distinct views, and highlight that providing a way for the typically disenfranchised youth and families to express their perspective is central to the functioning of SOCs.  Study findings were used by a local nonprofit organization to inform their advocacy efforts, and were incorporated into an enrollment process at a local mental health agency are part of a programmatic change pilot effort.

artículo:

Introduction
The System of Care (SOC) philosophy emerged in the mid-1980s to address the need for reform in child mental health (see, e.g., Knitzer, 1982; Stroul & Friedman, 1986). In the decades since, the approach has become the central element of child mental health policy in the U.S. (Hodges et al., 2010).  Funded currently via the federal Children’s Mental Health Initiative, SOCs have been developed around the country to address the diverse and multi-faceted needs of youth with severe emotional disturbances (SED) and their families.  Unfortunately, notwithstanding the federal dollars to support their development and implementation, and the manifold efforts to assess their fidelity of implementation (e.g., Bruns et al., 2010), SOCs have often struggled to implement key components of the philosophy (Cook & Kilmer, 2004), and youth and families encounter difficulties in accessing and receiving needed supports and services.  This study evolved out of a larger SOC evaluation effort – it sought to assess the needs of youth and families enrolled in a SOC.  By shedding light on the needs of the youth and families, it aimed to inform system change as well as advocacy efforts to improve service provision and system function.

At their core, SOCs seek to provide comprehensive networks of services and supports for children with severe emotional disturbances (SED) and their families. SOCs are designed to coordinate the efforts of multiple child-serving systems (i.e., mental health, child welfare, education, health, juvenile justice) and families’ natural supports in order to provide a continuum of care to youth and their families and eliminate gaps in service delivery. In these collaborative SOCs, plans of care are built on the existing strengths of the youth and family and tailored to their specific needs (see, e.g., Huang et al., 2005; Pumariega & Winters, 2003; Stroul & Friedman, 1986). Ideally, SOCs also reinforce the presence of natural community supports and resources and encourage families to utilize them effectively (Pumariega & Winters, 2003). 

The primary practice model within SOCs is the wraparound approach. Wraparound refers to a set of practice principles and a flexible, family-centered process of developing plans of care that provide individualized services and supports appropriate for each family (see Walker et al., 2004; see also VanDenBerg & Grealish, 1996). As a primary component of wraparound, youth and families actively participate in multidisciplinary teams (i.e., families have “voice and choice;” The National Wraparound Initiative, 2011) called Child and Family Teams (CFTs), which coordinate care and support for youth and families (Burns & Goldman, 1999). In order to be effective, teams must elicit information about the youth’s and family’s ongoing and changing needs, especially during transitions such as those from formal service systems to community-based supports (e.g., Walker et al., 2004).

Research on the SOC philosophy and wraparound approach has yielded mixed results (see Cook & Kilmer, 2004 for a review; see also Bruns, Sutter, Force, & Burchard, 2005; Epstein et al., 2003; Graves, 2005).  The literature points to modest improvements for the youth and families involved (e.g., Cook & Kilmer, 2004; Holden et al., 2003; US Department of Health and Human Services, 2003), but findings to date point to considerable variation in fidelity and implementation (Cook & Kilmer, 2004; Kilmer, Cook & Palamaro Munsell, 2010), with many communities struggling to adopt key SOC components into practice (Bruns et al., 2005; Cook, Kilmer, DeRusso, Vishnevsky & Meyers, 2007; Epstein et al., 2003; Walker & Schutte, 2005). For instance, many CFTs struggle to implement truly child- and family-centered, strength based service plans (Cook et al., 2007), and few incorporate natural supports into the plan of care or team meetings (Cook et al., 2007; Epstein et al., 2003; Walker & Schutte, 2005).   Although efforts supporting the assessment of SOCs’ fidelity have emerged (e.g., the National Wraparound Initiative, see  http://www.nwi.pdx.edu/ ), there are not clear mandates to evaluate the processes and implementation of SOC or wraparound. Rather, Congress has required that all federally-funded sites participate in the National Longitudinal Study, also known as the National Evaluation of Systems of Care (National Evaluation), to examine the effectiveness and impact of SOCs. 

The National Evaluation assesses self-reported service utilization, satisfaction with services, and overall youth and family functioning. This evaluation includes regular, standardized interviews with youth enrolled in a SOC and their caregivers, beginning at baseline (within 30 days of enrollment) and continuing with follow-up interviews every six months for up to three years. Participants complete multiple self-report measures regarding service experiences and functioning of the youth and family.

Although the National Evaluation assesses the strengths and limitations of existing services and can serve as an effective evaluation of “outcomes” by capturing multiple salient indicators (e.g., youth symptoms and impairment, youth behavioral and emotional strengths, caregiver strain, satisfaction with services), the protocol has gaps that warrant attention. Notably, it lacks items that identify youth and family unmet needs or service gaps, as identified by those involved in the system. Given the core tenets of the SOC philosophy, it is particularly important to attend to the needs of the system-identified youth, the family’s needs, and contextual influences on the youth’s adaptation, including the larger family environment and the family’s connectedness to the community (e.g., Cook & Kilmer, 2010a; 2010b; Kilmer et al., 2010).  In that vein, assessing the most pronounced needs of youth and families can provide them and their CFTs with specific feedback regarding areas in need of attention in the plans of care, as well as identify gaps in system functioning. Moreover, presenting the results of needs assessments within community and system contexts in which needs can be addressed can support the organization of services and inform well-targeted, community-based interventions (Gabhainn, Dolan, Canavan, & O’Higgins, 2009). Thus, ongoing assessment of needs constitutes a critical component of adequate service provision and system function.

The Current Study: Context and Rationale
Data for this study were collected as part of the local efforts to augment the National Evaluation protocol for the SOC in Mecklenburg County, North Carolina, an initiative known as MeckCARES. The study was initiated in response to informal observations by evaluation team members following the administration of interviews with youth and families. Specifically, evaluation team members frequently noted that, during the standardized interviews, caregivers expressed frustration that their needs were not being met. These perspectives were not being captured by the standardized format of the National Evaluation interview protocol. In response to both the local observations of unmet needs and the challenges with implementation identified in the extant literature, the evaluation team integrated additional items in the protocol to explore youth and family needs. The findings yielded from these needs assessment items were subsequently discussed with both caregivers and youth in focus groups. The sections that follow describe these processes and findings.

Method
Procedure
This study draws on data collected from 2010-2011 via the National Evaluation interviews of families involved in MeckCARES. The standardized, in-depth National Evaluation protocol assesses child and family outcomes, service utilization, and service satisfaction. These face-to-face interviews are completed with caregivers and youth every six months for up to three years. Caregivers received $30 Visa gift cards each time they completed an interview and youth received $20 Visa gift cards. 

Participants
Participants included 216 caregivers (one per family) who provided responses to the interview items within each timeframe. Participating caregivers included biological parents, foster parents, and adoptive parents, as well as service providers (group home workers, residential treatment staff, etc.). Caregivers were selected to participate for a given interview timeframe if the youth had spent the majority of the previous 6 months in their home or facility. 

Measure
Needs assessment. Following approval of the University of North Carolina at Charlotte’s Institutional Review Board, two qualitative items were added to the end of the National Evaluation’s interview protocol in October, 2010: What do you feel your child is in most need of at the current time?  What do you feel your family is in most need of at this current time? Caregivers who did not provide a response initially were prompted to consider the question more carefully before providing “nothing” as their final response. Given that these items were incorporated well into the evaluation period for this site, caregivers provided responses at varying time points for the interviews, from baseline through 36 months. Caregivers were not limited to the number of needs they could identify for their children or families. Interviewers recorded caregiver’s responses verbatim and prompted for clarification when necessary. After 100 responses were collected, two members of the evaluation team began the coding process.

Coding of needs data
After caregivers’ open-ended responses were entered into a spreadsheet, two evaluation team members organized and condensed responses into overarching themes and applied category labels. The categories and responses were shared with the evaluation team for feedback and suggestions. Based on this feedback, some responses were re-categorized and some categories were renamed. This process of team feedback and category modification was repeated once more to refine the organization of the data.

While the evaluation team completed the data coding process, it was important to confirm that the caregivers’ responses were accurately captured and coded.  As a next step, the evaluation team conducted a focus group with youth and their families to present the information, gather feedback on the responses and coding, and discuss how the information could be used.

Focus Group
The focus group was planned and executed in partnership with ParentVOICE, a local nonprofit organization that provides a range of supportive and advocacy services for families of youth with mental health challenges. ParentVOICE assisted the evaluation team by identifying potential youth and families, providing physical space for the event, and helping facilitate the focus group sessions.

Prior to contacting youth and families, the Institutional Review Board of the University of North Carolina at Charlotte approved this work. Youth and families were sent informational letters in the mail and were asked to call to confirm their participation in the focus group. They were provided with dinner and $30 Visa gift cards for participating. 

After receiving an agenda, youth and families were provided with brief information about the needs assessment data collection, the coding process, and the purpose of the focus group.  Both youth and families signed informed consent forms to participate in the study. Youth (N = 7) and their caregivers (N = 5) then separated so that the focus groups could be conducted independently.  Although presented in different formats (i.e., a PowerPoint presentation for caregivers and multimedia for youth), both groups were asked a similar series of questions (e.g., What do you think about the categories?  Do the categories make sense?  Do you agree with the categories?  Are the categories similar to your current needs?  What needs are you experiencing that are not on the list?). Multiple evaluation team members were present to transcribe the discussion.

At the end of the focus group, the evaluation team gathered the caregivers’ feedback about the focus group process. Caregivers were asked to complete brief 14-item evaluation surveys that included two open-ended items (e.g., What suggestions or comments do you have regarding the information that was presented today?) and 12 Likert-style items (e.g. The information about youth and family needs that was presented today accurately reflects my experience) to which caregivers rated how much they agreed or disagreed.

Results
This work included three key steps: (1) assessment of caregiver-reported youth and family needs during National Evaluation interviews; (2) collection of caregiver feedback regarding the needs assessment findings, groupings, and conclusions via a focus group; and (3) collection of youth feedback via a focus group. Results will be presented in this order.   
Needs identified by caregivers: Qualitative interview items
The 216 caregivers provided 394 total responses, and the coding process yielded ten categories from these responses. The most common caregiver-reported needs for youth were Services (e.g., therapy, mentoring) and Life Management (e.g., social skills training, activities with peers, anger management) and the most common needs for families were Financial (e.g., financial assistance, employment) and Services (e.g., therapy or counseling, support services, respite care).  Table 1 lists the categories identified and the frequencies of youth and family needs reported for each category. Examples of specific needs that fall under each category are provided in Table 2.

Caregiver Feedback
Overall, participant caregivers expressed that the ten categories represented their own current needs. They agreed that Services and Financial needs are the most common and most important to address in the community. Caregivers also confirmed that the categories identified by the evaluation team effectively described the individual caregiver responses.
A common trend emerged from the focus group when discussing the Services category.  Caregivers voiced frustration with the lack of availability and consistency of services, highlighting that services were not available, not affordable, or difficult to obtain for their children and families. Others reported that even those services that were offered were not always available on a consistent basis. For example, one caregiver stated that, “My problem is tapping into the services.”  Another caregiver described the lack of availability of mentors by stating that: There’s no father figure or other family members at home. It would be a great opportunity for her to connect to a mentor.” Finally, afterschool programs and extracurricular activities were noted by caregivers as largely inaccessible. They emphasized the need for affordable programs for their children.

On the focus group evaluation, caregivers not only confirmed the categories discussed in the focus group, they noted the importance of the current research in addressing community issues. Caregivers also indicated that they felt they had a voice in the focus group process and that holding more focus groups would be beneficial to improve the provision of services to their children and families. One caregiver in particular reaffirmed the importance of the focus group process: “It was really good information. It was good to see that MeckCARES is really trying to help their families.”

Youth feedback
Youth similarly confirmed that current needs were well represented by the data presented. Youth also agreed with many of the categories derived by the evaluation team. However, in contrast to the caregivers’ focus group, youth believed that modifications should be made to some of the category labels. For example, youth believed that Essential needs were better termed Mandatory needs because youth perceived these as basic necessities for all people. Youth also disagreed with the grouping of certain items within various categories. Many of these disagreements were based on the participants’ experiences and preferences. For example, many participants saw completion of high school, transportation, and access to technology as belonging in the Essential category. Also diverging from their caregivers’ views (i.e., rating Services and Life Management as the most common and important unmet needs), youth indicated that Essential and Family Support needs were most important. Youth emphasized that Essential needs must be primarily addressed for all people, and that Family Support was most important once Essential needs were met. Finally, youth stated that different definitions of family were important to consider, for example, including foster families in the definition of a family.

Discussion
The current study provided youth and caregivers the opportunity to voice their opinions about their needs, including available (or unavailable) services and supports within a SOC community. The National Evaluation of SOC has not previously incorporated an assessment of current needs of its youth and caregiver participants, thus leaving families unable to express their voices on these important issues. The current study addressed this gap by examining caregiver self-reported needs of youth and families enrolled in the MeckCARES SOC. The two brief items that were integrated yielded information beyond the standardized, required protocol, providing relevant data for system administrators and community stakeholders.

Notably, in a system designed to provide a comprehensive array of services and supports to the youth and families served, “services” were the primary caregiver-identified need for both youth and families. While these findings may not necessarily generalize to other sites, they suggest that steps such as building structures, dedicating resources to training, and even adding services do not necessarily translate into available, accessible, adequate, or effective services and supports for those in need. Critically, they also highlight the need for ongoing evaluation of needs, processes, and outcomes within these system change initiatives. Moreover, that caregivers also regularly reported needs that went well beyond traditional formal services (e.g., educational and life management needs for youth; financial needs of the family) is consistent with prior calls to attend to the broader ecological contexts of these children and families and address issues that may extend past the traditional purview of “mental health” (see, e.g., Brashears, Davis, & Katz-Leavy, 2011; Cook & Kilmer, 2010a; Kilmer et al., 2010).

Although limited by the relatively small focus group samples, the focus group findings affirmed caregivers’ perceptions of Services and Life Management as the most common needs of their children. However, the youth differed on what they thought were their most important needs, indicating that Essential needs and Family Support were of greatest salience. The youth focus group, in particular, helped draw attention to their distinct views and underscored the importance of including a focus group or other opportunity for them to provide their inputs. Furthermore, the caregivers appreciated having a forum in which their voices could be heard.

Future research efforts could examine caregivers’ specific concerns about services. For example, it would be useful to determine which services are inaccessible or unaffordable and for what reasons, whether the quality of services is of concern, or whether families are aware of all service options. Future research could also attempt to connect data regarding reported needs and, subsequently, the degree to which those needs have been met or unmet, with indicators of youth and caregiver functioning (see, e.g., Kilmer & Gil-Rivas, 2010 for an example in another context).

Because these findings have neither been replicated nor linked to other measures of family resources, functioning, and the like, the present results should be viewed as preliminary and conclusions must be drawn with care. That said, notwithstanding those caveats, these results have potential implications for advocacy efforts and systems change. For example, the needs data can help guide future ParentVOICE efforts to concentrate on needs of the youth and families. In addition, the findings of this study, in combination with future research, could aid in the modification and implementation of services and programs. As one case in point, the results of the current study were presented to the MeckCARES community collaborative, a state-mandated decision making body, with a mission centered on making programmatic recommendations to enact systemic changes. In response to this data presentation, the categories from the needs assessment data were incorporated into the MeckCARES enrollment form for early identification of current needs of youth and families as part of a local agency’s pilot SOC effort. Such efforts have the capacity to inform the work of local Child and Family Teams and improve the delivery of wraparound care planning and treatment.

The project reported here was developed (after hearing repeated caregiver concerns about the system’s functioning) to help capture the views and voices of those involved in a local SOC.  Such steps are consistent with the values of SOC and are necessary to access authentic representations of typically disenfranchised families’ experiences (e.g., Slaton, Cecil, Lambert, King, & Pearson, 2011). Indeed, working to address the gaps in standard evaluation protocols can increase the ecological validity of applied research and, in turn, contribute to well-targeted actionable recommendations to improve systems and benefit the youth and families served. 

References
Brashears, F., Davis, C., & Katz-Leavy, J. (2011). Systems of care: The story behind the numbers. American Journal of Community Psychology. doi: 10.1007/s10464-011-9452-z.

Bruns, E. J., Suter, J. C., Force, M. M., & Burchard, J. D. (2005). Adherence to wraparound principles and association with outcomes.  Journal of Child and Family Studies, 14, 521-534.

Burns, B. J., and Goldman, S. K. (Eds.) (1999). Promising practices in wraparound for children with serious emotional disturbance and their families. Systems of Care: Promising practices in children’s mental health, 1998 Series, Volume IV. Washington, D.C.: Center for Effective Collaboration and Practice, American Institutes for Research.

Bruns, E. J., Walker, J. S., Zabel, M., Matarese, M., Estep, K., Harburger, D., et al.  (2010).  Intervening in the lives of youth with complex behavioral health challenges and their families: The role of the wraparound process.  American Journal of Community Psychology, 46, 314-331. 

Cook, J. R., & Kilmer, R. P. (2004). Evaluating systems of care: Missing links in children’s mental health research. Journal of Community Psychology, 32, 655-674.

Cook, J. R., & Kilmer, R. P.  (2010a). Defining the scope of systems of care: An ecological perspective.  Evaluation and Program Planning, 33, 18-20. 

Cook, J. R., & Kilmer, R. P. (2010b). The importance of context in fostering responsive community systems: Supports for families in systems of care. American Journal of Orthopsychiatry, 80, 115-123.

Cook, J. R., Kilmer, R. P., DeRusso, A., Vishnevsky, T., & Meyers, D. C. (2007). Assessment of child and family team functioning using the Participant Rating Form.  In C. Newman, C. Liberton, K. Kutash, & R. Friedman (Eds.), 19th Annual Research Conference Proceedings: A system of care for children’s mental health: Expanding the research base (pp. 317–322). Tampa, FL: University of South Florida, The Louis de la Parte Florida Mental Health Institute, Research and Training Center for Children’s Mental Health.

Epstein, M. H., Nordness, P. D., Kutash, K., Duchnowski, A., Schrepf, S., Benner, G., et al. (2003). Assessing the wraparound process during family planning meetings. Journal of Behavioral Health Services & Research, 30, 352–362. 

Gabhainn, S. N., Dolan, P., Canavan, J., & O’Higgins, S.  (2009).  Current practice in meeting child health needs in family support services: Variation by service type and perspectives on future developments.  Child Care in Practice, 15, 109-125. 

Graves, K. N. (2005). The links among perceived adherence to the system of care philosophy, consumer satisfaction, and improvements in child functioning. Journal of Child and Family Studies,14, 403–415. 

Hodges, S., Ferreira, K., Israel, N., Mazza, J. (2010). Systems of care, featherless bipeds, and the measures of all things. Evaluation and Program Planning, 33, 4-10.

Holden, E. W., Santiago, R. L., Manteuffel, B. A., Stephens, R. L., Brannan, A. M., Soler, R., et al.  (2003).  Systems of care demonstration projects: Innovation, evaluation, and sustainability. In A. J. Pumariega & N. C. Winters (Eds.), The handbook of child and adolescent systems of care: The new community psychiatry (pp. 432–458). San Francisco, CA: Jossey-Bass.

Huang, L., Stroul, B., Friedman, R., Mrazek, P., Friesen, B., Pires, S., et al. (2005). Transforming mental health care for children and their families. American Psychologist, 60, 615-627.

Kilmer, R.P., Cook, J.R., & Palamaro Munsell, E.  (2010).  Moving from principles to practice: Recommended policy changes to promote family-centered care.  American Journal of Community Psychology, 46, 332-341.

Kilmer, R.P., & Gil-Rivas, V. (2010). Responding to the needs of children and families after a disaster: Linkages between unmet needs and caregiver functioning. American Journal of Orthopsychiatry, 80, 135-142.

Knitzer, J.  (1982).  Unclaimed children: The failure of public responsibility to children and adolescents in need of mental health services.  Washington, DC: The Children’s Defense Fund. 

Pumariega, A. J., & Winters, N. C. (Eds.). (2003). The handbook of child and adolescent systems of care: The new community psychiatry. San Francisco, CA: Jossey-Bass.

Slaton, A.E., Cecil, C.W., Lambert, L.E., King, T., & Pearson, M.M. (2011). What a difference family-driven makes: Stories of success and lessons learned. American Journal of Community Psychology.

Stroul, B. A., & Friedman, R. M. (1986). A system of care for children and youth with severe emotional disturbances. (Revised 1994 Edition). Washington, D.C.: Georgetown University Child Development Center, CASSP Technical Assistance Center.

The U.S. National Wraparound Initiative, led by Eric Bruns, Ph.D., Janet Walker, Ph.D., Trina Osher, Jim Rast, Ph.D. (2011). Wraparound basics. Retrieved from http://www.nwi.pdx.edu
U.S. Department of Health and Human Services. (2003). CMHS national evaluation: Data profile report. Atlanta, GA: ORC Macro.

Walker, J. S., & Schutte, K. M. (2005). Quality and individualization in wraparound planning.  Journal of Child and Family Studies, 14, 251–267.

Walker, J. S., Bruns, E. J., VanDenBerg, J. D., Rast, J., Osher, T. W., Miles, P., Adams, J., & National Wraparound Initiative Advisory Group (2004). Phases and activities of the wraparound process. Portland, OR: National Wraparound Initiative, Research, and Training Center on Family Support and Children’s Mental Health, Portland State University.

autores

Katherine P. Strater, Melissa Strompolis, Ryan P. Kilmer, James R. Cook

Katherine P. Strater, BA, is a doctoral student at the University of North Carolina at Charlotte. kpstrate@uncc.edu

Melissa Strompolis, MA, is a doctoral student at the University of North Carolina at Charlotte. mstrompo@uncc.edu

Ryan P. Kilmer, Ph.D., is an Associate Professor of Psychology at the University of North Carolina at Charlotte.  rpkilmer@uncc.edu

James R. Cook, Ph.D., is a Professor of Psychology at the University of North Carolina at Charlotte and President-Elect of the Society for Community Research and Action.  jcook@uncc.edu
 

comentarios (0)

agregar comentario