Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Predicting patterns of service utilization within children’s mental health agencies

BMC Health Services Research volume 19, Article number: 993 (2019) Cite this article

Abstract

Background

Some children with mental health (MH) problems have been found to receive ongoing care, either continuously or episodically. We sought to replicate patterns of MH service use over extended time periods, and test predictors of these patterns.

Methods

Latent class analyses were applied to 4 years of visit data from five MH agencies and nearly 6000 children, 4- to 13-years-old at their first visit.

Results

Five patterns of service use were identified, replicating previous findings. Overall, 14% of cases had two or more episodes of care and 23% were involved for more than 2 years. Most children (53%) were seen for just a few visits within a few months. Two patterns represented cases with two or more episodes of care spanning multiple years. In the two remaining patterns, children tended to have just one episode of care, but the number of sessions and length of involvement varied. Using discriminant function analyses, we were able to predict with just over 50% accuracy children’s pattern of service use. Severe externalizing behaviors, high impairment, and high family burden predicted service use patterns with long durations of involvement and frequent visits.

Conclusions

Optimal treatment approaches for children seen for repeated episodes of care or for care lasting multiple years need to be developed. Children with the highest level of need (severe pathology, impairment, and burden) are probably best served by providing high intensity services at the start of care.

Peer Review reports

Predicting patterns of service utilization within children’s mental health agencies

The natural history of mental health (MH) problems suggests that a sizable percentage of children with MH problems might need ongoing care, either continuously or episodically. There is extensive evidence of the continuity from childhood to adolescence for both externalizing [i.e., attention deficit hyperactivity (ADHD), oppositional defiant (ODD), and conduct disorders] and internalizing problems (i.e., anxiety, depression; [1, 2]). Many children have either ongoing MH problems or experience recurrent episodes of MH problems. Ongoing problems with depression occur for 10–18% and for anxiety, 41–66% [3]. Recurrence rates for depression are 50–70% in natural history studies and 15–47% of children have a relapse following treatment [3]. It is common for ADHD to persist for years [4] with 28–60% of children continuing to have ADHD for many years [5,6,7]. Similarly, about 20% of children have persistent ODD [8] while 50% may have other types of MH problems years later [9].

This presents a problem for how child and youth mental health service (CYMHS) delivery systems can best care for these children. Evidence-based treatments (EBTs) exist for the most common MH problems seen in CYMHS agencies (e.g., [10]). However, we know of no specific protocols outlining how best to care for children who might present for repeated episodes of care (EoC), or for whom treatment lasts years, rather than months. In general, little is known about how children with MH problems use services over extended time periods. We also need to be able to identify predictors of divergent care experiences, as a first step in developing protocols to care for children with ongoing or episodic MH care needs. The current study adds to the few studies (e.g., [11,12,13,14,15]) that have examined service use over multiple years and examines predictors of patterns of service use.

Patterns of service use over extended time periods

Paucity of research on children’s use of MH services over extended time periods is likely due to challenges conducting such research. A full understanding of service use over extended time periods requires multiple years of the following data, for each episode of care, for all children seen within a CYMHS agency: (a) problem type and severity at the start of services, (b) CYMHS received (e.g., was an EBT received), (c) predictors of treatment engagement and/or effectiveness (e.g., sociodemographics, therapeutic alliance), and (d) problem severity and (e) disposition (e.g., drop out, mutual agreement to end services) at the end of services. It is doubtful that any agency would have such a dataset. For example, in the UK Child Outcomes Research Consortium [16] dataset (> 250,000 cases), “only 24% [of cases] have meaningful outcome data” (p. 300). Prospective studies could obtain a number of the above variables, but recruitment and low participation rates may be a problem [17, 18]. For example, only 50% of youth participated in the Patterns of Care study in the US [18]. Loss to follow-up is a further problem in prospective studies (e.g., [19,20,21]) or studies recruiting clients after discharge [22]; these rates vary from about 30% [19] to over 90% [22]. In light of these challenges, we used administrative data from CYMHS agencies to examine patterns of service use over multiple years. This provided complete visit data on all children, but limited amounts of clinical information. We used a person-centered approach beginning with the child’s first visit and examining all visits within the next 4 years to better capture how families experienced services over time. When fiscal or calendar years are used (e.g., [23, 24]), children can be at various points in their care within a year, and thus the child/family’s pattern of service use over time is obscured.

Most studies that examined patterns of children’s MH services used periods of 1 year or less [11, 12, 19, 20, 25, 26]. Three studies had longer time frames and examined different aspects of service use: (a) 10–15 year retrospective parent-reports of services [13]; (b) rural-urban differences in service use based on 5 years of administrative data, [14]; (c) changes in service use with wraparound treatment based on 6 years of claims data [15]. The current study adds to the literature by examining all services used by children seen in five CYMHS agencies over a 4-year time period.

Our research team recently completed a study examining the patterns of service use over a 5-year time period for children age 4- to 11-years old at the time of their first visit to one of six CYMHS agencies in Ontario in 2000, − 01, or − 02 [27]. We identified five patterns of service use that were stable across agencies and across time (i.e., intake years 2000–02). For example, the Minimal care group (48% of cases) had the shortest duration of services and fewest visits. Two service use patterns (19% of cases) characterized children who had multiple EoCs; an EoC was defined as a minimum of three visits with at least 6 months in between the end of one EoC and the start of the next [28]. We do not know if these patterns were unique to these agencies or the study time-period. Thus, the first aim of the present study was to see if we could replicate these patterns of service use in a new sample of children and youth.

Predicting patterns of service use

The ability to identify predictors of divergent care experiences is an important first step in developing better ways to provide CYMH services. Our previous study, however, found few child or family characteristics differentiated patterns of care [27]. Thus, the second aim of the current study was to examine predictors of patterns of service use.

Andersen’s behavioral model of health services utilization framed our choice of predictors [29, 30]. We expected that service utilization would be predicted by predisposing (e.g., sex, age), enabling (e.g., burden of illness), and need (e.g., child psychopathology) factors. There have been few studies examining predictors of service use over extended time periods. Studies of accessing care have found mixed relationships for child age [31,32,33] and sex [34, 35] as predictors. Greater severity of psychopathology tends to be associated with service use [31, 35], as have higher levels of parental burden [36]. In light of these findings, we hypothesized that more severe child psychopathology and higher family burden would be associated with more intensive patterns of service use; i.e., patterns characterized by longer durations of care, more visits, and repeated EoCs. Descriptive statistics related to service use (e.g., total number of visits, duration of involvement, episodes of care) for each pattern were examined to inform the labelling and interpretation of the patterns.

Methods

We conducted secondary data analysis of administrative data for 2004–2010 from five CYMHS agencies that: (a) provided services for children and youth (age 5–18 years), and (b) were accredited by Children’s Mental Health Ontario. We purposely selected CYMHS agencies in Ontario that are located in, and serve, both rural and urban populations, and that were willing to partner with us. Access to publicly-funded CYMH care in Ontario is based on need, and does not depend on access to insurance coverage and/or the ability to pay, or require a diagnosis, and is available to any child or youth within the catchment area of the agency. In Ontario, specialized CYMHS for 0–18 year olds has been delivered by agencies funded by a ministry other than health [37] and has operated separately from MH care provided by family physicians, pediatricians and psychiatrists, which is also free and covered under provincial healthcare plans [38, 39].

Participant selection

Inclusion criteria were as follows: [1] the first face-to-face visit to the agency occurred in 2004–2006 (operationally defined as having no face-to-face visit in the previous 18 months); and [2] clients were between the ages of 5 to 13 years at the time of the first face-to-face visit. We used an upper age of 13 years to ensure that it was possible for the oldest children in the study to receive services 4 years after their first visit before potentially aging-out and/or being transferred to adult services at 18 years of age. We used a lower age of 5 years because the infant and preschool version of the Brief Child and Family Phone Interview (BCFPI; [40, 41]) did not yet exist [42]. We examined 4 years of data following the child’s first visit to the agency since most differences between service use patterns in our previous study [27] were evident within 4 years, with few additional differences evident in year 5. There were 12,643 cases that met the first criterion; the 24% (n = 3099) of cases who were less than age 5 and the 21% (n = 2711) who were older than 13 were removed.

Exclusion criteria were: [1] children with (a) a pervasive developmental disorder (i.e., Asperger’s syndrome, autism), or (b) a developmental disability (e.g., Down syndrome), or children who received services from a program specializing in these disorders at the agency at any point within the study window (n = 841). These cases were excluded, as children with these problems are known to need ongoing care and treatment models sensitive to ongoing care needs already exist [2, 43,44,45]. Clients who had an EoC within the previous 18 months were also excluded (n = 360); this meant that children were clearly starting a new EoC. Other studies have used a much shorter period of time (e.g., 12 weeks [46]; first 60 days of the year [25]) for excluding cases, or have not specified if children were excluded based on previous visits (e.g., [47, 48]).

Administrative data

Raw data in electronic format were received from each participating CYMHS agency including: child date of birth, sex, and visit data [e.g., date, type of contact (e.g., telephone, in-person, in-home visit)]. Only face-to-face visits were included. Telephone contacts were excluded because it was unclear whether these contacts were for administrative purposes (e.g., rescheduling appointments), or if treatment was provided. All non-direct contact appointments were also excluded (e.g., scheduling, report writing).

To examine patterns of service use, the first aim of the study, the date of each child’s first visit was set to a visit in ‘month 1’. Visit data were then recoded to indicate whether or not a child had been seen within each month in the 48 months (4 years) following the child’s first in-person visit.

For descriptive purposes, the total number of visits and duration of involvement (i.e., time between first and last visit), and percentage of cases with involvement lasting more than 2 years were calculated. The volume of services used for each pattern was calculated as a percentage as follows: the number of visits was summed across clients and agencies within each of the five patterns, and divided by the sum of all visits for all clients and agencies. To examine the intensity of service use, the total number of visits per year was examined. Visits were also organized into EoCs, using the previously published definition of an EoC [28].

Predictor variables

Child date of birth and sex were obtained from the administrative data. Date of birth was used to compute child’s age at the time of the first visit to the agency.

Brief child and parent phone interview (BCFPI)

The BCFPI assesses child psychopathology and other factors known to influence treatment engagement (e.g., impact of illness on the family), and was a mandated intake measure at Ontario CYMHS agencies between 2001 and 2015 [49]; due to a shift in government policy, its use was not mandated after 2015 (M. Barwick, personal communication, Nov 2019). The BCFPI assessment that was closest to the date of the child’s first in-person visit was used; i.e., within 8 months before the first visit or 1 month after. Four composite scales were calculated by combining multiple subscales [41]. Externalizing Behavior - 18 items related to regulating attention and impulsivity, cooperativeness, and conduct; Internalizing Behavior - 18 items related to managing anxiety and mood, as well as separation from parents. The Global Child Functioning scale (hereafter Child Impairment) is an index of impairment that measures social participation, quality of relationships, and school performance and achievement. Finally, the Global Family Situation scale (hereafter Family Burden), which reflects the burden of illness for the family, includes items related to the impact of the child’s problems on family activities (i.e., external family functioning) and family comfort (i.e., internal family functioning). The BCFPI psychopathology scales have good internal consistency (α ≥ 0.74) and test-retest reliability (r ≥ 0.54; [50]). In clinical samples, convergent validity include correlations with symptom counts on a diagnostic interview (r = 0.68–0.78); and confirmatory factor analyses provide support for construct validity [51]. Using age- and sex-based norms, raw scores were converted to T-scores based on comparisons to the general population. Groups were formed based on T-scores above or below the clinical cut-off (T-score ≥ 65; 93rd percentile).

A missing values analysis (conducted in SPSS v24 [52]) was completed on available BCFPI data. Missing values ranged from 0.3% (Externalizing Behavior) to 12.4% (Family Burden). Little’s MCAR test indicated that these missing values were not random [χ2(794) = 1073.811, p < 0.001]. Missing BCFPI values were imputed using an Expectation-Maximization algorithm, which iteratively verifies imputed values against other variables to reach the most likely value, preserving the relationships between variables.

Data analysis

Examining patterns of service use: latent class analysis

Multi-level latent class analysis (LCA; Latent Gold v. 4.5) was applied to data across all agencies to determine patterns or classes of service use [53]. LCA is a probabilistic model whereby for each participant the posterior probability of membership in each class is computed. Given the large sample size, even small likelihood-ratio (LR) chi-square values tend to be significant (indicating departure of observations from the model). Thus, we used common model fit criteria (i.e., Bayesian Information Criteria, BIC; Akaike Information Criteria, AIC; Consistent Akaike Information Criterion, CAIC) to determine the optimal number of classes [54, 55]. Given the sample sizes, small changes in the model fit might be suggested; thus, models ranging from 2 to 10 classes were applied to the data (e.g., [56]). Given that there are no established guidelines for what constitutes a substantive improvement in model fit for LCA, we used an a priori criterion to consider models with improvements in fit of 2% or more (averaging across the three fit criteria) over previous models to determine the optimal number of classes; we used this same criterion in our previous manuscript [27]. The program was set to start computations at 10 random points, to minimize the likelihood of deriving an unrepresentative local solution. For each solution, we specified 250 Expectation-Maximization iterations, followed by 50 Newton-Raphson iterations to optimize the class allocation [53]. A total of 500 bootstrap replications were computed. Visit descriptives for the different class solutions were reviewed with respect to substantive implications for understanding service use. Based on our previous study, it was hypothesized that five patterns of service use would be identified [27]. After selecting the number of classes, the contribution of the three client cohorts (defined by the calendar year of their first recorded visit; 2000, − 01, − 02), or the five CYMHS agencies in multi-level LCA was examined. The inclusion of these factors did not substantively improve the model fit; thus, the final LCA model did not include cohort or agency. The entropy value, an indicator of the certainty of classification of cases into classes, is reported; entropy values approaching 1.0 indicate clear delineation of classes [57]. Using probabilities estimated from the model, each client was allocated to a latent class [58].

Service use characteristics

Service use characteristics by pattern are reported. Differences in these variables between patterns of service use were examined using chi-square analysis or ANOVA, with Bonferroni post-hoc comparisons as applicable.

Predicting patterns of service use

Discriminant function analysis (DFA) was used to explore whether pattern of service use could be distinguished based on a combination of variables, including demographic factors (e.g., sex, age at first visit) and measures of psychopathology, child impairment, and family burden (i.e., BCFPI composite scales). Analyses were conducted in SPSS v24 [52]. A priori probabilities of assignment to patterns in classification were used. Wilks’ lambda is a ratio of the error variance to the pooled variance plus error variance and is the preferred statistic to measure model fit in DFA [59]; it is used to determine whether functions are significant, based on improvement in model fit (i.e., decreased values). Cross-validation was done using a jackknife, or leave-one-out, procedure within the Discriminate procedure [60, 61].

Descriptive analyses for predictors of service use

Descriptive statistics for predictor variables by pattern are reported. Again, differences in these variables between patterns of service use were examined using chi-square analysis or ANOVA, with Bonferroni post-hoc comparisons as applicable.

Results

The final sample included 5632 children (62.3% male) whose age (years) at the time of their first visit was: 5 (8%), 6 (10%), 7 (9%), 8 (11%), 9 (12%), 10 (12%), 11 (12%), 12 (13%), or 13 (14%).

Examining patterns of service use: latent class analysis (LCA)

Model fit improved in the LCA with the addition of each class from 2 to 10; however, the percentage improvement in the fit indices was less than 2% for models with more than 5 classes. Thus, the 5-class model was retained (see Additional file 1: Table S1). Multi-level modelling with the inclusion of cohort year (i.e., year of first visit – 2004, − 05, − 06) and agency did not result in improvements in the model fit. Thus, all analyses are reported collapsing across agencies and cohort years.

Figure 1 shows the probability of visits over 4 years by classes. The classes were labelled based on the probability of visits over time and the descriptive data on duration of involvement, number of visits, and EoCs (see Table 1): Minimal (53.2% of cases); Brief Episodic (7.9%); Acute (20.1%); Intensive (13%); and Ongoing/ Intensive-Episodic (5.8%).

Fig. 1
figure 1

Patterns of service use based on five child and youth mental health service agencies across 4 years. The y-axis shows the probability of a child having at least one visit in each month. The x-axis shows months, starting with month 1 (the child’s first visit at the agency) through to the end of the observation period (4 years after the child’s first visit). For example, at month 11 a child in the Ongoing/Intensive-Episodic class had a 0.63 probability of a visit; in other words, 63% of this group of children were seen in month 11. At month 21, a child in the Intensive group had a 0.22 probability of being seen, whereas a child in the Brief Episodic group had only a 0.08 probability of being seen. Classes were labelled based on examination of the probability of visits over time and descriptive statistics on the overall duration of involvement and number of visits, along with the number and distribution of episodes of care (see Table 1 and Additional file 1)

Full size image
Table 1 Service Use and Demographic Characteristics by Pattern of Service Use
Full size table

Service use characteristics

Differences in service use characteristics were evident across the patterns (see Table 1; Additional file 1: Tables S2-S4 provide additional details on service use by EoCs). The number of EoCs [χ2(12) = 4125.04, p < 0.001] and duration of involvement varied across patterns [F(4, 5627) = 2676.81, p < 0.001; all pairwise comparisons p < 0.05]. Two groups tended to have multiple EoCs - the Brief and Ongoing/Intensive-Episodic patterns. Within the Brief Episodic group, close to one-third (31.8%) of children had, on average, 1.7 visits spread over 4.7 months before starting their first EoC 2 years later; during their first EoC they had 12.9 visits within about 6 months, followed by close to 2 years (23.3 months) without a visit, and then a second EoC of 18 visits over 8 months. Within the Ongoing/Intensive-Episodic group, only 10% of children had visits before starting their first EoC; during their first EoC they had 62.4 visits over about 2 years, which for 54% of cases was their only EoC. For the remaining 46% of cases, their second EoC occurred after 9 months without a visit, and included 48 visits over 16 months. Interestingly, across the entire sample, 7.5% of cases returned about 21 months after the last visit in their last episode of care for an average of 1.4 visits (see Additional file 1). Finally, with respect to the total volume of services delivered, the Ongoing/Intensive-Episodic group accounted for 32%, and the Intensive group 26%, of all visits for the entire sample (see Table 1).

Predicting patterns of service use

Of the 5632 cases in the final sample, 3344 (59.4%) had a BCFPI. The percentage of cases having a BCFPI was virtually identical to the percentage of cases within each of the five patterns of service use (χ2 = 6.9, p = 0.139). Thus, the likelihood of a child having a BCFPI was not differentially related to future service use. Cases with a BCFPI were older (M = 10.2) than those without [M = 9.5; t (4107.16) = 9.958, p < 0.001], but did not significantly differ in terms of sex. Results from the DFA and descriptive analyses informed prediction of the patterns.

Discriminant function analysis (DFA)

Age, sex, and BCFPI composite scales were entered into a DFA. Two functions significantly discriminated between the five patterns, accounting for 94.5% of the explained variance (see Table 2). The overall chi-square test was significant, although with a high Wilks’ lambda (Wilks λ = 0.938, χ2 = 213.175, df = 24, p < 0.001), indicating that the model significantly predicts group members but a high proportion of variance is not accounted for by group membership [62]. Lambda improved when a second function was added (Wilks λ = 0.992, χ2 = 27.345, df = 15, p = 0.026; [62]). Classification of cases based on the new canonical variables correctly classified 51.6% of cases (51.4% in cross-validation).

Table 2 Predictors of Pattern of Service Use, Standardized Canonical Discriminant Function Coefficients and Structure Coefficients
Full size table

The first discriminant function (see Fig. 2 and Table 2) showed high Externalizing, Child Impairment, and Family Burden scores best differentiated the groups, while on the second function, primarily older age and, to a lesser degree, higher Internalizing scores did. Children in the Minimal pattern of service use were most distinguishable from those in the other groups as being lowest on Function 2 (reflecting lower Internalizing scores). The Brief Episodic group was most clearly differentiated by having the youngest children. The Acute and Intensive groups were similar on both functions. Finally, the Ongoing/Intensive-Episodic group had the highest mean scores on both functions, reflecting the highest levels of psychopathology, Child Impairment, and Family Burden.

Fig. 2
figure 2

Patterns of service use group centroids on two functions based on discriminant function analysis. The standardized discriminant function coefficients and structure coefficients are reported in Table 2. Scores above zero on Function 1 reflect higher levels of externalizing, impairment (i.e., greater impairment), and family burden. Scores greater than zero on Function 2 reflect older age and higher internalizing scores. Tables 1 and 3 present descriptive statistics for age and the Brief Child and Family Phone Interview (BCFPI) variables by class

Full size image

Descriptive analyses by pattern

Age at intake differed across patterns [F(4, 5627) = 11.8, p < 0.0001; see Table 1]; children in the Brief Episodic service use pattern were younger than in all other patterns (post-hoc p < 0.05), and those in the Minimal pattern were significantly older than those in the Intensive pattern (p = 0.003). Sex was associated with pattern of service use [χ2(4) = 11.279, p = 0.024]; however, the only significant subgroup difference was that there were proportionally more boys in the Intensive pattern of service use than expected [Adjusted residual χ2(1) = 11.022, p < 0.001].

On the BCFPI, the Minimal pattern had significantly lower (p < 0.01) Externalizing, Child Impairment, and Family Burden scores than all other use patterns (see Fig. 3, Table 3). Children with this pattern also had significantly lower (p < 0.01) Internalizing scores than the Intensive and Ongoing/Intensive-Episodic patterns. Children in the Acute pattern had lower (p < 0.05) Externalizing scores than the Ongoing/Intensive-Episodic pattern. The Ongoing/Intensive-Episodic pattern had higher (p < 0.05) Child Impairment scores than all other patterns except the Intensive group, and higher Externalizing than the Minimal and Acute groups.

Fig. 3
figure 3

Average Brief Child and Family Phone Interview (BCFPI) composite scale scores by service use pattern

Full size image
Table 3 Clinical Characteristics of Sample for Patterns of Service Use across the Five Child and Youth Mental Health Service Agencies
Full size table

Discussion

Five patterns of services use emerged from visit data for 5–13 year olds first seen in 2004, − 05, or − 06 at one of five CYMHS agencies in Ontario. Thus, for the first aim of the study we replicated five patterns of service use that were virtually identical to a previous study [27], which included cohorts of children first seen in 2000–02. The percent of cases classified into each pattern varied by only 3% (Additional file 1: Table S5 presents descriptive statistics for the two studies). Differences may be due to a shorter study duration (4 years vs 5), or a secular trend towards briefer treatment.

The second aim of the study was to examine predictors of patterns of service use. The accuracy in predicting children’s pattern of service use based on their demographic information (age) and BCFPI scores (Internalizing, Externalizing, Child Impairment, Family Burden) at the time of their first visit was just over 50%. Lower levels of Externalizing, Child Impairment, and Family Burden differentiated the Minimal care pattern from all others. In contrast, the Ongoing/Intensive-Episodic pattern had the highest levels on the BCFPI variables, with Child Impairment being significantly higher than all other patterns except the Intensive group, and higher Externalizing than the Minimal and Acute groups. There were few differences between the Brief-Episodic, Acute, and Intensive groups.

Some findings in the current study were not surprising. For example, in the Acute pattern, children tended to be seen for one EoC lasting about 10 months with an average of about 16 visits. Although we were unable to document the type of treatment children received, this pattern was the most aligned to the provision of an EBT protocol. For comparison, in a US study where agency staff in community CYMHS agencies delivered two EBTs, clients received an average of 16 treatment sessions delivered over 196–210 days [63].

We first discuss the Minimal care pattern, as it reflects 50% of all cases. We then focus on episodic vs ongoing service use, as little is known about how to best care for children receiving services in these ways. Findings related to predicting patterns are discussed, where relevant. We conclude with a discussion of limitations and over-arching implications for CYMHS.

Minimal vs episodic vs ongoing service use

Minimal care

Over 50% of cases were seen for only a few visits. These results are very similar to other studies. For example, analyses of US private health insurance data for child and youth mental health services found that 45% of cases were seen for less than 1 month, and 78% for less than 6 months [25]. In the Fort Bragg study, 61% of cases were treated for less than 6 months [64]. In a community-based prospective survey, 47–51% of children and youth had fewer than four counselling visits with a mental health specialist [46]. Private insurance might place limits on treatment duration [25, 46], resulting in a large number of cases with short treatment durations. This was not true in the Fort Bragg study [65], nor is it true in the Ontario system.

It is not clear whether shorter treatment durations and/or few sessions reflect inadequate care, as has been suggested by some authors. For example, Saloner et al. [46] used eight sessions as the minimum number of mental health visits. Eight sessions has been recommended as the minimum for adolescent depression, but the authors noted that “no meta-analyses of brief counseling treatments exist for adolescent populations” ([66], p. 100). Contrary data are provided in another treatment study for adolescents with depression; 31% of cases demonstrated substantive improvement (> 50%) by the second treatment session, without increased risk of poor outcomes, up to 2 years post treatment [67].

Minimal service use patterns may reflect well-known problems with client engagement in CYMHS [68,69,70]. However, with more than half of all children falling into the Minimal pattern, this finding also supports the need for brief intervention services and walk-in clinics [71,72,73]. In Ontario’s most recent child mental health treatment plan [74], brief services are one of seven core services to be available at all CYMHS agencies in the province. Children in the Minimal service use pattern had the lowest level of family burden, child impairment, and externalizing behaviors, and the fact that few children/families in this pattern returned for care in the following 4 years, suggests that just a few appointments may be sufficient to meet the needs of these families. Thus, the label “Minimal” should not be interpreted to mean inadequate or inappropriate care. Rather, it captures the relative number of visits and duration of involvement of this group of children compared to the other four patterns of service use. Research and practice would benefit from ongoing outcome monitoring and targeted studies on brief services to help inform our understanding of the appropriateness of brief treatments in CYMHS [75, 76].

Episodic service use

In this and our previous study, 15–20% of children were seen for more than one EoC within a 4–5 year period [27]. The Brief Episodic pattern of service had the highest percentage of children having two or more EoCs. Clinically, they were similar to children in the Acute and Intensive service use groups, as family burden, child impairment, and externalizing behaviors did not differentially predict these three patterns.

Only one other study reported similar data on EoCs. Warren et al. [77] analyzed administrative data from 1997 to 2008 for 4–17 year olds seen in either a community mental health agency (N = 3524) or a managed care setting. They reported that the average number of EoCs was 1.9 in the community agency. EoCs were, on average, 9.5 weeks long and consisted of 2.9 sessions. However, this study had a markedly different definition of an EoC; Warren et al. defined an EoC as 90 days without a visit, which presumably meant that an EoC could be only one visit, whereas we defined an EoC as a minimum of three visits with 180 days with no visits in between episodes [77]. There have been other studies that discuss EoCs, but again, data are not directly comparable. These studies focus on specific diagnostic groups (e.g., depression [78, 79]), medication use (e.g., for ADHD [80]), used much shorter time frames (e.g., 180 days, [25]), or combined health and mental health services [e.g., [46]].

What are the reasons that children receive more than one EoC? Sytema et al. [81] suggested that patterns of care “reflect both the functioning of mental health care system and the help-seeking behavior of its clients” (p. 1). The natural history of anxiety and depression is that many children re-experience these problems. In both community and clinical samples, 50–70% of youth with depression will experience another episode within 3 years; anxiety disorders also tend to be episodic [3]. If children experience disorders as waxing and waning over time, we would expect that experiencing more than one episode of an illness would lead to more than one EoC. Problems such as ADHD have been conceptualized as chronic, but treatment is often episodic. In a 7-year, population-based study of children and youth prescribed methylphenidate, one-third of cases had more than one episode (i.e., no prescriptions being filled for 4 months) of treatment, with 10.2% of cases having three or more episodes [80]. This suggests that some EoCs are due to the natural history of the underlying condition, whereas others occur due to variation in family help-seeking patterns. Episodic service use could also be due to dropout; this will be explored in a future article.

Ongoing service use

A sizeable proportion of children were seen many times; children in the Intensive and Ongoing/Intensive-Episodic patterns were seen, on average, for 33 and 87 visits, respectively. Virtually all cases (27% of the sample) within three of the service use patterns (Brief Episodic, Intensive, and Ongoing/Intensive-Episodic) had involvement for more than a year, and 24% of all cases were seen for more than 2 years.

Although there are no studies that provide directly comparable data, three studies report service use over longer time periods. Mueller et al. [82] analyzed data from children with severe mental and behavioral disorders treated within the publicly-funded system in Hawaii. After excluding all cases seen for fewer than 90 days, the average duration of a “service episode” was about one and a half years (560.2 days; SD = 372.2). While it was unclear how a service episode was defined, their data are similar to the current study in finding that many children are seen for longer than a year, and that there is considerable variability in durations of care. A 2-year follow-up from a randomized clinical trial conducted in community agencies in the US testing usual care versus standard EBTs versus modular treatment found that 13% (standard EBT) to 18% (modular) of cases received treatment from a community mental health clinic from 1 to 2 years after the trial ended. Finally, Goldstein et al. [13] reported that 50–60% of children who had anxiety or depression had “long” durations of treatment (“continuous treatment of several years or numerous brief periods”; p. 970) when assessed as young adults. Collectively, our data and those from these studies demonstrate that a sizeable percentage of children and youth are seen for years, not months, and have considerably more treatment sessions than any EBT protocol. The sector needs to consider how to best care for children whose needs extend over longer periods of time, the implications of their care in other sectors (e.g., health), and what happens to these children when they transition to adulthood.

Of note, children who were seen for the longest period of time and had the highest number of visits (Ongoing/Intensive-Episodic pattern) were characterized as also having the highest severity of problems, particularly in terms of externalizing behaviors and level of impairment; parents of these children also reported the highest level of burden/impact on their family. Thus, the long duration and high volume of sessions might be appropriate. While representing only 6% of cases, children in this pattern had 32% of all visits by volume over the 4-year study period. If these children could be adequately cared for while making modest reductions in total visits, this could free resources to manage other demands, such as reducing waiting times.

Recommendations for new approaches to CYMH care

Two key recommendations for CYMHS emerge from our findings. First, we recommend a triage approach, based on client characteristics at intake, be combined with a stepped-care approach. Triage is recommended to identify children and families at the extremes of need. Children with the lowest severity problems (i.e., the Minimal care pattern) could be directed immediately for single session or walk-in services. Children/families with the highest level of problems are most likely to need ongoing care; these cases are unlikely to benefit from brief interventions, which is the typical starting point in stepped care models. Children in the Ongoing/Intensive-Episodic pattern had the highest level of externalizing problems and impairment, and their parents reported very high levels of burden. These cases should be triaged into intensive services as soon as possible; this might include case management along with individual and parent treatment.

For children whose problems are neither very mild nor severe, a stepped-care model would be appropriate. This would be the Acute, Intensive, and Brief Episodic patterns in our sample, for whom the clinical characteristics at intake did not differ markedly. In stepped-care models [83], children first receive a low-intensity treatment (e.g., self-help CBT via internet) and if this treatment is not effective, more intensive and/or additional treatments are added (e.g., CBT delivered in-person, medication; [84, 85]).

Second, the sector needs to develop different ways to care for children over extended periods of time. A recent article suggested using a life-span perspective and argued for applying the idea of a “health home” for coordination of care for children and youth with mental health problems [86]. These authors propose an integration of various delivery models, including the chronic care model [87, 88], systems of care [89], wraparound [90], and the Institute of Medicine mental health intervention spectrum [91, 92]. A shortcoming of their proposal is that they appear to suggest that their model be applied to all children and youth with mental health problems. We maintain that this is not feasible, given ongoing, unmet demands for CYMHS. A chronic care approach may be appropriate. Children with chronic physical health problems receive regular follow-up appointments and adjustments to treatment plans are made as needed [93, 94]. By having regularly scheduled follow-up appointments that involve monitoring of clients’ health and refinement of treatment plans, relapse might be reduced and health status maintained or improved. Such a model might be appropriate to implement for children seen for extended periods of time (e.g., > 2 years) or repeated EoCs, which would include about a quarter of children in our sample. The specific elements in such a model need to be determined, as would the markers of when to shift away from an “acute” treatment approach. We also do not know what children and parents might do to maintain treatment gains following an acute treatment phase or after discharge, for children who are likely to need a second EoC. Understanding positive mental health behaviors should be explored in future studies.

Limitations

Analyses captured data only from participating CYMHS agencies. MH services provided in other sectors were not included and we know that children seen at one agency may be seen at other CYMHS agencies and other sectors [95, 96]. Examining service use over multiple years and across sectors is an important, but challenging, next step for future research. We recently linked the data from the current study to these children’s health care utilization data [97]. We are currently examining relationships between receipt of mental health services from physicians before, during, and after when children were receiving services from a children’s mental health agency [98], and predictors of receiving mental health services from physicians as young adults [99]. The service use patterns identified replicate those found in our previous work [27]. However, we cannot be certain that these patterns would be the same in all CYMHS agencies in Ontario or other jurisdictions.

The administrative data used was lacking a number of variables that could differentiate the patterns of service use such as single parent status, child welfare involvement, family socio-economic status, and ethnicity (e.g., [31, 100]). Future research might also consider agency-specific factors such as hours of operation and distance from clients’ homes, as ease of accessibility would also likely influence service use.

These analyses do not capture important elements related to families’ use of services. For example, we did not examine dropout or changes in outcomes. Dropping out of services and the magnitude of change in outcomes are both important variables that would likely impact service use over time [101, 102], as well as seeking help for a second time at the same agency [103], at other agencies and/or in other sectors.

Conclusions

Examining and understanding patterns of service use within CYMHS and other sectors is critical before tackling the development and testing of new treatment models. By using existing data routinely collected at agencies, data on service use patterns can be directly applied in ways that can: (a) help agencies refine service delivery systems to better meet the needs of their client populations; (b) better allocate resources in response to the needs of their client populations; and (c) start a process of developing new models of service delivery.

Availability of data and materials

The data that support the findings of this study are available from five CYMHS agencies in Ontario, Canada, but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of the CYMHS agencies, and in accordance with research ethics board policies.

Abbreviations

AIC:

Akaike information criteria

BCFPI:

Brief child and family phone interview

BIC:

Bayesian information criteria

CAIC:

Consistent Akaike information criterion

CBT:

Cognitive behavioral therapy

CYMHS:

Child and youth mental health services

DFA:

Discriminant function analysis

EBT:

Evidence-based treatment

EoC:

Episode of care

LCA:

Latent class analysis

LR:

Likelihood-ratio

MH:

Mental health

References

  1. Lahey BB, Krueger RF, Rathouz PJ, Waldman ID, Zald DH. A hierarchical causal taxonomy of psychopathology across the life span. Psychol Bull. 2017;143(2):142–86.

    Article  PubMed  Google Scholar 

  2. Forbes MK, Tackett JL, Markon KE, Krueger RF. Beyond comorbidity: Toward a dimensional and hierarchical approach to understanding psychopathology across the life span. Dev Psychopathol. 2016;28(4pt1):971–86.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Schraeder KE, Reid GJ. Who needs to transition? Defining a target population of youth with depression and anxiety that will require adult mental health care. J Behav Health Serv Res. 2017;44(2):3161–330.

  4. Faraone SV, Biederman J, Mick E. The age-dependent decline of attention deficit hyperactivity disorder: a meta-analysis of follow-up studies. Psychol Med. 2006;36(2):159–65.

    Article  PubMed  Google Scholar 

  5. Bussing R, Mason DM, Bell L, Porter P, Garvan C. Adolescent outcomes of childhood attention-deficit/hyperactivity disorder in a diverse community sample. J Am Acad Child Adolesc Psychiatry. 2010;49(6):595–605.

    PubMed  PubMed Central  Google Scholar 

  6. Hurtig T, Ebeling H, Taanila A, Miettunen J, Smalley SL, McGough JJ, et al. ADHD symptoms and subtypes: relationship between childhood and adolescent symptoms. J Am Acad Child Adolesc Psychiatry. 2007;46(12):1605–13.

    Article  PubMed  Google Scholar 

  7. Lecendreux M, Silverstein M, Konofal E, Cortese S, Faraone SV. A 9-Year follow-up of Attention-Deficit/Hyperactivity Disorder in a population sample. J Clin Psychiatry. 2019;80(3):18m12642.

  8. Boylan K, Vaillancourt T, Boyle M, Szatmari P. Comorbidity of internalizing disorders in children with oppositional defiant disorder. Eur Child Adolesc Psychiatry. 2007;16(8):484–94.

    Article  PubMed  Google Scholar 

  9. Christenson JD, Crane DR, Malloy J, Parker S. The cost of oppositional defiant disorder and disruptive behavior: a review of the literature. J Child Fam Stud. 2016;25(9):2649–58.

    Article  Google Scholar 

  10. Bernstein A, Chorpita BF, Daleiden EL, Ebesutani CK, Rosenblatt A. Building an evidence-informed service array: considering evidence-based programs as well as their practice elements. J Consult Clin Psychol. 2015;83(6):1085–96.

    Article  PubMed  Google Scholar 

  11. Choi KR, Briggs EC, Seng JS, Graham-Bermann SA, Munro-Kramer ML, Ford JD. Service usage typologies in a clinical sample of trauma-exposed adolescents: a latent class analysis. Psychol Trauma. 2018;10(6):652–61.

    Article  PubMed  Google Scholar 

  12. Lambert EW, Brannan AM, Breda C, Heflinger CA, Bickman L. Common patterns of service use in children's mental health. Eval Prog Plan. 1998;21(1):47–57.

    Article  Google Scholar 

  13. Goldstein RB, Olfson M, Wickramaratne PJ, Wolk SI. Use of outpatient mental health services by depressed and anxious children as they grow up. Psychiatr Serv. 2006;57(7):966–75.

    Article  PubMed  Google Scholar 

  14. Heflinger CA, Shaw V, Higa-McMillan C, Lunn L, Brannan AM. Patterns of child mental health service delivery in a public system: rural children and the role of rural residence. J Behav Health Serv Res. 2015;42(3):292–309.

    Article  PubMed  Google Scholar 

  15. Cosgrove JA, Lee BR, Unick GJ. A longitudinal evaluation of wraparound’s impact on youth mental health service use. J Emot Behav Disord. 2019;15:1063426619842568.

    Google Scholar 

  16. Fleming I, Jones M, Bradley J, Wolpert M. Learning from a learning collaboration: the CORC approach to combining research, evaluation and practice in child mental health. Admin Pol Ment Health. 2016;43(3):297–301.

    Article  Google Scholar 

  17. Garland AF, Brookman-Frazee L, Hurlburt MS, Accurso EC, Zoffness RJ, Haine-Schlagel R, et al. Mental health care for children with disruptive behavior problems: a view inside therapists’ offices. Psychiatr Serv. 2010;61(8):788–95.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Hazen AL, Hough RL, Landsverk JA, Wood PA. Use of mental health services by youths in public sectors of care. Ment Health Serv Res. 2004;6(4):213–26.

    Article  PubMed  Google Scholar 

  19. Ortega AN, Chavez L, Inkelas M, Canino G. Persistence of mental health service use among Latino children: a clinical and community study. Admin Pol Ment Health. 2007;34(4):353–62.

    Article  Google Scholar 

  20. Chavira DA, Garland A, Yeh M, McCabe K, Hough RL. Child anxiety disorders in public systems of care: comorbidity and service utilization. J Behav Health Serv Res. 2009;36(4):492–504.

    Article  PubMed  Google Scholar 

  21. Bickman L, Noser K, Summerfelt WT. Long-term effects of a system of care on children and adolescents. J Behav Health Serv Res. 1999;26(2):185–202.

    Article  CAS  PubMed  Google Scholar 

  22. James S, Charlemagne SJ, Gilman AB, Alemi Q, Smith RL, Tharayil PR, et al. Post-discharge services and psychiatric rehospitalization among children and youth. Admin Pol Ment Health. 2010;37(5):433–45.

    Article  Google Scholar 

  23. Cohen E, Snowden L, Libby A, Ma Y. The effects of capitation on outpatient mental health episodes of children. J Child Fam Stud. 2006;15(1):13–26.

    Article  Google Scholar 

  24. Grimes KE, Kapunan PE, Mullin B. Children’s health services in a “system of care”: patterns of mental health, primary and specialty use. Public Health Rep. 2006;121(3):311–23.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Harpaz-Rotem I, Leslie D, Rosenheck RA. Treatment retention among children entering a new episode of mental health care. Psychiatr Serv. 2004;55(9):1022–8.

    Article  PubMed  Google Scholar 

  26. Yeh M, McCabe K, Hurlburt M, Hough R, Hazen A, Culver S, et al. Referral sources, diagnoses, and service types of youth in public outpatient mental health care: a focus on ethnic minorities. J Behav Health Serv Res. 2002;29(1):45–60.

    Article  PubMed  Google Scholar 

  27. Reid GJ, Stewart SL, Barwick M, Cunningham CE, Carter JR, Evans B, et al. Exploring patterns of service utilization within children's mental health agencies. Manuscript submitted for publication. 2019.

    Google Scholar 

  28. Reid GJ, Stewart SL, Zaric GS, Barwick M, Carter JR, Neufeld RWJ, et al. Defining episodes of care in children's mental health using administrative data. Admin Pol Ment Health. 2015;42(6):737–47.

    Article  Google Scholar 

  29. Andersen R, Newman JF. Societal and individual determinants of medical care utilization in the United States. Milbank Mem Fund Q Health Soc. 1973;51(1):95–124.

    Article  CAS  PubMed  Google Scholar 

  30. Aday LA, Andersen RM. A framework for the study of access to medical care. Health Serv Res. 1974;9(3):208–20.

    CAS  PubMed  PubMed Central  Google Scholar 

  31. Sayal K. Annotation: pathways to care for children with mental health problems. J Child Psychol Psychiatry. 2006;47(7):649–59.

    Article  PubMed  Google Scholar 

  32. Zwaanswijk M, Verhaak PF, Bensing JM, Van der EJ, Verhulst FC. Help seeking for emotional and behavioural problems in children and adolescents: a review of recent literature. Eur Child Adolesc Psychiatry. 2003;12(4):153–61.

    Article  PubMed  Google Scholar 

  33. Zwaanswijk M, Van der EJ, Verhaak PF, Bensing JM, Verhulst FC. Factors associated with adolescent mental health service need and utilization. J Am Acad Child Adolesc Psychiatry. 2003;42(6):692–700.

    Article  PubMed  Google Scholar 

  34. Zimmerman FJ. Social and economic determinants of disparities in professional help-seeking for child mental health problems: evidence from a national sample. Health Serv Res. 2005;40(5 Pt 1):1514–33.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Ford T. Practitioner review: how can epidemiology help us plan and deliver effective child and adolescent mental health services? J Child Psychol Psychiatry. 2008;49(9):900–14.

    Article  PubMed  Google Scholar 

  36. Ryan SM, Jorm AF, Toumbourou JW, Lubman DI. Parent and family factors associated with service use by young people with mental health problems: a systematic review. Early Interv Psychiatry. 2015;9(6):433–46.

    Article  PubMed  Google Scholar 

  37. Duncan L, Boyle MH, Abelson J, Waddell C. Measuring children's mental health in Ontario: policy issues and prospects for change. J Can Acad Child Adolesc Psychiatry. 2018;27(2):88–98.

    PubMed  PubMed Central  Google Scholar 

  38. Martin D, Miller AP, Quesnel-Vallee A, Caron NR, Vissandjee B, Marchildon GP. Canada’s universal health-care system: achieving its potential. Lancet. 2018;391(10131):1718-1735.

    Article  Google Scholar 

  39. Madore O. The Canada Health Act: Overview and Options. Ottawa; Parliamentary Research Branch, Library of Parliament; 2003. Report No.: PRB 94-4E

  40. Cunningham CE, Pettingill P, Boyle M. The Brief Child and Family Phone Interview (BCFPI): Interviewers Manual. 2000. Report No.: November 28, 2000 Edition. Canadian Centre for the Study of Children at Risk, Hamilton Health Sciences Corporation, McMaster University.

  41. Cunningham CE, Pettingill P, Boyle M. The Brief Child and Family Phone Interview (BCFPI-3) - A Computerized Intake and Outcome Assessment Tool: Interviewers Manual. Hamilton: Canadian Centre for the Study of Children at Risk, Hamilton Health Sciences Corporation, McMaster University; 2002. Report No.: March 2002

  42. Niccols A, Cunningham C, Pettingill P, Bohaychuk D, Duku E. Infant mental health: the brief child and family intake and outcomes system. J Appl Behav Dev. 2018;42(6):588–96.

    Article  Google Scholar 

  43. Lai JKY, Weiss JA. Priority service needs and receipt across the lifespan for individuals with autism spectrum disorder. Autism Res. 2017;10(8):1436–47.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Turcotte P, Mathew M, Shea LL, Brusilovskiy E, Nonnemacher SL. Service across the lifespan for individuals with autism. J Autism Dev Disord. 2016;46(7):2480–9.

    Article  PubMed  Google Scholar 

  45. Lubetsky MJ, Handen BL, Lubetsky M, McGonigle JJ. Systems of care for individuals with autism spectrum disorder and serious behavioral disturbance through the lifespan. Child Adolesc Psychiatr Clin N Am. 2014;23(1):97–110.

    Article  PubMed  Google Scholar 

  46. Saloner B, Carson N, LeCook B. Episodes of mental health treatment among a nationally representative sample of children and adolescents. Med Care Res Rev. 2014;71(3):261–79.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Blader JC. Symptom, family, and service predictors of children's psychiatric rehospitalization within one year of discharge. J Am Acad Child Adolesc Psychiatry. 2004;43(4):440–51.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Stewart SL, Kam C, Baiden P. Predicting length of stay and readmission for psychiatric inpatient youth admitted to adult mental health beds in Ontario. Can Child Adolesc Ment Health. 2014;19(2):115–21.

    Article  PubMed  Google Scholar 

  49. Barwick MA, Boydell KM, Cunningham CE, Ferguson HB. Overview of Ontario's screening and outcome measurement initiative in children's mental health. Can Child Adolesc Psychiatr Rev. 2004;13(4):105–9.

    PubMed  PubMed Central  Google Scholar 

  50. Cunningham CE, Boyle MH, Hong S, Pettingill P, Bohaychuk D. The brief child and family phone interview (BCFPI): 1. Rationale, development, and description of a computerized children’s mental health intake and outcome assessment tool. J Child Psychol Psychiatry. 2009;50(4):416–23.

    Article  PubMed  Google Scholar 

  51. Boyle MH, Cunningham CE, Georgiades K, Cullen J, Racine Y, Pettingill P. The brief child and family phone interview (BCFPI): 2. Usefulness in screening for child and adolescent psychopathology. J Child Psychol Psychiatry. 2009;50(4):424–31.

    Article  PubMed  Google Scholar 

  52. Corp IBM. SPSS statistics for windows, version 24.0. 24th ed. Armonk, NY: IBM Corp; 2014.

    Google Scholar 

  53. Vermunt JK, Magidson J. Latent GOLD. 4.0 ed. Belmont, MA: Statistical Innovations Inc.; 2005.

    Google Scholar 

  54. Magidson J, Vermunt JK. Latent class models. In: Kaplan D, editor. The sage handbook of quantitative methodology for the social sciences. Thousand Oaks, CA: Sage Publications; 2004. p. 175–98.

    Google Scholar 

  55. Steiger JH, Fouladi RT. Noncentrality interval estimation and the evaluation of statistical models. In: Harlow LL, Mulaik SA, Steiger JH, editors. What if there were no significance tests? Mahwah, NJ: Erlbaum; 1997. p. 221–57.

    Google Scholar 

  56. Rindskopf D, Rindskopf W. The value of latent class analysis in medical diagnosis. Stat Med. 1986;5(1):21–7.

    Article  CAS  PubMed  Google Scholar 

  57. Celeux G, Soromenho G. An entropy criterion for assessing the number of clusters in a mixture model. J Classif. 1996;13:195–212.

    Article  Google Scholar 

  58. Magidson J, Vermunt JK. A nontechnical introduction to latent class models. Belmont: Statistical Innovations; 2002. Report No.: White Paper #1

  59. Tabachnick BG, Fidell LS. Using multivariate statistics. 6th ed. New York:NY: Harper & Row; 2013.

    Google Scholar 

  60. Shao J, Tu D. The jackknife and bootstrap. New York: Springer; 1995.

    Book  Google Scholar 

  61. Chernick MR. The jackknife: a resampling method with connections to the bootstrap. Wiley Interdisciplinary Reviews: Computational Statistics. 2011;4(2):224–6.

    Article  Google Scholar 

  62. Warner RM. Discriminant analysis. Applied statistics from bivariate through multivariate techniques. 2nd ed. New York, NY: Sage; 2013. p. 726–77.

    Google Scholar 

  63. Weisz JR, Chorpita BF, Palinkas LA, Schoenwald SK, Miranda J, Bearman SK, et al. Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: a randomized effectiveness trial. Arch Gen Psychiatry. 2012;69(3):274–82.

    Article  PubMed  Google Scholar 

  64. Andrade AR, Lambert EW, Bickman L. Dose effect in child psychotherapy: outcomes associated with negligible treatment. J Am Acad Child Adolesc Psychiatry. 2000;39(2):161–8.

    Article  CAS  PubMed  Google Scholar 

  65. Bickman L. A continuum of care - more is not always better. Am Psychol. 1996;51(7):689–701.

    Article  CAS  PubMed  Google Scholar 

  66. Lewandowski RE, Acri MC, Hoagwood KE, Olfson M, Clarke G, Gardner W, et al. Evidence for the management of adolescent depression. Pediatrics. 2013;132(4):e996–e1009.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Renaud J, Brent DA, Baugher M, Birmaher B, Kolko DJ, Bridge J. Rapid response to psychosocial treatment for adolescent depression: a two-year follow-up. J Am Acad Child Adolesc Psychiatry. 1998;37(11):1184–90.

    Article  CAS  PubMed  Google Scholar 

  68. de Haan AM, Boon AE, de Jong JT, Hoeve M, Vermeiren RR. A meta-analytic review on treatment dropout in child and adolescent outpatient mental health care. Clin Psychol Rev. 2013;33(5):698–711.

    Article  PubMed  Google Scholar 

  69. Gopalan G, Goldstein L, Klingenstein K, Sicher C, Blake C, McKay MM. Engaging families into child mental health treatment: updates and special considerations. J Can Acad Child Adolesc Psychiatry. 2010;19(3):182–96.

    PubMed  PubMed Central  Google Scholar 

  70. Edlund MJ, Wang PS, Berglund PA, Katz SJ, Lin E, Kessler RC. Dropping out of mental health treatment: patterns and predictors among epidemiological survey respondents in the United States and Ontario. Am J Psychiatry. 2002;159(5):845–51.

    Article  PubMed  Google Scholar 

  71. Gee D, Mildred H, Brann P, Taylor M. Brief intervention: a promising framework for child and youth mental health? Admin Pol Ment Health. 2015;42(2):121–5.

    Article  Google Scholar 

  72. Barwick MA, Uranjnik DJ, Sumner L, Cohen S, Reid GJ, Engel K, et al. Profiles and service utilization for children accessing a mental health walk-in clinic versus usual care. J Evidenced-Based Social Work. 2013;10(4):338–52.

    Article  Google Scholar 

  73. McGarry J, McNicholas F, Buckley H, Kelly BD, Atkin L, Ross N. The clinical effectiveness of a brief consultation and advisory approach compared to treatment as usual in child and adolescent mental health services. Clin Child Psychol Psychiat. 2008;13(3):365–76.

    Article  Google Scholar 

  74. Ministry of Children and Youth Services. Moving on mental health 2015 [updated 2015]. Accessed 21-Jan-2015. Available from: http://www.children.gov.on.ca/htdocs/English/topics/specialneeds/mentalhealth/moving-on-mental-health.aspx.

  75. Gondek D, Edbrooke-Childs J, Fink E, Deighton J, Wolpert M. Feedback from outcome measures and treatment effectiveness, treatment efficiency, and collaborative practice: a systematic review. Admin Pol Ment Health. 2016;43(3):325–43.

    Article  Google Scholar 

  76. Wolpert M, Ford T, Trustam E, Law D, Deighton J, Flannery H, et al. Patient-reported outcomes in child and adolescent mental health services (CAMHS): use of idiographic and standardized measures. J Ment Health. 2012;21(2):165–73.

    Article  PubMed  Google Scholar 

  77. Warren JS, Nelson PL, Burlingame GM, Mondragon SA. Predicting patient deterioration in youth mental health services: community mental health vs. managed care settings. J Clin Psychol. 2012;68(1):24–40.

    Article  PubMed  Google Scholar 

  78. Weersing VR, Weisz JR. Community clinic treatment of depressed youth: benchmarking usual care against CBT clinical trials. J Consult Clin Psychol. 2002;70(2):299–310.

    Article  PubMed  Google Scholar 

  79. Weersing VR, Iyengar S, Kolko DJ, Birmaher B, Brent DA. Effectiveness of cognitive-behavioral therapy for adolescent depression: a benchmarking investigation. Behav Ther. 2006;37(1):36–48.

    Article  PubMed  Google Scholar 

  80. Miller AR, Lalonde CE, McGrail KM. Children’s persistence with methylphenidate therapy: a population-based study. Can J Psychiatr. 2004;49(11):761–8.

    Article  Google Scholar 

  81. Sytema S, Giel R, Ten Horn GHMM. Patterns of care in the field of mental health: conceptual definition and research methods. Acta Psychiatr Scand. 1989;79(1):1–10.

    Article  CAS  PubMed  Google Scholar 

  82. Mueller CW, Tolman R, Higa-McMillan CK, Daleiden EL. Longitudinal predictors of youth functional improvement in a public mental health system. J Behav Health Serv Res. 2010;37(3):350–62.

    Article  PubMed  Google Scholar 

  83. Bower P, Gilbody S. Stepped care in psychological therapies: access, effectiveness and efficiency: narrative literature review. Br J Psychiatry. 2005;186:11–7.

    Article  PubMed  Google Scholar 

  84. Ollendick TH, Ost LG, Farrell LJ. Innovations in the psychosocial treatment of youth with anxiety disorders: implications for a stepped care approach. Evid Based Ment Health. 2018;21(3):112–5.

    Article  PubMed  Google Scholar 

  85. Pettit JW, Rey Y, Bechor M, Melendez R, Vaclavik D, Buitron V, et al. Can less be more? Open trial of a stepped care approach for child and adolescent anxiety disorders. J Anxiety Disord. 2017;51:7–13.

    Article  PubMed  PubMed Central  Google Scholar 

  86. de Voursney D, Huang LN. Meeting the mental health needs of children and youth through integrated care: a systems and policy perspective. Psychol Serv. 2016;13(1):77–91.

    Article  PubMed  Google Scholar 

  87. Bodenheimer T, Wagner EH, Grumbach K. Improving primary care for patients with chronic illness. JAMA. 2002;288(14):1775–9.

    Article  PubMed  Google Scholar 

  88. Bodenheimer T, Wagner EH, Grumbach K. Improving primary care for patients with chronic illness: the chronic care model, part 2. JAMA. 2002;288(15):1909–14.

    Article  PubMed  Google Scholar 

  89. Stroul BA, Blau G, Friedman R. Updating the system of care concept and philosophy. 2010. Accessed 15-March-2017.

    Google Scholar 

  90. Rosenblatt A. Bows and ribbons, tape and twine: wrapping the wraparound process for children with multi-system needs. J Child Fam Stud. 1996;5(1):101–16.

    Article  Google Scholar 

  91. Institute of Medicine. Reducing risk for mental disorders: Frontiers for preventive intervention research. Washington: National Academies Press; 1994.

  92. O'Connell ME, Boat T, Warner KE. Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington: National Academies Press; 2009. Accessed 4-Apr-2017

  93. Wagner EH, Austin BT, Von KM. Organizing care for patients with chronic illness. Milbank Q. 1996;74(4):511–44.

    Article  CAS  PubMed  Google Scholar 

  94. Lorig KR, Sobel DS, Stewart AL, Brown BW Jr, Bandura A, Ritter P, et al. Evidence suggesting that a chronic disease self-management program can improve health status while reducing hospitalization: a randomized trial. Med Care. 1999;37(1):5–14.

    Article  CAS  PubMed  Google Scholar 

  95. Leventhal T, Brooks-Gunn J, McCormick MC, McCarton CM. Patterns of service use in preschool children: correlates, consequences, and the role of early intervention. Child Dev. 2000;71(3):802–19.

    Article  CAS  PubMed  Google Scholar 

  96. Reid GJ, Cunningham CE, Evans B, Stewart M, Brown JB, Lent B, et al. Help-seeking for children with mental health problems: Parents’ efforts and experiences. Admin Pol Ment Health. 2011;38(5):384–97.

    Article  Google Scholar 

  97. Schraeder KE, Barwick M, Cairney J, Carter JR, Kurdyak P, Neufeld RWJ, et al. Cross-sectoral data linkage: Tracking mental health service utilization from childhood into adulthood. Poster presented at: 2018 International population data linkage network (IPDLN) conference;2018 Sept 12–14; Banff, Canada

  98. Reid GJ, Barwick M, Cairney J, Carter JR, Lau C, Kurdyak P, et al. Service use in the mental health and healthcare sectors within Ontario for children and youth with mental health problems. Poster presented at: North American Primary Care Research Group (NAPCRG); 2019 Nov 16–20; Toronto, Canada.

  99. Schraeder KE, Barwick M, Cairney J, Carter JR, Kurdyak P, Neufeld RWJ, et al. Re-accessing mental health care after age 18: A study of youth treated in community child and youth mental health agencies. Poster presented at: North American Primary Care Research Group (NAPCRG); 2019 Nov 16–20; Toronto, Canada.

  100. St. Pierre J, Leschied AW, Stewart SL, Cullion CM. Differentiating three year outcomes following tertiary child and youth inpatient psychiatric treatment. Final report to the Centre of Excellence for Child and Youth Mental Health at the Children’s Hospital of Eastern Ontario; 2008.

    Google Scholar 

  101. Dossett KW, Reid GJ. Predicting dropout from children’s mental health services: using a need-based definition of dropout. Child Psychiatry Hum Dev. 2019: https://0-doi-org.brum.beds.ac.uk/10.1007/s10578-019-00906-4.

  102. Dossett KW, Reid GJ. Defining dropout from children’s mental health services A novel need-based definition. J Child Fam Stud. 2019;25:1 in press.

    Article  Google Scholar 

  103. Sarmiento C, Reid GJ. Re-accessing community mental health services for children and adolescents. J Behav Health Serv Res. 2019: https://0-doi-org.brum.beds.ac.uk/10.1007/s11414-019-09668-2.

Download references

Acknowledgements

The research team would like to thank the staff at each of the participating agencies for their time and energy. John Lyons (University of Chicago) is also acknowledged for providing valuable ideas related to the interpretation and implications of our results. We also thank the research staff who contributed to this project including Gesine Alders, Kaitlyn Bruyns, Nicole Etherington, Stephanie Mowat, and Ayla Visser.

Funding

This project was supported by a grant from the Canadian Institutes of Health Research (#106442). G. Reid was supported by the Children’s Health Research Institute, London, ON. The funding bodies had no role in study design, data collection, analysis, interpretation, or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Departments of Psychology, Family Medicine, and Paediatrics, The University of Western Ontario, Westminster Hall Room 319E, London, ON, N6A 3K7, Canada

    Graham J. Reid

  2. Children’s Health and Therapeutics, Children’s Health Research Institute, London, Canada

    Graham J. Reid & Shannon L. Stewart

  3. Faculty of Education, The University of Western Ontario, London, Canada

    Shannon L. Stewart & Alan Leschied

  4. Research Institute, The Hospital for Sick Children, Toronto, Canada

    Melanie Barwick

  5. Department of Psychiatry, University of Toronto, Toronto, Canada

    Melanie Barwick

  6. The Dalla Lana School of Public Health, University of Toronto, Toronto, Canada

    Melanie Barwick

  7. Quality Improvement, Vanier Children’s Services, London, Canada

    Jeffrey Carter

  8. Departments of Psychology and Psychiatry, The University of Western Ontario, London, Canada

    Jeffrey Carter & Jeff St. Pierre

  9. Departments of Psychology, Psychiatry, and Neuroscience, The University of Western Ontario, London, Canada

    Richard W. J. Neufeld

  10. Child and Parent Resource Institute, London, Canada

    Jeff St. Pierre

  11. Department of Psychology, The University of Western Ontario, London, Canada

    Juliana I. Tobon

  12. St. Michael’s Hospital Academic Family Health Team, Toronto, Canada

    Juliana I. Tobon

  13. Departments of Family Medicine and Epidemiology and Biostatistics, The University of Western Ontario, London, Canada

    Evelyn Vingilis

  14. Ivey Business School, The University of Western Ontario, London, Canada

    Gregory S. Zaric

  15. Department of Epidemiology and Biostatistics, The University of Western Ontario, London, Canada

    Gregory S. Zaric

Authors
  1. Graham J. Reid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shannon L. Stewart
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Melanie Barwick
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jeffrey Carter
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alan Leschied
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Richard W. J. Neufeld
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jeff St. Pierre
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Juliana I. Tobon
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Evelyn Vingilis
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Gregory S. Zaric
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

GJR, SLS, MB, JC, AL, RWJN, JSP, JIT, EV, and GSZ designed the study. GJR, SLS, RWJN, EV, GSZ contributed to the analyses. GJR, SLS, MB, JC, AL, RWJN, JSP, JIT, EV, and GSZ wrote the manuscript. GJR, SLS, MB, JC, AL, RWJN, JSP, JIT, EV, and GSZ read and approved the final manuscript.

Corresponding author

Correspondence to Graham J. Reid.

Ethics declarations

Ethics approval

This study received ethics approval from The University of Western Ontario Research Ethics Board and from the local ethics board for each participating CYMHS agency. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committees and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. For this type of study, formal consent is not required.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1: Table S1.

Model Fit Indices for Testing 2–10 Class Solutions for Patterns of Service Use across Five Child and Youth Mental Health Service Agencies. Table S2. Mean Number of Visits in Each Period of Service Involvement across 4 Years by Service Use Pattern. Table S3. Breakdown of Duration of Involvement (in Months) across 4 Years by Service Use Pattern. Table S4. Distribution of Cases across Episodes of Care over 4 Years by Service Use Pattern. Table S5. A Sample Size and Number of Episodes by Pattern of Service Use in the Previous Study and the Present Study. B Service Use Descriptive Statistics by Pattern of Service Use in the Previous Study and the Present Study.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Reid, G.J., Stewart, S.L., Barwick, M. et al. Predicting patterns of service utilization within children’s mental health agencies. BMC Health Serv Res 19, 993 (2019). https://0-doi-org.brum.beds.ac.uk/10.1186/s12913-019-4842-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12913-019-4842-2

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us