Skip to main content

Table 2 Main findings: evaluation and outcomes

From: Cultural competence in mental health care: a review of model evaluations

Study Evaluation Outcomes
Ferguson (2003) Likert ratings (1–5) of overall value, clarity of objectives, instructor effectiveness
For second cohort (2000, N = 55):
Showed high scores on all of these: means 4.1 to 4.4 for each domain, and for two year bands 1999 and 2000 :
Intention to change: M1: 5.4% (n = 55), M2: 48.1% (n = 54), M3: 30.2% (n = 43)
Actual Change in behaviour: M1: 16.6% (n = 42), M2: 21.4% (n = 42), M3: -
Hadwiger (1999) Cultural Sensitivity (40% of course marks)
Evidence of context of own cultural background
Ethnocentric attitudes
Power orientation
Egalitarian relationship
Trust in relationships
Respect for patient during hypothetical negotiations
Manner of addressing hypothetical patients
Accuracy of content (30% of course marks)
Process (30% of course marks)
Nursing students were able to become more aware of how their own culture affects the nursing care
Able to refine cultural competence skills using hypothetical cases and narrative writing
Actual marks or origins of students not given
Siegel et al (2003) † For each level and domain, experts identified key performance indicators identified, performance measures defined, and data sources outlined.
163 indicators
231 measures
Without a formal commitment to the development of a process and the dedication of resources for this effort, cultural competence would be difficult to achieve.
Reduced to 85 measures
Services.53% had put into place services that had been adapted or developed for specific cultural groups.
CC Outcomes. 60% of administrative entities indicated that outcome measures could be analyzed for specific cultural groups.
CC Training and Education.
73% indicated staff members receive ongoing education and training related to CC.
87% selected, developed, and/or provided CC training materials to agencies under their purview but only one provided financial assistance to agencies under its purview for conducting CCT
Services. 87% of the service entities indicated that they had services adapted or developed for specific cultural groups. 29% of these, providing culture-specific services was the mission of the agency; while for the remaining 71%, culture-specific services had been put in place in response to the perceived needs of clients in the community.
CC Training and Education.
75% indicated that staff of receive ongoing education and training on CC. 87% said all new employees receive CC education and training as part of their orientation.
75% said that professional education (for example, grand rounds) included racial/ethnic/cultural issues.
CC Outcomes. Outcome measure data were collected inconsistently at the five agencies responding to this question, but all conducted consumer satisfaction surveys. Sixty percent of those responding indicated that the outcome measures could be analyzed for specific cultural groups. 50% said that CC was included in staff performance evaluations.
Kim-Godwin (2001) Literature review and concept analysis lead to 3 constructs that were evaluated: 1) health care systems, 2) health outcomes, and 3) cultural competence scale ratings. In factor analyses, cultural knowledge emerged as a components of cultural sensitivity and cultural skills
All 13 participants reported that cultural competent care resulted in positive health outcomes in their practice. Specifically, increases in prenatal visits, higher rates of immunization, reduced morbidity and mortality, increased compliance, increased trust, increased self worth, more interest in promoting health. (Actual accounts not presented, only surmises findings).
Kondrat et al (1999) Nature of interactions between service providers and Caucasian and African American consumers with SMI
Themes: Types and locations of service delivery
Structure of delivery services
Formal and informal organisational culture
Decision making process
Perceptions of interactions, processes and decisions
Analysis based on 700 observations across four sites
Constant comparison analyses
All four agencies incorporated policies to support diversity, yet outcomes for diverse clients varied.
11 clusters of activity:
Differentiating: B > C
1. Agency work culture: pro-agency culture:
2. Openness/boundary flexibility
3. Prevalent supervisory style: consistent, pro-active, and supportive
4. Team functioning and decisions
5. Attitudes towards clients:
6. Demonstration of programme commitment to diversity
7. Level of acceptance
8. Diversity as a clinical issue
9. Clinical orientation
10. Level of interdisciplinary work
11. Organisation of service
There was little evidence that race or culture was routinely considered in making treatment decisions
Kirmayer et al (2003) Participant observation of the first 100 referred cases.
29 referring clinicians for 47 cases completed service evaluation information
Specialized cultural consultation services can play a major role in educating clinicians and in developing innovative intervention strategies
Cases seen by the team demonstrated the impact of cultural misunderstandings: incomplete assessments, incorrect diagnoses, inadequate or inappropriate treatments, and failed treatment alliances.
86% of clinicians referring patients to the service reported high rates of satisfaction, but many indicated a need for longer term follow up.
41%: increased knowledge of social, cultural or religious aspects of cases
21%: increased knowledge of psychiatric or psychological aspects of their cases
48% : improved treatments
31%: improved communications, empathy, understanding, therapeutic alliance
14%: increased confidence in diagnosis, treatment
Dissatisfaction with:
14%: lack of treatment or more intensive follow up
14% unavailability or inappropriateness of recommended resources
10% concerns about the cultural appropriateness of the cultural broker
10%: too much focus on social context rather than psychiatric issues
For 21 cases, some aspects of the recommendations were not implemented: patient non-compliance (13), lack of staff or resources (9), spontaneous improvement (7).
Frusti et al (2003) Consultant employed to assess drivers, linkages, culture and measurement strengths and weaknesses of organisation Drivers: 1) nursing diversity committee promotes supportive work environment by sponsoring educational activities & newsletter
2) Nursing recruitment and retention committee
3) Transcultural patient care committee, provides up to date resources about influence of culture on health
Linkages:1) Managers and staff share department committee responsibilities, and feed into a shared decision making process
2) Nursing and human resources departments conduct annual planning to identify shared goals, and recruitment targets national and local nursing organisations
3) Summer intern programme to recruit under represented groups
Culture: 1) education and orientation to culture of nursing department, leadership roles developed; focus groups indicated managers are trusted, 75% of participants said they were set up to succeed by their nurse mangers
2) primary values: needs of patients first, best nursing care in the world Measurement: Recruitment data, retention data, compared with national benchmarks
Stork et al Used data from Rosenbaum (1999) study of cultural competence in manage care contracts.
Analysis of contract excerpts for cultural competence definitions, and requirements for service provision.
Open ended interviews with officials in five states to examine written cultural competence requirements.
Purposive sample of states that
1) that had contract with cultural competence provisions
2) more comprehensive requirements than other states, reflecting early implementation
3) were average in resources and populations
4) had officials who could talk in depth about contracts
Rosenbaum reported on 37 states, of these 27 had cultural competency requirements, and 10 met criteria. : contract language comprehensive, 2) specific wording about practices rights to culturally competent services
States selected because of geographic, ethnic and racial diversity
Interview: definition of CC
Contract language/standards
Methods to measure and enforce standards
Methods to track cultural competence
Methods to track consumer
enrolment/satisfaction/service use by ethnic/racial groups
Lack of indicators for cultural competence, reluctant to enforce existing standards, disagreement over costs, lack of constituency in training and tracking
4 of five states included their own definitions of cultural competence in their contracts
• Relate to client with sensitivity, understanding, respect for clients' culture
• Understanding social, linguistic, ethnic and behavioural characteristics of a community or a population and the ability to translate systematically, that knowledge into practices in the delivery of services-identify and value difference, acknowledge interactive dynamics of cultural differences, continuously expand cultural knowledge/resources, collaborate with community re provisions and delivery, commit to cross cultural training, develop policies to provide relevant, effective, programs for diverse populations
• Ability to serve individuals of all ages, ethnic groups, in a manner appropriate to their age and unique cultural background.
• A set of congruent behaviours, attitudes and practices and policies that are formed within an agency and among professionals that enable the system, agency and professionals to work respectfully, effectively, responsibly, in diverse situations. Essential elements include: valuing diversity, understanding dynamics of difference, institutionalising cultural knowledge, and adapting to and encouraging organisational diversity.
Themes identified: contract language, contract deliverables, procedures for monitoring and oversight, data collection, provider assessments
Contractual deliverables: submit a plan to include translations of written material and access to interpreters at no extra cost, legally mandated.
Oversight/agency: assign responsibility to a specific agency.
Oversight mechanisms: readiness reviews, site reviews, before roll out. Complaint tracking, consumer satisfaction surveys.
Collecting client data: three stated did not collect enrolment data, disenrollment, provider changes, service use or satisfaction by race/ethnicity. Two states can assess requests to change provider by ethnicity, and whether change requested is a result of language problems. None of the states used the cultural data on their client to indicate lack of cultural competence. No state asks clients to rate their cultural competence of provision
Penalties: None, and none are enforced.
Assessment of CC: determined by provider to MCO/MBHO documentation of training, available ;personnel, representative services as contract deliverables dictate.
US Department of Health and Human Services (HRSA) Organisational cultural competence assessment profile assesses domains, focus areas and indicators
Domains: As in Table 1.
For each domain there are Indicators which have a) structure, b) process and c) outcome
Findings suggest that the Assessment Profile can be useful even in its current form as an organizational framework and a guide to an organization's own development of indicators and measures of cultural competence
CC must be integrated into other organisational domains of activity
Organisational values must be tackled first.
Structures, process and outcomes agreed for each of the subheading:
Organisational Values: Leadership, investment and documentation, Information and data retrieval for cultural competence, Organisational flexibility, Community Involvement and Accountability, Board Development, Policies
Planning, monitoring, evaluation: Client, community and staff inputs, Plans and Implementation, Collection and use of cultural competence data
Communication: Understanding communication needs of clients, Culturally competent oral communication/written/other communication, Communication with community, Organisational communication
Staff development: Training commitment, Training content, Staff Performance
Organisational infrastructure: Financial, Staffing, Technology, Physical facility characteristics, Linkages
Services/Interventions: Client family community input, Screening/assessment/care planning, Treatment and follow up