Skip to main content
  • Research article
  • Open access
  • Published:

An evidence-based approach to understanding the competency development needs of the health service management workforce in Australia

Abstract

Background

Competent managers are essential to the productivity of organisations and the sustainability of health systems. Effective workforce development strategies sensitive to the current competency development needs of health service managers (HSMs) are required.

Purpose

To conduct a 360° assessment of the competence of Australian HSMs to identify managerial competence levels, and training and development needs.

Methods

Assessment of 93 middle-level HSMs from two public hospitals (n = 25) and five community health services (CHS) (n = 68), using the Managerial Competency Assessment Partnership (MCAP) framework and tool, conducted between 2012 and 2014 in Victoria, Australia.

Results

Mean competency scores from both self- and combined colleagues’ assessments indicated competence (scores greater than five but less than six) without guidance, but many HSMs have not had extensive experience. Around 12% of HSMs were unable to demonstrate the competency of ‘evidence-informed decision-making’ and 4% of HSMs were unable to demonstrate the competency of ‘enabling and managing change’.

Conclusion

The assessments confirmed managerial competence for the majority of middle-level HSMs from hospitals and CHS in Victoria, but found competency gaps. In addition, the assessment confirmed managerial strengths and weaknesses varied across management groups from different organisations. These findings suggest that the development of strategies to strengthen the health service management workforce should be multifaceted.

Practice implications

A focus on competency in performance evaluation and development using the MCAP framework and tool not only provides insights into performance of HSMs, but also has the potential to provide an organisation strategic advantage through succession planning and advancing managers’ competence via learning needs analysis and targeted professional development. Linking competencies of HSMs to organisational objectives and strategies provides optimal use of the human resource capacity, improving the organisation’s productivity and sustainability.

Peer Review reports

Background

Empirical evidence has demonstrated a link between the development and competence of managers and positive productivity of organisations [1, 2]. In healthcare, leadership styles and health services managerial development has demonstrated an impact on healthcare service provision, workforce and work environments [3, 4]. However, the Australian healthcare system is considered to be at risk of inadequate or even dangerous management practice [5] and poor management is one of the contributing factors to high turnover amongst healthcare professional staff [6]. The potential inadequacy in leadership and management of the senior executive team has been outlined by a number of official inquiries into the quality and safety scandals in the Australian public hospitals [7, 8].

Health services managers (HSM) may be inadequately equipped for their management responsibilities as the role of health service management has not been well defined in Australia and HSM are often recruited based on a clinicians’ seniority and clinical leadership rather than management competency and capability [9]. Although HSMs tended to possess postgraduate qualifications, such qualifications were largely non-management related, usually in the field of clinical practice of the appointed HSM. A recent study conducted in Victoria determined that more than half of the middle-level HSM, and more than 70% of the senior level HSM had been awarded postgraduate qualifications [10, 11]. However, less than 10% of these qualifications were management specific [10, 11]. This suggests that formal managerial development for personnel in healthcare is currently not a requirement of appointment to middle and senior level HSM positions in Australia, and raises questions on the managerial competency of personnel recruited to HSM roles. In contrast, certification of healthcare managers by a professional body is usual practice in the USA [12].

The importance of the training and education of HSMs to confront the complex and dynamic health systems is well recognised [9, 13], and empirical evidence strongly suggests that training is effective for managerial development [14]. Managerial professional development can be achieved through a number of mechanisms such as providing formal education through universities in the field of management; in-service training; in-house use of mentors and study groups; offsite intensive training, and seminars and conferences [15]. To facilitate managerial development, efforts may be seen in the form of investment in professional development activities by healthcare organisations; training and development opportunities provided by professional institutions and various non-formal educational bodies; short training courses provided in partnership between universities and professional institutions, and formal higher degree university programs. However, to progress the design of initiatives to support HSM managerial development, an understanding of current competency levels and learning needs is first required. To date, competency assessment has not been widely embedded into the performance management process of healthcare organisations where HSMs’ managerial competence is assessed regularly to inform the requirements for training and education. Therefore, unlike the USA [12], the management competency requirements and learning needs of HSMs in Australia are unclear.

A competency-based education framework has been widely applied to health service management training in many higher education institutions in the US and the UK [16, 17] and has been recommended by the Australian Institute of Health and Welfare as detailed in the National Health and Hospital Reform Commission (2009) report [5]. However, the application of such an education framework in Australia has been limited, most likely due to the lack of understanding of competency requirements for HSMs in Australia [10]. Analysis of the postgraduate health service management degrees offered by Australian universities confirms a high level of variation in the core foundation knowledge for managing health services [18]. A more recent analysis of the 13 Australian masters’ level programs in health administrations also confirmed that no consistent approach in considering the actual competency requirements of health service management workforce has been adopted in designing the existing health administration programs [19]. Although some of the programs are mapped against the capability framework provided by the Australasian College of Health Service Managers as part of the accreditation requirements, a competency-based approach has not yet been fully adopted. Potentially more effective workforce training and development strategies that are sensitive to the current competency development needs of HSMs [13] and more effective investment in the development of the competence of HSMs [13, 20, 21] can be implemented.

It has been recommended that competency-based assessment, education and training for HSMs be established and adopted in Australia [22]. In 2011, a Management Competency Assessment Partnership (MCAP) project was established in Victoria, Australia. The project developed and validated a leadership and management competency framework and a management competency assessment tool. The MCAP Leadership and Management Competency Framework (MCAP LMCF) was developed through a rigorous and staged process, adapting the frameworks and assessment processes described by Catano & Campbell, C. (2007) [23] and Garman, Tyler & Darnall (2004) [24]. The MCAP LMCF is purpose built for the healthcare setting and HSM competency assessment, consisting of the following six core management competencies that are measured by 79 behavioural items [11, 12, 24]:

  • C1 Evidence-informed decision-making (Evidence)

  • C2 Operations, administration and resource management (Resources)

  • C3 Demonstrated knowledge of healthcare environment and the organisation (Knowledge)

  • C4 Interpersonal, communication qualities and relationship management (Communication)

  • C5 Leading people and organisations (Leadership)

  • C6 Enabling and managing change (Change)

(Appendix provides details of these behavioural items)

Based on the MCAP LMCF, an online evidence-based management competency assessment tool (the MCAP Tool), comprising a 360° assessment (involving self, supervisor, colleague, and supervisee) was developed and validated in Victoria between 2012 and 2014 [11, 12, 24, 25].

The purpose of this paper is to examine the competence of middle-level HSMs in Victoria, using the MCAP 360° assessment, and to identify competency based training and education directions for health service managers by determining:

  1. 1.

    The managerial competencies of health service managers at individual and group levels;

  2. 2.

    The managerial strengths and areas for improvement of health service managers; and

  3. 3.

    Any variation of the competence of health service managers between different healthcare organisations and settings.

Methods

This cross sectional descriptive study examined the competency levels and developmental needs of HSMs working in public hospitals and community health centres (CHS) in Victoria, Australia.

Ethics approval was obtained from the Human Research Ethics Committee of La Trobe University (HREC No. S15/75) prior to conducting the project. Consent to participate was sought from all participants online before the start of the competency assessment.

Participants

Health service managers from two public hospitals and five CHS in Victoria, Australia, were invited to participate in the assessments. Middle-level managers from these organisations were targeted for recruitment, with no specific exclusion criteria. For the purposes of this study, middle-level managers were considered those employed in roles classified as Level 4 (L4) managers in public hospitals and Level 3 (L3) managers in CHS. In Australia, management level is defined according to the supervision structure, where the Chief Executive Officer is the Level I manager of the healthcare organisation. The participants are referred to as the primary participating managers (PPM). Supervisors (more senior managers), peers (managers from the same management level within the same organisation) and reports (staff who report directly to the primary participating managers) of the PPMs were then invited to conduct assessments on the PPM. The organisations involved in the study were a convenience sample of partner organisations of the MCAP project. The organisations represented metropolitan and regional sites to improve generalisability of results across the state.

Competency assessment

The MCAP 360° subjective assessment included a self-assessment (completed by the PPMs themselves), a supervisor assessment, a peer assessment and a report assessment. A peer was a colleague working in the same area as the PPM, but not directly supervising or reporting to the PPM. The development of the MCAP assessment and identification of core managerial competencies for the healthcare sector is described in Liang et al. (2013a, 2013b) [11, 25]. The MCAP is a reliable and valid measure of HSM managerial competency [26].

Assessments were completed online using a web-based platform developed specifically by the research team for hosting and analysing all assessments. The web-based platform contained the MCAP Assessment Tool, previously outlined in Liang et al. (2017) [25]. To minimise entry error, the assessment process limited the numerical responses to each question and unanswered questions were labelled as missing. Questions relating to participants’ age, sex, years of management experience, educational backgrounds and professional memberships were also included in the subjective self-assessment to collect demographic information on the PPMs. Table 1 summarises the descriptors for the 7-point Likert scale used on each of the 79 behavioural items (Appendix) for the six competency areas of the MCAP assessment. For the purpose of this study, a score of five (competent, no guidance is required) or more is defined as ‘competent’. Scores less than five are defined as less than fully competent.

Table 1 Behavioural scale / levels for self- and colleague’s assessment

Data management and analysis

Raw data from the server hosting the website were downloaded into MS Excel files for consistency checking. The mean scores for each competency were calculated from the scores of the behavioural items, as were the ‘combined competency’ scores. In addition to the results for the four subjective assessment types (self, supervisor, peer and report), a ‘combined colleagues’ variable was calculated from the mean scores of the supervisor, peer and supervisee assessments to protect the confidentiality of the colleagues who completed the assessments on the PPM. In addition, item and competency scores were grouped into five levels (Not yet fully competent: < 3.0 and 3.0- < 4.0; Competent (requiring vs not requiring guidance): 4.0- < 5.0 and 5.0- < 6.0; Proficient or higher: 6.0–7.0). Scores less than 5.0 were considered less than competent. The data were then imported into into SPSS© version 22 for statistical analysis. Univariate analyses for both individuals and groups (management and organisational levels), and all demographic variables were completed. The distribution of continuous variables was checked for normality. Appropriate bivariate analyses to investigate the effect of possible competency predictors included age, sex, management level, number of years as a manager, education, professional membership and sector. The selection of tests depended on the distribution of the dependent variable (competency scores) and the independent variable type (continuous or categorical). In addition, t-tests or Mann-Whitney U tests were used to compare mean competency scores between the self-assessments and separately for the supervisor, peer and report assessments. Following this, multivariate analyses were performed using univariate analysis of variance and general linear modelling. These analyses were used for assessing the effects of multiple predictors (see above) of the mean competency scores. P values less than 0.05 were considered statistically significant.

The results are presented as mean scores for each behavioural item (79), the six core competencies and the combined competencies. The differences between the means of the supervisor and the self-assessment scores were calculated and analysed. The percentage of participants whose supervisors scored the participant greater or less than two standard deviations from the mean difference was calculated. In addition, the percentage of participants whose supervisors scored the participant greater or less than one standard deviation from the mean difference was calculated. Further details on score calculation for the MCAP are provided in Howard et al. (2017) [27].

Results

Ninety-three middle-level managers from two public hospitals (n = 25) and five CHS (n = 68) participated in the assessments (85% of the managers invited). Three-hundred and nineteen of their colleagues participated in the 360° subjective assessment. Colleagues included 90 supervisors, 95 peers and 134 reports. One hospital and two CHS sites decided to invite two peers and two reports to complete the assessments for each PPM, leading to the greater number of peer and report assessments. Participation was comprehensive by PPMs completing the 360° subjective assessments, with a near 100% completion rate for the 79 behavioural items provided to test the six core competencies. Of the 5250 total items assessed by the MCAP 360° of all PPMs, only 3% of item numbers had missing data points.

Demographic and educational background: Public hospitals

Of the 25 public hospital PPMs, 76% were female, and the average age of the group was 47 years. The median period for the PPMs in a similar position was 6.0 years. Eighty percent (20/25) of PPMs held postgraduate qualifications, of which 50% (10/20) were in health or business management. Fourteen of the 25 hospital managers (56%) had current professional membership associated with their clinical discipline. Only four of the 25 (16%) managers had current membership of the Australasian College of Health Service Managers (ACHSM) – the professional organisation for Australian HSMs. (See Table 2.)

Table 2 Demographic details of participants by sector

Demographic and educational background: CHS

Of the 68 CHS PPMs, 77% were female with an average age of the group of 44 years. The median period for the managers in a similar position was 5.0 years. Fifty-five percent (37/68) of PPMs held postgraduate qualifications, of which 41% (15/37) were in health or business management. Twenty-six of the CHS PPMs (38%) had maintained professional membership associated with their clinical discipline, and only one PPM had maintained membership of ACHSM. Table 2 summarises these results.

Overall competency scores for middle-level HSMs

Table 3 shows the mean ‘combined competency’ scores from the self-assessment (SA) and combined-colleague assessment (CCA) for PPMs, presented by sector and assessment types. Both the SA and CCA recorded mean scores were > 5 for both hospital and CHS managers. There were no statistically significant differences between the SA competency means and CCA competency means between sectors. However, the CCA scores were all significantly higher than the SA scores for both sectors (Mann-Whitney U = 446, p = 0.010) and (Mann-Whitney U = 2965, p = 0.004).

Table 3 Combined competencies score statistics by sector and assessment type

Table 4 summarises the mean scores for each of the six core competencies from the SA and CCA by sector. Mean scores received for each core competency from both SA and CCA were all greater than five. However, the scores from SA and CCA and between competencies varied. Scores received from the CCA for each of the competencies were consistently higher than those from the SA. For hospital PPMs, the differences were all statistically significantly (p ranging from < 0.0005 to 0.045). For the CHS PPMs, the differences were only statistically significant for C2 (Resources) (p = 0.011) and C3 Knowledge (p < 0.0005). There were no significant variations in the SA mean competency scores for each core competency, each behavioural item and the combined competencies across sectors and organisations.

Table 4 Mean scores for six competencies by sector and assessment type (self and combined colleagues)

Table 5 provides comparisons between the mean scores of the self-assessments and the mean scores of individual colleague assessments (supervisor, peer and report). Consistently, PPMs, supervisors and peers gave C4 (Communications) and C5 (Leadership) the highest scores amongst the six competencies. PPMs gave the lowest scores to C2 (Resources) and C3 (Knowledge) while supervisors and peers consistently gave the lowest scores to C1 (Evidence) and C6 (Change).

Table 5 Mean scores for six competencies by assessment type (all)

In general, the mean competency scores received from supervisors, peer and reports were higher than the mean competency scores as self-assessed by the PPMs. The analytical tests for statistical significance of the differences between the mean self-assessment scores and the mean individual colleague assessment scores were as follows (details available in Table 6):

  • Self-assessment versus supervisor scores: only the mean score for C3 (Knowledge) assessed by supervisors were statistically significantly higher than the mean self-assessment score;

  • Self-assessment versus peer scores: means for all the six competencies assessed by peers were statistically significantly higher than the mean self-assessment score;

  • Self-assessment versus report scores: means for all the six competencies assessed by reports were statistically significantly higher than the mean self-assessment score.

Table 6 Differences between mean self-assessment scores and mean colleague scores (supervisor (a), peer (b) and report (c)): t-tests for equality of means

Figure 1 shows the means scores for competency 3 (Knowledge) by assessment type. It is typical of all the analyses for the mean scores for competencies 1–6 and combined competencies.

Fig. 1
figure 1

Mean scores for competency 3 (Knowledge) by assessment type (sectors combined)

The percentage of PPMs assessing themselves in low, medium and high score categories are provided in Table 7. Between 18 and 24% of managers assessed themselves as less than fully competent (< 5.0) for competencies C1 (Evidence), C2 (Resources, C3 (Knowledge) and C6 (Change). Conversely, more PPMs assessed themselves fully competent or higher (scores 5.0–7.0) for competencies C4 (Communications) (91%) and C5 (Leadership) (89%). However, there were no significant differences in the distribution of competency groups between the management levels for all 79 behavioural items, the six competencies or the combined competencies scores.

Table 7 Percentage of managers at different competency levels (grouped)

Between three and 4 % of participants assessed themselves more than two standard deviations higher (between 1.8 and 2.3 points on the behavioural scale) than their supervisors, suggesting overconfidence by the participant (Table 7). Seventy-five to 80 % of these were CHS level III managers. Between 12 and 18% of participants assessed themselves more than one standard deviations higher (0.7 to 1.1 points) than their supervisors (Table 8). In addition, between 14 and 19% of supervisors assessed the participant more than one standard deviation higher (1.1 to 1.4 points) than the self-assessment.

Table 8 Number and percentage of self-assessment scores greater or less by more than two standard deviations or by more than one standard deviation compared to the supervisor scores

Predictors of competency

Age was positively correlated with the means of the self-assessed six core competencies and the combined competency scores (correlation coefficients range: 0.25 to 0.35; p values ranging from 0.017 to 0.001). However, when adjusted for length of service as a middle level manager in a generalised linear model, age ceased to be significantly related (p values range: 0.56 to 0.186). Sex, length of service as a level 3/4 HSM, educational experience and membership of a professional institution were not statistically associated with the means of the self-assessed six competencies or the combined competencies both by the appropriate bivariate analyses, univariate analyses of variance and generalised linear modelling.

Identification of potential behavioural items for improvement of individual primary participating managers (PPM)

There were great variations of scores between participants as self-assessed for the individual behavioural items in each of the competencies. Table 9 provides the mean scores from the self-assessments and the combined colleagues’ assessments for each of the competencies for three PPMs selected to represent low, average and high scorers. The table also presents the number of behavioural items that received a mean score less than five (less than fully competent) for each of the competencies. Behavioural items receiving a score less than five suggest that the PPM would benefit from further training. Where there are more items for the SA that are low scoring, this suggests a participant has underestimated their competency compared to their supervisor’s assessment. Where there were more low scoring items presented by the supervisor, the ‘high’ scoring participants had overestimated their competency compared to their supervisors’ assessments. This provides insight to their perception of competency, and can be used to explore where PPMs could improve their overall managerial competence, targeting professional development opportunities to the identified low scoring behavioural items. This demonstrates the applicability of the tool to individual managers to demonstrate competency gaps for targeted professional development.

Table 9 Results from three participating managers: competency means and number of items with scores less than five; self- and combined colleagues’ assessments

Discussion

This study has found the majority of middle-level HSMs in Victorian public hospital and community health centres demonstrate the core managerial competencies adequately and have some level of experience in managing health services. Both the SA and CCA mean scores from the MCAP 360° assessments were greater than five for all competencies measured. Furthermore, the proportion of HSMs that self-assessed scores above 5.0 ranged from 76 to 91%. According to the MCAP competency scale, a mean competency score 5.0 or higher indicates that the assessed participant is perceived as competent in applying the core behaviours in their managerial roles independently [27]. These results suggest that the competency level of the HSM workforce in Victoria is mostly appropriate for their roles and responsibilities.

Inevitably, there was some variation in competency assessment scores between managers employed across Victoria, with some scoring highly and some scoring low. The highest and the lowest mean combined competency scores from the self-assessment for hospital L4 managers were 6.46 and 4.22 respectively, while for CHS L3 managers they were 6.63 and 3.81 respectively. Discrimination between those excelling at core managerial responsibilities and those requiring support could be a powerful tool for healthcare organisations. Identifying top performing managers for “recruitment, competency management, learning and development, performance management, compensation and succession planning” is an important strategy in managing talent at the organisation level ([28], p41). An ability to identify the current workforce with an aptitude for managerial competency may lead to opportunities to recruit such people to positions of management strategically, and future research to explore the role of the MCAP in succession planning is warranted.

Two competencies were identified as having the largest proportion of managers demonstrating proficiency or higher (scores greater than six); number 6 (enabling and managing change) and number 4 (interpersonal, communication qualities, relationship management). The latter finding is not surprising, as the competency of ‘interpersonal, communication qualities and relationship management’ has long been identified as important for managers [23, 28,29,30] forming an integral component of professional and management training. Further, these competencies are also evident in clinical practice, with the Australian Health Practitioner Regulation Authority National Boards of Medicine, Nursing and the allied health disciplines all stipulating such behaviours in practice thresholds and standards (https://www.ahpra.gov.au/) The majority of the participants involved in this study were clinicians prior to commencing their HSM role, and 40% of participants still maintained membership of their clinical professional organisation. Staff transitioning from a clinically focussed role to a managerial role will have developed their competencies during their career as a clinician. Understanding how to support individuals transitioning from clinician to HSM role to transform their work competencies from the context of clinical practice to the context of HSM is research that should to be completed.

Apparent opportunities for improvement in competence of HSMs has been confirmed through this study, with 9–24% of participants scoring less than 5.0 in the self-assessment of a core competency. A mean score 4.0 or higher, but less than 5.0 indicates that the participant is competent but requires occasional guidance, suggesting a proportion of HSMs require support in their role. The competencies of ‘evidence-informed decision-making’ (20%), ‘resource management’ (23%) and ‘demonstrated knowledge of healthcare environment and the organisation’ (24%) were the three competencies that appear to require strengthening among participating HSMs, with the greatest proportion of participants scoring less than 5.0. Managers currently make limited use of the available evidence relevant to management and organisational practices [31, 32]. This finding may be related to the more recent addition of ‘evidence-informed decision-making’ being considered an essential competency for HSMs [22, 30]. A recent review of the formal Health Administration programs conducted within Australian universities, confirmed that ‘evidence-informed decision-making’ has not been widely accepted as a learning objective of the 13 Master of Health Administration degrees in Australia [10]. Evidence-based approaches to management can add to ‘substantive expertise’ ahead of ‘fads and popular tools’ of the time ([33], p84). Further emphasis on implementing an evidenced-based approach to management should be considered for HSM professional development curriculum and evaluation of the learning and impact on patient outcomes with development of competency in HSM should be completed.

Low performance may not necessarily be simply the capability or ‘fault’ of the individual receiving lower scores, as performance can be related to confidence or self-efficacy, insufficient opportunities to develop competency, low access to training and education opportunities, or unrealistic expectations of the team leader/manager or organisation [34]. The observed MCAP scores received from the CCA for each competency were consistently higher than those from the SA. Where an individual is lacking confidence, they may be more likely to rate themselves lower. The lower SA than CCA scores are consistent with previous literature that has demonstrated higher performance ratings are received from supervisors, peers and reports compared to self-ratings [34]. However, there was a subgroup of participants who assessed themselves higher than their supervisors (Table 8). This may be an example of the 'Dunnung-Kruger' effect which is a cognitive bias among people of low capability who asses their cometency as greater than it is [35, 36].This study did not measure potential moderating factors that may influence MCAP scores, and therefore future research is needed to explore the relationship between competency scores and confidence, experience and self-efficacy to understand reasons for low or high scores. From the organisation’s perspective, it is important to engage the low performing manager effectively to identify reasons for lower competency scores, such that skills and competencies requiring improvement can be targeted in professional development plans that are formulated to address the lower levels of observed competence.

Variation of competency between sectors and organisations

In addition to variations between individuals within the same sector and managerial level, there are variations between mean scores received for six competencies between hospital and community health sector HSMs, with managerial strengths and weaknesses varying between sectors and organisations, in both the SA and CCA data. While competency requirements may be similar between sectors and across management levels, the actual demonstration of the competencies appears to vary, suggesting that the learning and professional develop needs differ. The design of ongoing management education and development should be context sensitive [37]. The constant reform of health systems across Australia and internationally has demonstrated the complexity of the HSM role [13, 14] and both practitioners and academics have called for a better understanding of the context of the HSM role in developing a capable, resilient and sustainable health management workforce [3]. The results from this study reaffirm this requirement and indicate that a ‘one size fits all’ approach to training is not likely to be effective. While system-wide strategies are important to set the training and development direction, separate consideration should be given to different management levels and groups from different sectors and from different organisations to contextualise the learning. Workforce development strategies need to be formulated in collaboration between policy-makers and managers at system, organisational and team levels, and future research evaluating the impact of such models of education, professional development, and training on health service provision is needed [38, 39].

Limitations of the study

Although there is evidence linking management competency and individual performance [14], competency is context sensitive [37] and management competency may be influenced by various organisational factors. The participants were volunteers from organisations who agreed to participate and were not randomly selected for involvement. While this may not affect the internal validity of the study, it raises the question of the generalisability of the results to other populations of healthcare managers. Further evaluation beyond the Australian state of Victoria is needed to understand competency levels nationally.

Self-assessment is recognised as a potentially inaccurate method of assessing competence and may not reflect performance [40, 41, 42, 43], and this study measured competence by both self-assessment and colleague assessments subjective perspective only. Measurement of performance as a quantitative outcome is needed to confirm the relationship between self- or colleague assessed competency. While this study makes a strong addition to the literature and understanding competency levels of HSM in Australia, future studies should aim to examine manage competency levels and performance simultaneously.

Practice implications for healthcare sectors and organisations

The exemplars provided in Table 6 illustrate the possible individual assessment data that can be provided through completion of the MCAP 360° assessment. The MCAP assessment not only provides the competence level for each of the core competencies, but also identifies behavioural items requiring improvements amongst each of the core competencies from self and colleagues’ perspectives. The five areas that the selected participating managers self-identified as requiring improvement were as follows:

  • Use timely and appropriate questioning/investigation to identify the nature of a problem, issue or opportunity;

  • Anticipate decision implementation problems/ impacts and develops and communicates appropriate contingency plans;

  • Manage budgets in accordance with organisational objectives;

  • Anticipate and plan for changes in policies affecting funding to the organisation/unit;

  • Empower others to achieve goals.

This rich information about an individual manager supports the supervisor to design an individual professional development program, targeted at both specific behavioural items highlighted for development, but also specific behavioural items that are contextual to the organisation and specific to the role and tasks of the manager.

A focus on competency approaches such as the MCAP is not just about the performance of individual managers, but there is also the potential for such frameworks to be embedded within an organisation’s strategic direction. The ‘resource-based view of strategy emphasises the importance of an organisation’s core competencies in achieving sustainable advantage’ [38]. These core organisational competencies link to the internal capabilities of the organisation, which, in health care, substantially support the competence of the main resource, the workforce, and the human resource function that underpins it. Competency modelling, for which the MCAP provides the means, is an important innovation of process to assist organisations focus on job-related information and the skills required to manage others. Competency modelling may be useful to the agenda of secure job analysis, to distinguish performance of individuals, and to monitor and analyse how competencies might change at differing organisational levels or over time. Linking competencies of leaders, managers and employees to organisational objectives and strategies strengthens the potential use of the human resource capacity to respond to organisational sustainability [38, 44].

Conclusion

The competency of the HSM workforce in Victoria, Australia, is appropriate for the roles and responsibilities the HSMs are expected to achieve. However, a relatively significant proportion of HSMs are demonstrating less than optimal competence that requires support or guidance suggesting that there is a learning professional development need. The most apparent area for development is the application of evidence-based decision making. Variations in competency are observed between hospital and community health sectors, indicating that professional development opportunities need to be contextualised for management level, sector and organisation. Further examination of HSM competency using the MCAP competency-based approach across other Australian States and in international contexts will provide a foundation to embed managerial competency framework into performance management and professional development of HSMs.

Abbreviations

ACHSM :

Australasian College of Health Service Managers

CCA :

Combined colleague assessment

CHC :

Community health centres

HSM :

Health services managers

MCAP LMCF :

Management Competency Assessment Partnership Leadership and Management Competency Framework

MCAP :

Management Competency Assessment Partnership

PPM :

Primary participating managers

SA :

Self-assessment

References

  1. Mabey C, Ramirez M. Does management development improve organizational productivity? A six-country analysis of European firms. Int J Hum Res. 2005;16:1067–82.

    Google Scholar 

  2. Macpherson A, Holt R. Knowledge, learning and small firm growth: a systematic review of the evidence. Res Pol. 2007;36(2):172–92.

    Article  Google Scholar 

  3. Walston S, Khaliq A. The importance and use of continuing education: findings of a National Survey of hospital executives. J Health Admin Educ. 2010;27(20):113–25.

    Google Scholar 

  4. Cummings GC, MacGregor T, Davey M, Lee H, Wong CA, Lo E, Muise M, Stafford E. Leadership styles and outcome patterns for the nursing workforce and work environment: a systematic review. Int J Nurs Stud. 2010;47(3):363–85.

    Article  Google Scholar 

  5. National Health and Hospital Reform Commission. A healthier future for all Australians – Final report. Canberra: NHHRM; 2009.

    Google Scholar 

  6. O’Brien-Pallas L, Griffin P, Shamian J, Buchan J, Duffield C, Hughes F, et al. The impact of nurse turnover on patient, nurse and system outcomes: a pilot study and focus for a multicentre international study. Policy Politics Nurs Pract. 2006;7(3):169–79.

    Article  Google Scholar 

  7. Francis R. Report of the mid Staffordshire NHS Foundation trust public inquiry – volume 3: present and future. London: NHS Foundation Trust; 2013.

    Google Scholar 

  8. Willis P. Quality with compassion: the future of nursing education: report of the Willis Commission. London: Royal College of Nursing; 2012.

    Google Scholar 

  9. Martins JM, Isouard G. Health service managers in Australia part 1: service, geographical and category distribution. Asia Pac J Health Manag. 2012;7(2):16–28.

    Google Scholar 

  10. Liang Z, Howard P, Leggat S. Competence of health service managers: how do we assess them? Paper presented at the Society of Health Administration Program in education symposium. In: Brisbane; 2013.

    Google Scholar 

  11. Liang Z, Howard P, Koh L, Leggat S. Competency requirements for middle and senior managers in community health services. Aust J Prim Health. 2013;19:256–63.

    Article  Google Scholar 

  12. Stefl ME, Bontempo CA. Common competencies for all healthcare managers: the healthcare leadership alliance model. J Healthcare Manag. 2008;53(6):360–73.

    Article  Google Scholar 

  13. Liang Z, Leggat S, Howard P, Koh L. What makes a hospital manager competent at the middle and senior levels? Aust Health Rev. 2013b;37:566–73.

    Article  Google Scholar 

  14. Landry A, Stowe M, Haefner J. Competency assessment and development among health-care leaders: results of a cross-sectional survey. Health Serv Manag Res. 2012;25:78–86.

    Article  Google Scholar 

  15. Briggs DS. SHAPE declaration on the organisation and management of health services: a call for informed public debate. Asia Pacific J Health Manag. 2008;3:10–3.

    Google Scholar 

  16. Maurer J, Kimberly A, Haefner J, Stuart A, William C. Beliefs about 'improvability' of career-relevant skills: relevance to job/task analysis, competency modelling, and learning orientation. J Org Behav. 2003;24:107–31.

    Article  Google Scholar 

  17. Campbell C, Lomperis A, Gillespie K, Arrington B. Competency-based healthcare management education: the saint Louise University experience. J Health Admin Educ. 2006;23:135–68.

    Google Scholar 

  18. Fine D, O'Connor S, Shewchuk R. Building an understanding of the competencies needed for health administration practice. J Healthcare Manag. 2005;50(1):32–47.

    Article  Google Scholar 

  19. Ritchie DJ, Yen ML. Health services management development: what formal knowledge should support the skills and experience required? Aust Health Rev. 2013;37(2):189–93.

    Article  Google Scholar 

  20. Liang Z, Isouard G, Briggs D, Leggat S, Howard P. Developing competent health service managers: how much do we know and how much have we done? Sydney: The Society of Health Administration Program in Education Symposium; 2014.

    Google Scholar 

  21. Bradley E, Cherlin E, Busch S, Epstein A, Helfand B, White W. Adopting a competency-based model: mapping curricula and assessing student progress. J Health Admin Educ. 2008;25:37–51.

    Google Scholar 

  22. Isouard G, Martins JM, Friedman L. H. Competency in innovation, creative and innovative thinking: challenges within the health management course curriculum. J Health Admin Educ. 2015;32(3):257–69.

    Google Scholar 

  23. Catano V, Campbell C. Performance appraisal of behavior-based competencies: a reliable and valid procedure. Pers Psychol. 2007;60:201–30.

    Article  Google Scholar 

  24. Garman A, Tyler J, Darnall J. Development and validation of a 360-degree-feedback instrument for healthcare administrators. J Healthcare Manage. 2004;49:307–21.

    Article  Google Scholar 

  25. Liang Z, Howard P, Leggat S, Bartram T. Development and validation of health service management competencies. J Health Organ Manag. 2018;32:157–75.

    Article  Google Scholar 

  26. Liang Z, Howard P, Leggat S. 360° management competency assessment: is our understanding adequate? Asia Pac J Hum Resour. 2017;55(2):213–33.

    Article  Google Scholar 

  27. Howard P, Liang Z, Leggat S, Karimi L. Validation of a management competency assessment tool for health service managers. J Health Organ Manage. 2018;32:113–34.

    Article  Google Scholar 

  28. Schachter D. Managing your talent will ease the looming rush to retirement. Inf Outlook. 2008;12:40–1.

    Google Scholar 

  29. Calhoun J, Dollett L, Sinioris M, Wainio J, Butler P, Griffith J, Puttallo J, Warden G. Development of interprofessional competency model for healthcare leadership. J Healthcare Manage. 2008;53:375–91.

    Article  Google Scholar 

  30. Liang Z, Howard PF. Competencies required by senior health executives in NSW, 1990 -1999. Aust Health Rev. 2010;34:52–8.

    Article  Google Scholar 

  31. Kovner A, Rundall T. Evidence based management reconsidered. Front Health Serv Manag. 2006;22:3–22.

    Article  Google Scholar 

  32. Liang Z, Howard P, Leggat S, Murphy G. A framework to improve evidence-informed decision-making in health service management. Aust Health Rev. 2012;37:566–73.

    Article  Google Scholar 

  33. Liang Z, Howard P. Evidence-informed managerial decision-making - what evidence counts? (part two). Asia Pac J Health Manage. 2011;6:12–21.

    Google Scholar 

  34. Church A. Do higher performing managers actually receive better ratings? A validation of multirater assessment methodology. Consult Psychol J. 2000;52:99–116.

    Article  Google Scholar 

  35. Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing One's own incompetence leads to inflated self assessments. J Personal & Soc Psych. 1999;77(6):1121–34.

  36. Schlosser T, Dunning D, Kerri L, Johnson K, Kruger J. How unaware are the unskilled? Empirical tests of the “signal extraction” counterexplanation for the Dunning–Kruger effect in self-evaluation of performance.J Econ Psych 2013; 39: 85-100

  37. Epstein R, Hundert E. Defining and assessing professional competence. J Am Med Assoc. 2002;287:10.

  38. Sitzmann T, Ely K, Brown KG, Bower KN. Self-assessment of knowledge: a cognitive learning or affective measure? Acad Manag Learn & Educ. 2010;9(2):161–91.

    Article  Google Scholar 

  39. Decker PJ, Durand R, Ayadi F, Whittington W, Kirkman D. Self-assessment of management competencies and intention to change. Acad Educ Leader J. 2014;18(4):129–48.

    Google Scholar 

  40. Dunnette M. My hammer or your hammer. Hum Resour Manag. 1993;32:373–84.

    Article  Google Scholar 

  41. Brett J, Atwater L. 360 degree feedback: accuracy, reactions and perceptions of usefulness. J Appl Psychol. 2001;86:930–42.

    Article  CAS  Google Scholar 

  42. Smither J, London M, Reilly R. Does performance improve following multisource feedback? A theoretical model, meta-analysis, and review of empirical findings. Pers Psychol. 2005;58:33–66.

    Article  Google Scholar 

  43. Clardy A. The strategic role of human resource development in managing core competencies. Hum Resource Dev Int. 2008;11:183–97.

    Article  Google Scholar 

  44. Campion M, Fink A, Ruggeber B, Carr L, Phillips G, Odman R. Doing competencies well: best practices in competency modelling. Pers Psychol. 2011;64:225–62.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This study was funded by three industry partners (Victorian Department of Health, Victorian Healthcare Association and the Australasian College of Health Service Managers) and through in-kind contributions of 14 Victorian Health Services. The funding supported the collection of data through the employment of a research assistant.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Author information

Authors and Affiliations

Authors

Contributions

All authors have read and approved the manuscript. ZL was the project leader, conceptualising the research question, designing methodology, collecting data, analysing data, interpreting data and writing the manuscript. FCB was involved in review of the literature for this manuscript, data analysis, interpretation of results and writing the manuscript. PFH was involved in conceptualising the research question, designing methodology, collecting data, analysing data, interpreting data and writing the manuscript. DB was involved in interpretation of the data and preparation of the manuscript. SL was involved in conceptualising the research question, designing the methodology, data interpretation and writing of the manuscript. DW was involved in creation of the online MCAP assessment tool, and thereby collection of data. He was also involved in data analysis and interpretation, and preparation of the manuscript. DE was involved in interpretation of the data analysed and preparation of the manuscript. AR was involved in interpretation of data analysis, and writing of the manuscript.

Corresponding author

Correspondence to Felicity C. Blackstock.

Ethics declarations

Ethics approval and consent to participate

Ethics approval was obtained from the Human Research Ethics Committee of La Trobe University (HREC No. S15/75) prior to conducting the project. Consent to participate was sought from all participants online before the start of the competency assessment.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests and no conflicts of interest related to the publication of this manuscript.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Six leadership and management competencies with some behaviour examples

C1 Evidence-informed decision-making

C1.1 Use timely and appropriate questioning/investigation to identify the nature of a problem, issue or opportunity.

C1.6 Use evidence to question and improve existing practice and processes

C1.10 Set and use measures to evaluate decision outcomes

C1.11 Support and encourage colleagues and subordinates to use evidence to guide decision-making.

C2 Operations, administration and resource management

C2.3 Interpret basic financial statements

C2.5 Develop budgets in accordance with organisational objectives

C2.8 Design and develop appropriate roles and reporting structure (across a range of areas) in accordance with organisational objectives

C2.13 Conduct regular two-way performance review & development discussions to support staff development

C3 Demonstrated knowledge of healthcare environment and the organisation

C3.3 Demonstrate understanding of the roles of key stakeholders in health and how they interact

C3.4 Demonstrate understanding of the highly professionalised health workforce

C3.5 Apply relevant legislation and accountability frameworks specific to healthcare settings

C3.11 Effectively navigate organisational structures, roles and relationships in order to achieve work goals

C4 Interpersonal, communication qualities and relationship management

C4.4 Engage confidently and constructively in verbal and non-verbal interactions with others

C4.6 Produce written reports/materials, which are appropriate for both audience and purpose

C4.7 Invest time and effort in working and engaging with stakeholders

C4.17 Show awareness of, and sensitivity to, the feelings of others

C5 Leading people and organisation

C5.5 Balance the values and priorities of both organisation and profession(s)

C5.6 Lead, develop and evaluate performance to build an effective team

C5.8 Adapt leadership style to suit the situation

C5.9 Establish and maintain a personal and professional support network

C6 Enabling and managing change

C6.1 Explain the need for change in an effective way

C6.2 Assess readiness for change and plans accordingly

C6.8 Evaluate the processes and outcomes of change

C6.9 Recognise and tolerate ambiguity

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, Z., Blackstock, F.C., Howard, P.F. et al. An evidence-based approach to understanding the competency development needs of the health service management workforce in Australia. BMC Health Serv Res 18, 976 (2018). https://0-doi-org.brum.beds.ac.uk/10.1186/s12913-018-3760-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12913-018-3760-z

Keywords