Is organizational progress in the EFQM model related to employee satisfaction?

Abstract

Background

To determine whether there is greater employee satisfaction in organisations that have made more progress in implementation of the European Foundation for Quality Management (EFQM) model.

Methods

A series of cross-sectional studies (one for each assessment cycle) comparing staff satisfaction survey results between groups of healthcare organisations by degree of implementation of the EFQM model (assessed in terms of external recognition of management quality in each organisation). Setting: 30 healthcare organisations including hospitals, primary care and mental health providers in Osakidetza, the Basque public health service. Participants: Employees of 30 Osakidetza organisations. Intervention: Progress in implementation of EFQM model. Main outcome measures: Scores in 9 dimensions of employee satisfaction from questionnaires administered in healthcare organisations in 4 assessment cycles between 2001 and 2010.

Results

Comparing satisfaction results in organisations granted Gold or Silver Q Awards and those without this type of external recognition, we found statistically significant differences in the dimensions of training and internal communication. Then, comparing recipients of Gold Q Awards with those with no Q Certification, differences in leadership style and in policy and strategy also emerged as significant.

Conclusions

Progress of healthcare organisations in the implementation of the EFQM Excellence Model is associated with increases in their employee satisfaction in dimensions that can be managed at the level of each organisation, while dimensions in which no statistically significant differences were found represent common organisational elements with little scope for self-management.

Background

The public Basque Health Service, Osakidetza, is a public body run by the Regional Government of the Basque Country with the mission of providing health services to the population of the autonomous region, around 2,200,000 people. Osakidetza is formed by a set of service provider organisations (including acute hospitals, medium- and long-stay hospitals, psychiatric hospitals, primary care organisations, mental health networks, and integrated healthcare organisations, as well as a centre for transfusion and human tissues and an emergency services provider), the number and structure of which have varied over the course of the study (from 31 service providers in 2001 to 26 in the present day).

The healthcare organisations that compose the Basque Health Service are granted different levels of independence and responsibility depending on the management area. Hence, while some areas, such as strategic policies, and the contracting and payment of staff, are coordinated/centralised at the corporate level; responsibility for others, such as the development of local management plans, training, the organisation of processes and the management of alliances, is decentralised, with the separate healthcare organisations being allowed a greater level of independence.

Osakidetza has been involved in quality management activities since 1992. Some of the first milestones along this path were becoming a partner of Euskalit, the Basque Foundation for Quality Promotion [1] (an organisation that promotes improvements in management in the Basque region) and adopting in 1995 the European Foundation Quality Management (EFQM) Excellence Model [2] as a model of reference for total quality.

At the corporate level, the adoption of the EFQM model was driven by a range of strategies and measures, including the adaptation of the model to the healthcare setting, an extensive programme of staff training in this management model and the establishment of a schedule of two-yearly self-assessments, through which healthcare organisations are encouraged to use the EFQM model on a voluntary basis [3–6]. At the same time, a series of tools and methodologies were adapted and introduced to favour the rolling-out of the EFQM model [7], including the 5-s principles [8], management by processes [9–11], the development of strategic plans [12–14], satisfaction surveys with both patients and health service staff [15], balanced scorecards, and ISO 9000, 14000 and 18000 certification, as well as other activities in the fields of management innovation and social corporate responsibility.

In this general corporative context, the EFQM has not been adopted evenly across the health service, with healthcare organisations progressing at different paces and reaching different degrees of implementation of the model. In line with this, a range of external awards have been received by different organisations, including recognitions both regionally (7 Gold and 21 Silver Qs for Quality from the Basque Government) and internationally (1 European Special Prize and 2 Iberoamerican Quality Prize).

One of the areas that the EFQM model aspires to influence is the perception by staff of the organisation in which they work (EFQM criterion 7 a) [2]. Given this, 15 years after Osakidetza decided to adopt the Total Quality Principles and EFQM Excellence Model to drive improvements, this study set out to explore whether there was a relation between the degree of implementation of the model in different healthcare organisations of the regional service and the mean level of satisfaction among employees in these organisations.

There is evidence in the literature of a positive association between use of the EFQM model and the performance of organisations in the industrial sector [16–18], with studies generally focussing on economic and financial results [16, 18]. In the health sector, however, as revealed by the review of Minkman et al. [19], there is only weak evidence of the impact of this model, it being almost exclusively limited to descriptive accounts. These include descriptive studies of the progress in rolling out the EFQM model in healthcare organisations in Osakidetza [4, 20].

There is some more sound evidence from the USA, regarding the Baldrige Health Care Criteria Framework, a total quality management similar to the EFQM model. Specifically, a study [21] in 220 hospitals demonstrated a significant positive association between meeting the criteria in the various categories of the framework and performance in the results’ dimensions. In particular, the strongest association was found between degree of adherence to the quality improvement model and the “staff and work systems results” dimension, this being interpreted by the authors as evidence of a relationship between a quality culture in an organisation and greater employee involvement and satisfaction, consistent also with the results of authors such as Shortell et al. on the positive association between quality improvement implementation and perceived human resource development [22].

In this context, the present study aims to assess whether there is an association between the degree of implementation of the EFQM model in 30 healthcare organisations of Osakidetza, at various time points (on the basis of a series of assessments between 2001 and 2010), and the perception of the health service employees, in terms of their level of satisfaction.

Methods

A cross-sectional study design was used for each period under consideration. As in previous studies [16, 18], whether an organisation had received (or not) an external quality award was used as a proxy for the degree of implementation of the EFQM model; the granting of an award being regarded as a milestone in the organisation, the culmination of previous efforts to deploy quality management based on this excellence model. Specifically, the external assessments considered were the Silver (>400 points) and Gold (>500 points) Q Awards granted by the Basque Government.

Employee perceptions (EFQM criterion 7a, “People results”) were compared, on the one hand, in the organisations which had obtained some type of Q award (Silver or Gold) and those which had not and, on the other, in organisations given the Gold Q Award (the highest level of excellence in terms of the model considered) and those without any Q recognition.

Assessment of employee perceptions of the organisations in which they work was based on the staff satisfaction surveys that have been carried out periodically by service providers in Osakidetza since 2001. There were various cycles of assessment (2001–2003, 2004–2006, 2007–2008, and 2009–2010), the questionnaire and data collection procedure being modified/adapted in each cycle and, hence, the results are not fully comparable between these periods. For this reason, the statistical analysis was limited to cross-sectional comparisons within each assessment cycle, longitudinal analysis being avoided. For the purposes of this study, the results were standardised across the surveys on a scale of 0 to 10.

The construct validity and reliability of the satisfaction questionnaire used with the health service employees was tested on the basis of a sample of 7,145 questionnaires (corresponding to 28.7% of the entire staff of Osakidetza) conducted in year 2010. On the one hand, principal component analysis was performed confirming the existence of 9 separate dimensions within the satisfaction construct, which together explained 70% of the variance of the construct. Comparisons were made between the organisations for each of these dimensions. Further, the high reliability of the tool was confirmed by calculating Cronbach’s alpha, both for the complete set of items on the questionnaire (0.97) and for the items in each dimension. A description of the items of the questionnaire and the corresponding values of Cronbach’s alpha are listed by dimension in Table 1.

Table 1 Dimensions and items on the staff satisfaction questionnaire used in the Basque health service (2010) and Cronbach’s alpha by dimension from 2010 survey results

The population covered by these surveys was the entire workforce (both permanent as well as temporary staff who had been working in the organisation for at least six months). Table 2 summarises data on the size and composition of the samples of employees who completed the questionnaire in each period.

Table 2 Staff satisfaction survey sample size by assessment cycle and healthcare organisation

For this study, data were collected for 30 healthcare organisations including acute, psychiatric, and medium- and long-stay hospitals, as well as providers of outpatient mental health services and primary care. The statistical analysis was carried out independently for each of the four assessment cycles studied and, within each cycle, data were grouped into categories to allow two separate comparisons: i) organisations that had received some type of Q Award (regardless of whether it was Silver or Gold) vs. those without this type of external recognition; and ii) organisations with a Gold Q vs. those with no awards. In a given assessment cycle, organisations were considered to have received external recognition if they had been given a Q Award during or at any time prior to the corresponding cycle.

Table 3 shows the sizes of the resulting groups for each assessment cycle. As can be seen, the sample size is relatively small in some cases. For instance, by the end of the first cycle, only one organisation had obtained a Gold Q Award. As the statistical techniques used cannot be applied to a single observation, for this cycle it was not possible to make the comparison between recipients of Gold Q Awards and organisations without this type of external recognition. Further, in two groups there are just four organisations (the group with some type of Q Award in 2001–2003, and that with Gold Q Awards in 2004–2006); for these cases, the results should be interpreted with caution.

Table 3 Number of healthcare organisations in each study group by assessment cycle

Given that the objective of this study was to assess whether organisations with Q Awards were rated more highly than those without this type of external recognition, the null hypothesis of equality of the means was tested, with a one-tailed independent samples t-test. Prior to this test, however, the Shapiro-Wilk test was conducted to check whether the data was normally distributed. In cases where the normality assumption was violated, non-parametric tests such as the Kruskal-Wallis test were used instead.

This study is framed within a research project that was presented and approved in a commissioned research call granted by the Basque Office for Health Technology Assessment (Osteba) an agency of the Basque Government’s Department of Health. During the project approval process, it was externally reviewed and evaluated and it was decided that it is not necessary to have an Ethics Committee approval in order to ensure compliance with existing conventions and standards in research.

The results of the questionnaires are based on Osakidetza’s (Basque Health Service) professionals responses and were provided by the Quality and Human Resources Department of Osakidetza. Involved professionals participation was voluntary and anonymous, and obtained data was treated as absolutely confidential and anonymous, according to Spanish data privacy law.

Results

The results of the comparisons between organisations which had and had not received Q Awards and between recipients of Gold Q Awards and those without this type of external recognition are summarised in Tables 4 and 5. Each cell contains, for each assessment cycle and dimension analysed, the differences observed in the mean scores (on a scale from 0 to 10) for those which had and had not received the awards and the standard deviation (in brackets).

Table 4 Differences in mean results ( t -test) on staff satisfaction questionnaire (0–10 scale) between the group of organisations with Q certification (Silver or Gold) and those without certification, per cycle and dimension

Table 5 Differences in mean results (t- test) on staff satisfaction questionnaire (0–10 scale) between the group of organisations with Gold Q Award and those with no Q certification, per cycle and dimension

Regarding the level of significance of these differences, the cells with bold numbers correspond to the cases when the one-sided t-test produced a p-value of less than 0.05. This means that for these dimensions there was sufficient evidence in these cycles to state, with a confidence of 95%, that organisations which do have external recognition were rated more highly by their employees than those which do not. Four cells in Table 4 are marked with an asterisk indicating that, as the normality assumption was violated, the value in the cell corresponds to the result of the non-parametric Kruskal-Wallis test, rather than the t-test. Accordingly, the standard deviation is not given for these cases.

Comparison between organisations with some type of Q Award vs. those with none

First, we present the comparison between recipients of some type of Q Award (Gold or Silver) and those without any Q Awards (Table 4). In the first assessment cycle (2001–2003), statistically significant results were only found for one dimension. The group of organisations that had received Q Awards was small, however, and we should, therefore, be cautious in the interpretation of these results.In the other periods, ratings were consistently higher in the group with, compared to that without Q Awards, in two dimensions (p <0.05): Training and professional development, and Internal communication (Figure 1). In addition, although with a lower level of significance (0.05 < p < 0.10), organisations with Q Awards performed better in the areas related to Leadership and management style and Policy and strategy. A similar trend was observed in indicators concerning Working conditions and Working environment. In most of the periods, however, it was not possible to reject the null hypothesis that the means were equal for the areas of Health and safety at work and Pay.

Figure 1

Boxplots comparing the groups for the two variables that showed largest differences throughout the entire period.

Comparison between organisations with Gold Q Awards vs. those with no awards

The comparison between recipients of Q Gold Award and those without any Q Awards is presented in Table 5. In the first period, it was not possible to compare results for recipients of Gold Q Awards with those that had received no awards as the sample size was not sufficiently large. In the rest of the periods, consistent with the fact that there were greater differences in terms of degree of implementation of the EFQM model between these groups, the pattern was much more clear for the four dimensions identified in the first analysis, namely those related to Training and professional development, Internal communication, Leadership style and Policy and strategy. These differences between organisations with Gold Q Awards and those with no awards were highly significant for all the three periods considered. On the other hand, for the dimension related to Working conditions, the differences were only significant in two periods and less strongly so. As regards Health and safety at work and Pay, no significant differences were found between the groups in any of the assessment cycles. There were also no significant differences for Working environment. For Technical and material resources there was only a significant difference in the last assessment cycle.

Discussion

The results in this study indicate that the satisfaction of health service employees in organisations that have made the most progress towards the implementation of the EFQM model (as reflected in the possession of a Gold Q Award) was significantly greater in several areas, namely training, internal communication, leadership style and strategy/policy, than among those working for organisations that have not gone so far in the implementation of the model (without any Q award). These differences were still significant in the training and communication factors when comparing recipients of any type of Q Award with those who have yet to receive this type of external recognition.

Comparing the differences in the two analysis (with vs. without Q Awards, and with a Gold Q vs. without any of this type of award), in the periods 2007–2008 and 2009–2010, we found that the magnitude of the differences between organisations with a Gold Q Award (those that had made more progress in terms of the EFQM) and those without any awards is greater than between organisations with and without any type of Q Award, across all of the dimensions with highly significant differences (p <0.05). It seems therefore that the differences in term of satisfaction increase with progressive deployment of the model (being the greater among employees of organisations with Gold than Silver Q Awards).

This study contributes to the evidence showing the long-term impact of a complex organisational intervention such as the adoption of the EFQM model in a series of healthcare organisations within a single public entity, Osakidetza. In fact, there is little data in the literature on the implementation of the EFQM model in healthcare settings and results have not been conclusive [19]. Further, while other authors have reached similar conclusions, none of the previously reported studies have covered as broad a period of time or as many organisations (in terms of diversity and number) as the present one. For instance, the positive effect of employees’ satisfaction on organisational improvement has been demonstrated by some authors [23], while it is also recognised that it is difficult to quantify and assess its impact on outcome indicators [24] or on the general performance of the organisation.

This study has various limitations. On the one hand, although external recognition as a criterion for assessing progress towards establishment of the EFQM model has been used by other authors [16, 18], this approach may be questionable as an organisation could decide not to apply for the awards, despite having adopted the model. In the case of Osakidetza, this circumstance is improbable given that the corporate strategy of the health service (as set out in the Strategic Plans [12–14]) explicitly encourages member organisations to obtain these types of external recognition.

On the other hand, a wide range of factors may affect employee satisfaction, and hence attributing any observed effect to the deployment of the EFQM model is questionable. Nevertheless, the pattern of differences observed seems to support the hypothesis of the management model having an impact on the results, namely because the areas in which significant differences were found corresponded to those in which there is the greatest margin for independent management at the level of the separate healthcare organisations (training, internal communication, leadership style, etc.), while those in which differences were not significant were related to common corporative elements, more homogeneous across all the organisations in Osakidetza (such as health and safety at work and pay).

Another factor that could have affected the differences between organisations to some extent is that all the healthcare organisations within Osakidetza have been exposed to the EFQM model and that all of them have adopted it to a greater or lesser extent. Despite this, we found significant differences for organisations judged to have made the most progress, by means of a greater degree of implementation of the model and seeking (and being granted) external recognition, which were rated more highly in terms of employee satisfaction than those that have made less progress.

Finally, in the comparison between organisations granted Gold Q Awards vs. those without any Q Awards, the effect of the model was seen to be maintained over time, and some positive effects even appeared in the last assessment period, namely in relation to satisfaction with technical and material resources. In the comparisons between organisations with any Q Awards versus those with no Q Awards, however, there seems to be a reduction in the effect with time. Although the results of the surveys are not fully comparable between assessment cycles, such an effect could be attributable to an organisation making less effort once it had obtained a Silver Q award. In fact, a qualitative analysis associated with this study also indicated that organisations to some extent overstrained to obtain the Silver Q award and after that tended to relax, while the Gold Q Award takes longer to be awarded and is granted in a more mature setting and, therefore, the conditions are more stable. This apparent reduction in the differences (to be interpreted with caution given the limitations in terms of comparability between the surveys carried out in the different periods), could also be due to a narrowing of differences over time between organisations with a lower level of external recognition (Silver Q award) and without external award, as they have all been exposed to quality promotion policies from the corporate level.

Conclusions

The progress made by Osakidetza healthcare organisations in the implementation of the EFQM model is associated with a greater level of employee satisfaction in fields that can be managed independently by each organisation, while no significant differences were observed in those aspects that are more homogeneously and centrally managed across the regional health service and in which each organisation has less scope for self-management.

This study also shows that the greater the progress in the implementation of the model (as in the case of the organisations with Gold Q Awards compared to those with Silver Q Awards), the greater the differences in terms of staff satisfaction with the less advanced organisations (those without any external recognition) and in a broader range of dimensions.