Development of nursing quality care process metrics and indicators for intellectual disability services: a literature review and modified Delphi consensus study

Abstract

Background

Nursing process quality care metrics and indicators are quantifiable measures of the nursing care delivered to clients. They can be used to identify and support nurses’ contribution to high quality, safe, client care and are lacking in specialist intellectual disability nursing. In a national Nursing Quality Care-Metrics project for Irish intellectual disability services, a set of nursing quality care process metrics and associated indicators were established for intellectual disability services.

Methods

A two-stage design approach was undertaken; a broad scoping review of the literature and a modified Delphi consensus process. The Delphi included a four round e-Delphi survey and a consensus meeting. Four hundred one intellectual disability nurses working in Ireland were recruited for the surveys and 20 stakeholders attended the consensus meeting.

Results

From the review, 20 existing and 16 potential intellectual disability nursing metrics were identified for nurses to prioritise in the e-surveys. After the four survey rounds, 12 intellectual disability nursing metrics and 84 associated indicators were identified. Following the consensus meeting, these were reduced to 12 metrics and 79 indicators.

Conclusions

This first set of intellectual disability nursing process metrics and associated indicators has been identified for implementation in practice. These metrics while developed in Ireland have international relevance and their application and appropriateness in practice needs to be evaluated.

Background

Quality care and patient safety dominates research literature on healthcare [1] and is dependent on having dependable data and a structure to support analysis of such data. Poor nursing care and variations in nursing practices affect quality of care [2, 3] and generating indicators sensitive to nursing is difficult due to the invisible nature of nursing care [4, 5]. However, nursing metrics create the possibility to measure essential nursing care processes, recognise when care is dropping below the obligatory standard and enable staff to make enhancements to decrease the risk to patients\clients and their families [6]. This is relevant to people with intellectual disability (ID) as they are a vulnerable group as they experience; high levels of morbidity, hospitalisation, premature mortality [7] and health disparities [8]. While people with ID are living longer they are more likely to experience comorbidities [9, 10], with greater prevalence of coronary heart disease, cardiovascular disease, obesity and diabetes [11, 12].

Identifying measures of nursing care processes in specialist ID nursing (ID nursing) incorporate all actions related to care provision, from interpersonal relationships of care to technical delivery. This can be difficult to capture given the sometimes indiscernible nature of nursing and the fact that Ireland and the United Kingdom (UK) are the only countries with specific undergraduate programmes and registration for ID nurses. Additionally, there is limited evidence specific to ID nursing care metrics as compared to other branches of nursing. This may be compounded by the fact that not all nursing activities such as communication and compassion can be easily measured.

The concept of ID has evolved over the years to one where ID is now seen as a natural part of the human condition and defined by the environmental and social contexts in which the person lives [13]. ID nursing in Ireland originated in 1959 when it developed as a branch of nursing and initially was known as “Mental Sub-Normality Nursing”, and later changed to “Mental Handicap Nursing” before its present form “Intellectual Disability Nursing” [14]. In its early days, ID nursing services were delivered within institutional care settings employing an illness-oriented care model [14]. However, since the 1980s, a social model of care has been emphasised and ID services transferred from institutional care to community settings, focusing ID nursing on a biopsychosocial educational model [15]. Roles of ID nurses are multifaceted involving the delivery of person centred and holistic care. This includes the provision of health education, promotion and management, respecting individuals’ identity, providing multisensorial and alternative therapies, and supporting personal and social development [14]. ID nurses also support independent living, provide community based support and increase social awareness of people with ID and their needs [14, 16]. Within Ireland’s population (4.5 million), there are 28,388 people registered on the 2017 National Intellectual Disability Database (NIDD) receiving, or identified as needing, services for a diverse and complex range of health and social needs [17]. However the true total population of people with an ID in Ireland is undetermined [18] and internationally a 1 to 2% rate of the total population is generally accepted.

To value and make visible the role of ID nurses and improve client outcomes, there is a need to identify and measure current activities related to care processes. In Ireland a national research project was conducted to identify and develop a set of evidence-based nursing care process metrics and associated indicators across seven practice areas (acute care, mental health, public health nursing, children, midwifery, older person, and intellectual disability services). Each individual practice area group consisted of an academic team, a Nursing and Midwifery Practice Development (NMPD) Director (chair), the National Quality Care-Metrics lead, NMPD project officers, and key stakeholders including Directors of Nursing, clinical practitioners and service users. Within the project, the definition of quality care process metrics and indicators by Foulkes (2011) was adopted [19] (Table 1). Quality measures are frequently classified into three types; structure, process and outcomes [20]. Structure measures reflect factors such as the availability of staff and facilities. Process measures consider whether care interventions adhered to best practice guidance, and outcome measures consider the changes as a result of the care delivered. Outcome measures are frequently used in healthcare; however measuring process can be a more efficient measure of quality [21] as they are amenable to direct action to enable continuous improvement by care providers. While generally, metrics and indicators are a statement evaluated against an agreed standard, the fact the registration of ID nurses does not occur beyond Ireland and the UK fulfilling the aspect of evaluating against an agreed standard can be difficult. This emphasises the necessity for ID nursing to create it evidence base from and within practice, and this paper reports on the development of ID nursing process metrics and indicators identified through a scoping literature review and modified Delphi consensus process.

Table 1 Quality care process metrics and indicators [19]

Methods

This study aimed to identify a set of process metrics and indicators for ID nursing. The study process (Fig. 1) consisted of two stages; a broad scoping review of the literature followed by a modified Delphi consensus process.

Fig. 1

Study process

Stage 1-broad scoping literature review

A broad scoping review of the literature using a systematic process using Moher et al’s [22] proven and robust processes and utilising Covidence online platform [23]. Scoping reviews are useful in areas where there may be a lack of empirical research evidence and conceptual ambiguity [24]. The search was not limited by study design but widened to include all types of sources, including grey literature. The literature search was originally conducted as a national collaboration across all seven practice areas (older persons services, intellectual disability nursing, mental health nursing, acute nursing, public health nursing, children’s nursing and midwifery). The aim of the review was to identify quality care process metrics and relevant indicators for nursing/midwifery, and to identify the current evidence base (last 10 years - 1st of January 2007 and 1st of January 2017) across eight databases (CINAHL, Psyinfo, EMBASE, ASSIA, Pubmed, CDRS, DARE, CENTRAL) and grey literature utilising the search terms: nurs*:ab,ti OR midwi*:ab,ti AND (‘minimum data set’:ab,ti OR indicator*:ab,ti OR metric*:ab,ti OR ‘quality measure*’:ab,ti) AND [english]/lim AND [2007–2017]/py.

The database and grey literature search identified 15,304 results (Fig. 2), all titles and abstracts and full-text were reviewed independently by two reviewers and 112 publications met the inclusion criteria. No articles reported ID nursing process metrics and indicators. However, 14 articles were selected from the generic nursing literature to be used to extract ID nursing related process metrics and indicators. In addition, grey literature provided by ID services nationally were searched together with publications identified from hand searches. This resulted in a further 21 documents. In total, 35 publications were identified (Fig. 2) for review and data extraction and where appropriate, research studies were independently critically appraised using the Crowe Critical Appraisal Tool - CCAT [25].

Fig. 2

PRISMA Flow Diagram [24]

Stage 2: Delphi consensus process

A Delphi consensus process includes gathering expert opinions on an area to build consensus to reach an understanding of a problem with possible solutions using a series of questionnaires and group communication techniques [26]. A classical Delphi technique consists of the following features: anonymity, controlled feedback, iteration, statistical group response and stability [27]. In this study, a modified Delphi was used where the Delphi-consensus technique included a four round electronic survey of nurses and was modified by the addition of a face-to-face consensus meeting. Process metrics identified from the literature review were prepared for rounds one and two of the e-survey. Participants could vote on which metrics they considered important. In rounds three and four participants could vote on which indictors they considered important. This process concluded with a final face-to-face consensus meeting with key stakeholders to agree the final set of quality care process metrics and indicators for ID nursing. The consensus meeting of key stakeholders (n = 20) consideration grade and geographical representation and consisted of NMPD project officers, clinical practitioners (clinical nurse managers, clinical nurse specialists, staff nurses), practice development co-ordinators and service users representation.

For the Delphi surveys, a purposive sample was followed for recruitment of participants (nurses working in ID Services across Ireland). Therefore, ID nurses working in ID services nationally were recruited via national adverts and with the support of NMPD officers who distributed an information package to ID nurses (n = 4393). The sample size was calculated as 353 considering a representative of 4393 ID nurses, which would give a confidence level of 95% and confidence interval of ±5. Prospective participants had a chance to email the research team directly to obtain more information and clarify any issues before they made their decision to participate. Participants were emailed an invitation, further information with instructions related to the first e-Delphi survey instrument and the online survey link.

The end phase of the e-Delphi process comprised of a face-to-face consensus meeting with key stakeholders to review and discuss the findings from the e-Delphi surveys with the aim of building consensus on the relevance and wording of the final set of ID metrics and their associated indicators. The key stakeholders (n = 20) represented clinical expertise, nursing leadership and service provision across Ireland. The process was informed by a review of the literature carried out prior to the meeting, which identified ways to manage the consensus meeting [28,29,30,31]. To assist participants in voting for metrics and indicators, an evaluation tool was developed by the researchers following another review of the literature and expert review process.

Data collection

The e-Delphi surveys collected information related to demographics (work place, grade, year of experience) and included the list of metrics/indicators. Questions related to metrics/indicators asking ID nurses to rate each metric/indicator on a 9-item Likert scale (1 to 3 “not important”, 4 to 6 “important”, and 7 to 9 “very important”). Data collection occurred between June and October 2017 through online e-Delphi survey rounds with each round open for 21 days. All who expressed an interest to participate were provided with a weekly e-mail as a reminder/update. Experts participating in the face-to-face consensus meeting were asked to vote “yes/no” for each metric and indicator using a paper-based voting system.

Data analysis

The findings of the scoping review informed the metrics in the survey rounds and the modified Delphi consensus technique was used to finalise the suite of ID nursing metrics and indicators through four rounds and a face-to-face consensus process. This enabled nurses working in ID services nationally to select metrics and indicators they deemed most relevant to their professional clinical practice. Data gained from the e-Delphi surveys was analysed using simple descriptive statistics to summarise data. Responses to each round were collated, analysed, and redistributed to participants for further comment in successive rounds and re-rated by participants, with knowledge of the group’s results from the previous round. Consensus on inclusion of a metric/indicator was agreed prior to commencement. It was determined that where 70% or more of nurses scored the metric/indicator as 7 to 9 and less than 15% scored the metric/indicator as 1 to 3 this metric/indicator was deemed included [32]. A similar approach was followed for the face-to-face meeting where metrics and indicators voted yes by more than 70% of the experts were included.

Ethical issues

Ethical approval was obtained from the researchers universities Research Ethics Committee (No: 2016_12_12_EHS). Participants gave consent to participate by clicking on an ‘I consent to participate in this study’ electronic link prior to being permitted to access the e-Delphi rounds. The online survey software system which maintained the survey data was protected behind a firewall. Only the research team had access to the data through use of a user identifier and password.

Results

Following the literature review process, 36 intellectual disability nursing metrics were identified. Sixteen new metrics were identified from the generic nursing literature and 20 current metrics from the 2015 practice areas Standard Operating Procedure for Nursing and Midwifery Quality Care-Metrics [33] (Fig. 3). Four hundred one nurses working in ID were recruited and an overall response rate of 50% was achieved for all rounds. Most of the nurses participating in all rounds were at staff nurse level and their average years of experience was over 20 (Table 2).

Fig. 3

Existing and new ID metrics for Round 1 of the e-Delphi survey

Table 2 Profile of the nurses participating in Delphi rounds

A total of 233 ID nurses participated in round 1, resulting a response rate of 58.1%. Thirty-five of the 36 metrics were rated between 7 and 9 by 70% or more of the participants and rated between 1 and 3 by less than 15% of participants and hence included in round 2. In addition, participants had the chance to add suggestions for additional areas of practice (round 1) to be considered as possible metrics in round 2. 208 qualitative comments were analysed and categorised under 23 common themes, and mapped under existing or new metrics. These comments led to nine new areas of practice for metrics (environmental restraints, meaningful and purposeful activities, sexuality and relationship, family centred care, advocacy, transition planning, life stages and social inclusion, nutritional health, long-term conditions, managing personal finances) creating 35 existing and nine new metrics for round 2.

The 233 nurses participated in the round 1 were sent invitation for the round 2. A total of 218 nurses completed round 2 with a response rate of 93.5%. In round 2, 43 of the 44 metrics met the inclusion criteria with only sexuality and relationship (68.37%) scoring as “very important” by less than 70%. Following round 2, a working group meeting was convened between intellectual disability nursing members, academic, nurse leaders and clinical experts and the 43 metrics were re-formulated into 12 metrics. Seven of these new metrics had little or no supporting literature and required indicator development by the working group members before distribution in round 3 of the e-Delphi survey.

The set of 12 metrics with their 95 respective indicators were sent to participants in round 3. All ID nurses who initially expressed their willingness to participate in the study before the first round were able to participate in round 3. Since there were 6 nurses dropped out during the first two rounds, 395 nurses were sent invitation for the round 3, thus round 3 had 234 participants and a response rate of 59.2%. Ninety-three of the 95 indicators associated with the 12 metrics reached the 70% threshold and these were included. The two indicators which did not reach the 70% threshold were excluded and related to environment and safeguarding metrics. Also within round 3, nurses were given the opportunity to add comments and suggest other indicators. 88 qualitative comments were provided and following analysis the indicators were refined, merged, or separated where needed, resulting in 84 indicators going forwarded to round 4. The 234 nurses participated in the round 3 received the invitation link for round 4 which was conducted with participation of 177 nurses, thus had a response rate of 75.6%. All indicators distributed in round 4 of the e-Delphi Survey were rated above 70%. This resulted in 12 intellectual disability metrics and 84 associated indicators prior to consensus meeting.

To complete the process a face-to-face final consensus meeting was held in November 2017 with participation of key stakeholders. Each metric and respective indicator was discussed and voted on, with guidance provided to clarify ground rules in the consensus meeting. Some editing to wording was agreed and through discussion some indicators were collapsed prior to voting. This resulted in all 12 metrics and 79 of the 84 associated indicators reaching the 70% threshold to be included in the new set of intellectual disability nursing process quality care metrics and indicators (Table 3).

Table 3 Final set of Intellectual Disability Nursing Process Metrics and Indicators

Discussion

Although the grey literature identified a pre-existing set of ID nursing metrics developed in Ireland, in the literature review there was a lack of developed metrics in which all the traits of a metric (care process, standard and measurement) were present. In addition, the metrics that were identified from the literature review were developed from general nursing evidence rather than ID nursing. This can be partly explained by the fact that specific ID nursing degree programmes are only offered in Ireland and the UK and generally fall within mental health nursing in other countries. Consequently, metrics and indicators related to ID nursing are limited in the international literature and the process metrics that do exist are often derived from generic nursing processes. While such evidence may have clinical relevance and applicability there is difficulty in using generic nursing process as the specific needs of people with ID are often absent [34].

The development of nursing metrics from and within the specialist area of ID nursing is essential as people with ID are living longer [35]. They experience earlier mortality [36], and are more commonly diagnosed with comorbidities than people without ID [37, 38]. This disparity is acknowledged as being due to inequity as opposed to having a solely physiological basis [39]. Persons with ID admitted to acute, general settings may have their needs unrecognised and thus unmet. This is partly explained as general nurses who have no ID nursing experience report feeling underprepared, inadequate and struggle with communication barriers [40]. This is not surprising given that most nurses may not have cared for a person with ID during their undergraduate training or studied any theoretical or applied content regarding best practice when nursing a person with ID [34]. The development and translation of ID nursing metrics into other nursing care environments can assist in dealing with the health inequalities experienced by people with ID and facilitate health equity [41]. However, as many health care organisations at present are not equipped or do not have trained staff to work with people with ID [42,43,44] there is a need to address matters relating to service provision such as; metrics, continuous professional development and education.

Given the lack of research evidence underpinning ID nursing metrics and indicators, different forms of evidence including practice evidence and service user evidence were needed in guiding the development of the metrics and indicators within this project. The incorporation of the grey literature was essential in identifying aspects of nursing care process and areas of practice that warranted consideration based on views of practitioners, organisational policies and Irish and international regulatory authorities. Within this project, grey literature was sourced from ID services nationally and supplemented by hand searching to ensure a comprehensive search strategy and this literature provided valuable background to existing and potentially new metrics and should support implementation of the metrics given the practice evidence that was utilised and the involvement of practitioners within the project.

Given the absence of research evidence, ID nurses need to generate a robust accessible research base stressing the needs of people with ID and effective care strategies to meet those needs [45]. Such research should attempt to capture and illuminate the very heart and kernel of care in order to develop an evidence-based quality service [14]. This is further emphasised by the development of ID nursing over the years where, historically ID nurses worked in large facilities but with philosophical and funding changes many large institutions have closed and most people with ID are cared for by nurses working in community settings of various types such as; special education settings, small facilities, residential homes, or personal homes [46]. In addition, known that there is an ageing population and the health inequalities people with ID face across their lifespan there is a need for specialised nursing care [38, 47]. However, while ID nurses need to make their contributions visible [14] they often struggle to identify and explain what distinguishes them and sets them apart from other allied health professionals and the value nursing brings to the health care team [48, 49]. Thereby the production and use of ID nursing metrics will contribute and support quality care provision in a measurable manner.

ID nursing necessitates a wide range of specified skills so as to meet the varied health, advocacy, societal, behavioural needs of persons with ID. It is a complex role in that it is not a uniquely technical nursing role, but a unique relational role due to the restrictions in intellectual function and adaptive behaviour [50, 51]. Nurses who have developed specific skills to work successfully with people with ID are crucial in safeguarding the delivery of high quality healthcare in a diversity of settings with improved outcomes for health and well-being. Recent reports have emphasised unsatisfactorily high numbers of preventable deaths of people with ID that can be accredited to limitations in communication, diagnostic overshadowing, misdiagnosis, poor standards of care and the denial of rights [52,53,54,55,56]. There is a need for nursing metrics developed from and within the specialist area of ID nursing to support quality care provision as they become embedded in healthcare governance and management systems [57].

Within the study it was important to be aware of the quality of the metrics and indicators. To assist this process the authors used a framework to enable the consensus group to evaluate metrics and indicators against the four key attributes of; process focused, important, operational and feasible. This ensured that the metrics and indicators were considered as process focused and important to ID nursing practice and practitioners. Within the tool the third domain (operational) raised some concerns as not all of the metrics and indicators had reference standards and/or research evidence supporting them. However, they had a strong practice evidence base and support of the consensus group. This in turn impacted on the final appraisal trait of feasibility as the lack of indicators within the literature have an impact on implementation. However, as the indicators were developed within the Delphi process and by the work stream members’ one could argue that they will be feasible for practice implementation. Prior to implementation it is strongly recommended that metrics and indicators are piloted before complete usage to avoid unintentional and adverse consequences [58], thus pilot testing of these indicators is recommended and planned.

This study attempted to address this deficit by engaging key stakeholders nationally and this has resulted in a set of ID nursing process care metrics for Ireland. Given Ireland’s unique position in having ID nurse education and registration, the metrics developed are relevant, applicable and measurable for ID practice and can be adopted by other nursing care agencies across the world. This global applicability is important given the international quality and safety agenda and the necessity to support the rights of vulnerable people such as those with ID. In many international healthcare organisations the measurement of care is frequently suggested as a means to improve care outcomes and experiences [59]. In Ireland, Health Information and Quality Authority’s Regulations and Standards for Residential Services for Adults and Children with Disabilities (2013) [59] are set out under eight themes. The first four relate to the quality and safety of services provided and the remaining four relate to the workforce and their capacity and capability to provide services of appropriate quality and safety. The development of the set of ID process metrics in this study will assist in supporting quality and safety in ID services and support improved care experiences and outcomes. Developing a set of ID metrics will assist in having a clear standard and measurement of care delivery which is important given that variation in care delivery is well documented in both the literature and in reports and investigations of healthcare regulators. Quality of care is a complex concept and although it is central to modern healthcare policy [60, 61], it can be challenging to measure.

Limitations

While this study produced a set of ID nursing metrics it also highlighted that the evidence underpinning the metrics is practice based rather than research evidence based. This may lead to applicability to practice but an evaluation of their implementation is needed to consider their use in Ireland and translation outside of Ireland. While the study aimed to represent nurses working in ID services nationally the only 4.9% of the total ID nurse population in Ireland participated across the Delphi phase. This reflects both non access to work emails in ID services and non-participation.

Conclusion

The aim of the Nursing Quality Care Metrics project was to identify a final set of nursing quality care process metrics and associated indicators for ID to enable the provision of evidence of nursing’s contribution to safe, high quality patient care. Twelve nursing care process metrics and 79 indicators for ID were identified and having ID nursing metrics data available for use by front line staff and management levels creates the potential to measure fundamental nursing care processes and enable staff to make enhancements to minimise the risk to patients and their families. Nurses working in ID in Ireland are educated to degree level and are therefore well positioned to influence improvements and positive outcomes for clients in their care. However, the challenge to date has been the identification of the specific measures that are sensitive to the influence of nurses and having consensus that they are appropriate and relevant. The need for ID nursing specific metrics and indicators is evident as they utilise more specific teaching, social and emotional support focused approaches rather than a traditional medical model focused approach to care. Therefore, this project offers a unique contribution, in developing a set of national metrics and indicators for ID nursing through a process of involvement with key stakeholders and practitioners. The resulting metrics and indicators are pertinent to the explicit needs of people with ID and provide an opportunity to nurses internationally who support people with ID (either through general, mental health or developmental disability nursing services) to define and quantity the quality of care provided.