Skip to main content
Download PDF

Co-design of a patient experience survey for arthritis central intake: an example of meaningful patient engagement in healthcare design

BMC Health Services Research volume 19, Article number: 355 (2019) Cite this article

Abstract

Background

To describe the process of patient engagement to co-design a patient experience survey for people with arthritis referred to central intake.

Methods

We used a participatory design to engage with patients to co-design a patient experience survey that comprised three connected phases: 1) Identifying the needs of patients with arthritis, 2) Developing a set of key performance indicators, and 3) Determining the survey items for the patient experience survey.

Results

Patient recommendations for high quality healthcare care means support to manage arthritis, to live a meaningful life by providing the right knowledge, professional support, and professional relationship. The concept of integrated care was a core requirement from the patients’ perspective for the delivery of high quality arthritis care. Patients experience with care was ranked in the top 10 of 28 Key Performance Indicators for the evaluation of central intake, with 95% of stakeholders rating it as 9/10 for importance. A stakeholder team, including Patient and Community Engagement Researchers (PaCER), mapped and rated 41 survey items from four validated surveys. The final patient experience survey had 23 items.

Conclusion

The process of patient engagement to co-design a patient experience survey, for people with arthritis, identified aspects of care that had not been previously recognized. The linear organization of frameworks used to report patient engagement in research does not always capture the complexity of reality. Additional resources of cost, time and expertise for patient engagement in co-design activity are recognized and should be included, where possible, to ensure high quality data is captured.

Peer Review reports

Background

Globally, osteoarthritis (OA) causes 12.8 million years lost to disability (YLDs) and is ranked 13th, in the top causes of top causes of global YLDs in 2013, ranked higher than cancer (6.7 million YLDs) and hypertensive heart disease (11.9 million YLD) [1]. Osteoarthritis is the most common form of arthritis, and there is no cure. OA affects approximately 12.5% of people in Canada and rheumatoid arthritis (RA) 1%, causing pain, disability, and a significant human burden [2]. Unlike OA, early detection and referral to specialist services for people with RA can significantly affect outcomes [3] but established wait time benchmarks are rarely achieved [4]. A survey of 4565 Canadians aged 20 or older, found that the average time between onset of symptoms and OA diagnosis was 7.7 years [5]. During this period, patients can experience pain, depression, anxiety and increasing disability [6]. Understanding the patients experience as they wait for specialist services can help shape the delivery of health care to provide care that is commensurate their needs. For example, in a qualitative study, using a patient-to-patient method where patient researchers with OA engaged others with OA to co-develop a model for the referral to specialist services, the most salient, overarching theme was ‘supporting us managing a meaningful life with arthritis’ [7].

When patients are engaged as researchers they bring their health experiences and as partners in healthcare system design can improve the quality of care, and enable healthcare providers and administrators to meet the needs of a growing number of patients living with chronic health conditions [8, 9]. In the UK, patient and public involvement in health-care planning, service development, healthcare policy and research have gained growing importance over the past two decades [10]. In Canada, several initiatives have embraced patient-centered care approaches but change has been slow [11]. However, in Alberta, a number of initiatives have firmly position patients in the decision-making process when shaping healthcare redesign [12, 13].

One example of a project that has consistently engaged patients in decision-making relates to the provision of arthritis care. Funded by Alberta Innovates (AI), Alberta Health Services (AHS), the Arthritis Society, and the Canadian Institute for Health Research, “Optimizing Centralized Intake to Improve Arthritis Care”, has a specific objective to identify the optimal referral management and triaging strategy across varying central intake processes in Alberta. Central intake (CI) systems are single point-of-entry for referrals, usually from primary care to a health care service, with the aim to streamline services, reduce bottlenecks and reduce wait times for specialist assessment [14]. Models such as these have the potential to improve timeliness and patient-centeredness of elective surgical procedures [15] and have been proposed as one of the approaches to tackle waiting time for patients with suspected RA [16].

The starting point for this initiative was to identify the needs of patients with osteoarthritis, rheumatoid arthritis, and First Nations Metis and Inuit patients with rheumatoid arthritis (Phase1 – see Fig. 1). The Patient and Community Engagement Research (PaCER) group were asked to lead this phase of the research at the start of the research project [17]. The PaCER method is a peer-to-peer inductive research approach designed to create a robust collective patient voice and maximize patient engagement throughout the research process [7, 12, 13, 18]. Using the same process, patients with RA provided their perspectives on the challenges they face in accessing and navigating the health care system, and what they viewed as key components of an effective system that would be responsive to their needs [19]. The First Nations Metis and Inuit patients with rheumatoid arthritis provided several recommendations including the importance of clinic staff who understand their needs; the importance of developing wellness strategies specifically for this cohort of patients [20]. A total of 25 recommendations from the PaCER studies were identified.

Fig. 1
figure 1

Summary of study outcomes from Phases 1 & 2

Full size image

These recommendations were then used to inform, in addition to other stakeholders, the development of a set of 28 Key Performance Indicators (KPIs) to evaluate centralized intake for arthritis care and the detailed process is reported elsewhere [21]. Stakeholder meetings including healthcare providers, managers, researchers, and patients with OA or RA provided input on potential KPIs and aligned with five of six quality dimensions from the Alberta Quality Health Matrix of appropriateness, accessibility, acceptability, efficiency, and effectiveness [22]. At this stage of the research cycle, patients and PaCER researchers were collaborating and contributing through participation and discussion to the research activity [17]. Literature reviews were conducted to ensure KPIs were based on best practices and harmonized with the six core performance measures of the National Arthritis Alliance of Canada’s Measurement Framework [23] and finalized using the RAND developed ExpertLens methodology [24]. The final phase involved a panel (n = 28) of patients, the patient engagement researchers from PaCER, care providers and system administrators, who prioritized KPIs using a Multi-Criteria Decision Analysis process [21]. Importantly, KPIs reflecting patients experience were ranked in the top 10 by patients, PaCER researchers and other stakeholders on the panel, with 95% rating it as 9/10 for importance [21]. (Phase 2 – see Fig. 1). These KPIs informed the development and evaluation of an optimal centralized intake system for arthritis care, as well as providing valuable indicators of quality of care for OA and RA patients. The findings also created the impetus to develop a patient experience survey as the patient experience was central to the evaluation of the central intake model (Phase 3).

Capturing the extent of patient involvement in the research process has uncovered some challenges, such as inadequate reporting of patient involvement, and a lack of valid and reliable tools to capture the impact of patient and public engagement [10]. The Guidance for Reporting Involvement of Patients and Public (GRIPP) checklist [25, 26] addresses this gap but has been criticized for assessing the impact of patient engagement in research, rather than the quality of the reporting [27]. Recognizing that there was limited understanding of how patients were being engaged in rheumatology research, a recent review sought to develop a framework to advance reporting of patient engagement [17]. They proposed a new framework based on a scoping review and analysis of 30 publications related to patient engagement. Their model, the Patient Engagement in Research Description (PED) framework, identifies three over-arching categories of who, how, and when, that should reported in relation to patient engagement in the research process. They use the term patient research partner (PRP) to describe patients, their family members, and informal caregivers who participate in health research initiatives. The ‘who’ should describe the affiliation and research-relevant health characteristics of the PRP. How they are engaged would describes the process of engagement and includes initiation of engagement, the method of contribution and level of engagement e.g. informed, consulted, collaborated and led. Finally, the category ‘when’ captures the different time points along the trajectory or life cycle of the research project. In describing the process of co-design in our study we also describe the who, how and when of patient engagement.

The specific aim of this paper is to describe our process of co-designing a patient experience survey for people with arthritis referred to a central intake clinic. For the future, the intention is to embed the patient experience survey in the routine evaluation of service redesign for centralized intake to improve arthritis care.

Methods

Study design

The design of the study is essentially one that reflects a constructivist approach where knowledge is constructed through collective approaches [28]. Engaging patients to capture their experiences in the design and re-design of healthcare services have been a central concept of improvement initiatives and includes core principles of equity, understanding experiences, and improving services [29]. In healthcare improvement co-design has been identified as an approach to participatory design, involving all stakeholders in a process to ensure the results meet their needs and is usable [30]. We have used the term co-design to describe the process of for developing the patient experience survey, collaboratively with patient researchers.

The process of developing the patient experience survey comprised of three connected phases: 1) Identifying the needs of patients with osteoarthritis and rheumatoid arthritis, 2) developing a set of key performance indicators for the evaluation of CI systems for arthritis, and 3) the co-design process for developing items for a patient experience survey for arthritis care. Phases 1 and 2 have been briefly described in the background as they were instrumental in the development of the survey and a summary of key outcomes from these two phases are provided in Fig. 1. For this study the patient experience survey was a core component of the baseline evaluation to be conducted in the future.

Institutional ethical review and approval were obtained for each of the phases (University of Calgary Conjoint Health Research Ethics Board ID: REB13–0822). In describing our process, we have attempted to make transparent the key elements of who, how and when, from the PED framework [17].

Setting of the study

The study was carried out in an urban setting in Western Canada, within a single provincial healthcare organization. Physicians are paid on a fee-for-service (FFS) model. The central intake referral system for arthritis care in the province serves over 4.3 million residents.

Participants

To identify items that would ultimately be included in the patient experience survey a small working group comprising of PaCER researchers (JM, ST) and academic researchers (n = 4) were convened from the broader research team. It was important to have patient stakeholders who could bring both their own experiences of living with arthritis but were also patient researchers and a part of the Patient and Community Engagement Research group [12]. The two PaCER researchers were co-investigators on the research team and had been involved in several studies in this program of research around the delivery of arthritis care. Both of them had previously conducted research with patients to ascertain what was important for people with OA seeking support for their symptoms [7]. The four academic researchers consisted of the principal investigator for the research who was a senior health services researcher and health economist. The two co-investigators responsible for the evaluation phase of the study had experience of mixed methods, qualitative research and health care evaluation, and the research coordinator was experienced in managing complex research studies. All the academic researchers had extensive experience of patient engagement in health research.

Procedure and analysis

The working group participated in 10 bi-monthly one-hour telephone conference calls (Nov 2015 – April 2016) with follow up calls and email contact between meetings, as required. The purpose of these meetings was to develop the patient experience survey. To do this the meetings were very task focused with PaCER researchers contributing by participating all aspects of this process and completing research-related tasks i.e. rating survey items. The following two sections describe in detail the identification and selection of the patient survey items (see Fig. 2).

Fig. 2
figure 2

Decision making for item selection in the development of the patient experience survey

Full size image

The first task was to identify which recommendations from the PaCER reports should be reflected in the patient experience survey. The working group were provided with the 25 recommendations from the three patient cohort reports i.e. OA (11 recommendations), RA (7 recommendations), and First Nations, Metis and Inuit RA cohort (7 recommendations) on a spreadsheet. They were then provided with definitions of each of the patient-identified themes created from the OA and RA PaCER recommendations (right knowledge, right professional support, and right professional relationships) [7, 19]. Following discussion with the working group an additional theme of ‘timeliness of care and communication’ was added to reflect the wider scope of the central intake process. The working group was asked to sort recommendations into one of the four themes. They were also asked to rate each recommendation as “yes”, “no”, or “maybe” for inclusion. These responses were first documented on their individual spreadsheet and sent to the research coordinator who collated the results onto a final group spreadsheet. The group then compared individual ratings, discussed items and came to a consensus for which ones to include/exclude.

The next stage was creating a pool of survey items from validated instruments as a starting point for generating a patient experience survey item pool. These included the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) patient survey questions [31], the Patient Experience Survey instrument for the Canadian Institute for Health Information (CIHI) Primary Health Care Survey project [32], rheumatoid arthritis outpatient satisfaction survey [33], and the UK’s National Health Service Outpatient Questionnaire [34]. Forty-one survey items were identified as relevant to the 13 patient recommendations and screened to ensure none replicated PaCER recommendations and were not outside the scope of the project (JP/DM). This pool of items was then shared with the working group for a mapping exercise to determine which items would be included and identify gaps. Again, a rating of ‘yes’, ‘no’ and ‘maybe’ was used. Similarly, the task was completed individually, and then as a group, by comparing ratings and discussing items. Similar to the process for identifying which recommendations should be reflected in the patient experience survey, the responses were first documented on their individual spreadsheet and sent to the research coordinator who collated the results onto a final group spreadsheet. The group then compared individual ratings, discussed items and came to a consensus for which ones to include/exclude.

Finally, the updated list of questions along with a standardized scoring method was compiled into a draft survey and presented to arthritis patients at one clinic (n = 4) who provided feedback on the contents of the survey. They were asked to comment on the clarity of questions, length of the questionnaire, and suggestions for change.

Results

The first task of identifying which of the 25 recommendations from the PaCER reports should be included in the survey resulted in 3 OA and 2 RA recommendations being excluded as being out of scope e.g. recommendation for the development of clinical guidelines, and recommendations for further research. There was minimal discrepancy between the raters in their evaluations with the exception of the 7 recommendations from the First Nations, Metis and Inuit (FNMI) cohort. Whilst the patient-researchers recommended inclusion, the academic researchers had excluded them, as several of these were particularly unique to the cohort and not common for both RA and OA cohorts. For example, incorporating traditional wellness strategies was particularly important [35] and yet the need for peer support, identified by other cohorts as important, was not highlighted as a need [20]. Given the challenges to include recommendations that were not common across the other two cohorts it was decided that a separate patient experience survey for this group capturing specific requirements, such as the delivery of services based on indigenous approaches to health and healing, will be developed at a future time. It was also important to ensure that survey items would reflect the recommendations from the Health Quality Council of Alberta, for improved patient experience and outcomes: accurate information between providers and patients; maintaining continuity between providers; understanding the individual responsibilities of providers; and ensuring patients are aware of who to contact for assistance [36]. A total of 13 recommendations were made for quality arthritis care, from the PaCER reports for patients with osteoarthritis and rheumatoid arthritis.

From the pool of survey items from validated instruments the rating process eliminated a total of 18 items from the patient experience survey item pool for the following reasons: replication of PaCER recommendations (5), outside the scope of the project (10), included activities patients would not experience e.g. laboratory results (3). Feedback from the four arthritis patients at one clinic led to minimal changes and the survey was considered complete (see Additional file 1). The final patient experience questionnaire had a total of 23 items.

Discussion

Engaging patients or users, to capture their experiences in the design and re-design of healthcare services has been a central concept of improvement initiatives and includes core principles of equity, understanding experiences, and improving services [29]. In healthcare improvement co-design has been identified as an approach to participatory design, involving all stakeholders in a process to ensure the results meet their needs and is usable [30]. A variety of activities have been identified as part of this process, such as gathering experiences, bringing patients and healthcare staff together, and using triggers to identify shared priorities for improvement. In this paper, we use the term ‘co-design’ to describe our process of patient engagement to co-design a patient experience survey for people with arthritis referred to a central intake clinic. From this process, we identify several aspects that bring a unique perspective and warrant discussion. Firstly, the process of patient engagement in co-designing a survey identified indicators of arthritis care that had not featured in validated surveys related to the patient experience of health care. Secondly, the concept of integrated care appears to be a core need from the patients’ perspective in terms of the delivery of high quality arthritis care. Thirdly, the different roles in co-design and some of the issues that warrant further consideration. Finally, suggestions for future directions and some of the limitations and challenges of this work are explored.

In this discussion, we draw on the framework offered by Hamilton and colleagues to describe who the patients were, how we engaged with them and at what stages of the research cycle [17]. The model of patient engagement was primarily from Patient and Community Engagement Research (PaCER), as well as patients outside this group, and led the initial phase to identify what elements of arthritis care were important to them, using an inductive research approach [7, 19]. Findings from these initial studies were used to inform, along with other stakeholders, the development of a set of 28 Key Performance Indicators (KPIs) to evaluate centralized intake [21]. The importance of the patient experience, identified by all stakeholders as one of the top 10 KPI’s, led to the development of the survey. Hamilton’s model [17] was helpful in providing descriptors to identify important components for reporting patient engagement but the linear organization often belies the complexity of reality. In particular, we found a need for flexibility in the level of engagement with patient researchers. Patient researchers moved from collaborating, then leading, and then collaborating, depending on the task. Recognising the importance of flexibility in the model and the interplay between descriptors, around how engagement takes place is important. Our experience suggests that involving the end user is essential for integrated knowledge translation and capturing meaningful data through participation with others, reflection, and a goal of pursuing practical solutions for issues that concern people [37]. These initiatives intend to close the gap between research and practice. In the United States, recognizing the significant gap between research and application to clinical practice, a new taxonomy for patient engagement in patient-centered outcome research sought to reduce this discrepancy [36]. However, prior to external review with key stakeholders, the co-authors created three drafts with no patient stakeholder involvement, although they did in further iterations. Unless the patient stakeholder is involved at the outset it may be difficult to shape the future content or direction of outputs. In our study, the level of engagement for this stage was collaborative and iterative where decision-making was shared between patient and academic researchers [17]. We experienced minimal discrepancy between individual ratings for both the PaCER recommendations and survey item mapping, with the exception of those recommendations from the First Nations, Metis and Inuit cohort. This discrepancy perhaps reflected the academic researchers being more cognizant of the limited research resources available. However, during this process, we were surprised to find that several PaCER recommendations were not covered in the validated instruments survey items. This necessitated the development of survey items from these recommendations (see Table 1 for examples). This also resulted in a shift in the level of engagement, whereby the patient researchers were assigned authority over decision-making for an aspect of the research process, but executive power remained with other stakeholders [17]. Our initial premise being that these validated instruments would have captured all the important items. Yet closer scrutiny of one validated survey - Consumer Assessment of Healthcare Providers and Systems (CAHPS®) [31] - state that ‘patients with chronic conditions were represented’ in each of the four focus groups used to create an item pool. However, no details of the chronic conditions, and whether any of these patients had arthritis, were provided. This gap underlines the importance of ensuring that end users are involved at the earliest opportunity in the development of a survey to facilitate the generation of high quality data.

Table 1 Recommendations for high quality healthcare for OA patients from PaCER reporta and example survey items
Full size table

The patient experience survey centered on three themes derived from arthritis patient’s needs: the right knowledge, the right professional support and the right professional relationship [7]. Together these themes reflect the concept of integrated care, that has been defined as “patient care that is coordinated across professionals, facilities, and support systems; continuous over time and between visits; tailored to the patients’ needs and preferences; and based on shared responsibility between patient and caregivers for optimizing health” [38]. Patients clearly understand the important components for integrated care and identify when integration and coordination of care, do or do not happen, in their experiences with the healthcare system [39]. The importance of integrated or seamless care across primary and secondary centers for arthritis care, along with other long-term conditions has been highlighted [40, 41] and adopted widely, including Europe [42, 43], Australia [44], New Zealand and Canada [45]. A systemic review of integrated primary-secondary care identified improved elements of disease control and service delivery at a modestly increased cost, although the impact on clinical outcomes was reported to be limited [46]. Interestingly, reduced health care costs were a prominent feature in a systematic review of integrated care models for patients with chronic disease [47]. It is curious to ask what might be the outcomes that patients would value being measured, beyond clinical and cost? Unfortunately, barriers to integrative care models are extensive, often requiring organizational changes, greater personal and professional collaboration and primary care physician education [40]. In Alberta, Canada, a facilitated model of integrated care to improve arthritis detection and treatment, in an urban Aboriginal population, was successfully embedded in a primary care setting [48]. The Alberta Health Services 2017–2020 business plan has prioritized improving the experience of patients and families, and for the first time includes the specific goal of ‘ensuring our health system is integrated and coordinated between providers and patients’ [49]. Details on how this will be achieved are limited but it provides a strong impetus for the co-design of new systems for integrated care.

The process of co-design requires careful consideration of several factors, including when to involve patients in the research process, how to involve them and some of the potential challenges involved. For this study the two patient researchers had been involved from the outset, both in the earlier OA and RA PaCER studies (phase 1), and the development of the KPIs (phase 2). Engaging patients in all phases of the research study is seen as highly desirable [50]. Pragmatically, the process incurred challenges that have been well reported in the literature relating to patient engagement. Meaningful patient engagement incurs higher use of resources around time, costs and support [51, 52] and brings additional logistical challenges [50]. However, we believe the benefits outweigh the constraints around complexity and additional timing that co-design requires. Constructing a team with expertise in meaningful patient engagement, methodology and including patient researchers is essential for the production of high quality data.

Future directions

Patient engagement and measuring the patient experience is recognized as fundamental to enhancing service delivery, but care providers must also be engaged to listen and take action on patient feedback. The development of high quality patient experience data needs to be matched by the receivers (healthcare providers) being receptive to the information, to then close the loop and make the co-design of healthcare a reality. The role and importance of effective communication in quality improvement has been emphasized [53, 54], but systematic and structured communications strategies are not usually formally integrated into quality improvement initiatives [55]. As we become more sophisticated at developing high quality patient experience surveys, we suggest that new approaches are required close the feedback loop, by engaging health providers and creating the impetus for quality improvement. We have described our process of patient engagement to co-design a patient experience survey for people with arthritis referred to a central intake clinic. Using the Patient Engagement in Research Description (PED) framework to describe in more detail the process of patient engagement. This continues to be a work in process that will feed into our larger project and future work and firmly situates the patient and their experiences at the center.

Limitations

There were several limitations to the study. The survey requires further development and testing, beyond establishing face validity, to ensure it is a validated and robust tool for measuring the patient experience of central intake [56, 57]. In terms of developing items for the patient experience, we were less successful in identifying those related to patients needs around professional support. This warrants further exploration, from a patient perspective, to understand what sort of professional support could be offered in primary care, prior to referral to CI. The operationalization of the patient experience survey for this population is currently being explored and will contribute to the validity of the survey.

Conclusion

The purpose of this paper was to describe the process of patient engagement to co-design a patient experience survey for people with arthritis referred to central intake. Patient engagement and co-design of a patient experience survey identified aspects of care that had not been previously recognized. The concept of integrated care is a core requirement from the patients’ perspective for the delivery of high quality arthritis care. Frameworks to describe the level of patient engagement in research are helpful for reporting this activity but need to recognize the non-linearity of the process. New approaches are needed to prepare and facilitate care providers to receive and act upon data from patient experience surveys. Additional resources of cost, time and expertise for patient engagement in co-design activities are recognized and should be included, where possible, to ensure high quality data are captured.

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available due to the complexity of the multi-phased study but are available from the corresponding author on reasonable request.

Abbreviations

AHS:

Alberta Health Services

AI:

Alberta Innovates

CAHPS:

Consumer Assessment of Healthcare Providers and Systems

CI:

Central Intake

FNMI:

First Nations, Metis and Inuit

GRIPP:

Guidance for Reporting Involvement of Patients and Public

KPIs:

Key Performance Indicators

OA:

Osteoarthritis

PaCER:

Patient and Community Engagement Researchers

PED:

Patient Engagement in Research Description

PRP:

Patient Research Partner

RA:

Rheumatoid arthritis

YLDs:

Years Lost to Disability

References

  1. Global Burden of Disease Study Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 301 acute and chronic diseases and injuries in 188 countries, 1990 2013: a systematic analysis for the global burden of disease study 2013. Lancet. 2015;386:743–800.

    Article  Google Scholar 

  2. Bombardier C, Hawker G, Mosher D. The impact of arthritis in Canada: today and over the next 30 years. Arthritis Alliance of Canada. 2011. http://www.arthritisalliance.ca/images/PDF/eng/Initiatives/20111022_2200_impact_of_arthritis.pdf. Accessed 20 Mar 2019.

    Google Scholar 

  3. Harris JA, Bykerk VP, Hitchon CA, Keystone EC, Thorne JC, Boire G, Haraoui B, Hazlewood G, Bonner AJ, Pope JE. Determining best practices in early rheumatoid arthritis by comparing differences in treatment at sites in the Canadian early arthritis cohort. J Rheumatol. 2013;40:1823–30.

    Article  Google Scholar 

  4. Widdifield J, Bernatsky S, Bombardier C, Paterson M. Rheumatoid arthritis surveillance in Ontario: monitoring the burden, quality of care and patient outcomes through linkage of administrative health data. Healthc Q. 2015;18:7–10.

    Article  Google Scholar 

  5. MacDonald KV, Sanmartin C, Langlois K, Marshall DA. Symptom onset, diagnosis and management of osteoarthritis. Health Rep. 2014;25:10–7.

    PubMed  Google Scholar 

  6. Parsons GE, Godfrey H, Jester RF. Living with severe osteoarthritis while awaiting hip and knee joint replacement surgery. Musculoskeletal Care. 2009;7:121–35.

    Article  Google Scholar 

  7. Miller JL, Teare SR, Marlett N, Shklarov S, Marshall DA. Support for living a meaningful life with osteoarthritis: a patient-to-patient research study. Patient. 2016;9:457–64.

    Article  Google Scholar 

  8. Batalden M, Batalden P, Margolis P, Seid M, Armstrong G, Opipari-Arrigan L, Hartung H. Coproduction of healthcare service. BMJ Qual Saf. 2016;25:509–17.

    Article  Google Scholar 

  9. Wilson MG, Lavis JN, Gauvin FP. Designing integrated approaches to support people with multimorbidity: key messages from systematic reviews, health system leaders and citizens. Healthc Policy. 2016;12:91–104.

    PubMed  PubMed Central  Google Scholar 

  10. Mockford C, Staniszewska S, Griffiths F, Herron-Marx S. The impact of patient and public involvement on UK NHS health care: a systematic review. Int J Qual Health Care. 2012;24:28–38.

    Article  Google Scholar 

  11. Baker GR, Fancott C, Judd M, O’Connor P. Expanding patient engagement in quality improvement and health system redesign: three Canadian case studies. Healthc Manage Forum. 2016;29:176–82.

    Article  Google Scholar 

  12. Marlett N, Shklarov S, Marshall D, Santana MJ, Wasylak T. Building new roles and relationships in research: a model of patient engagement research. Qual Life Res. 2015;24:1057–67.

    Article  Google Scholar 

  13. Shklarov S, Marshall DA, Wasylak T, Marlett NJ. ‘‘part of the team”: mapping the outcomes of training patients for new roles in health research and planning. Health Expect. 2017;20:1428–36.

    Article  Google Scholar 

  14. Lopatina E, Damani Z, Bohm E, Noseworthy T, Conner-Spady B, MacKean G, Simpson C, Marshall DA. Single-entry models (SEMs) for scheduled services: towards a roadmap for the implementation of recommended practices. Health Policy. 2017;121:963–70.

    Article  Google Scholar 

  15. Damani Z, Conner-Spady B, Nash T, Stelfox HT, Noseworthy TW, Marshall DA. What is the influence of single-entry models on access to elective surgical procedures? A systematic review. BMJ Open. 2017;7:e012225.

    Article  Google Scholar 

  16. Hazlewood GS, Barr SG, Lopatina E, Marshall DA, Lupton TL, Fritzler MJ, Mosher DP, Steber WA, Martin L. Improving appropriate access to care with central referral and triage in rheumatology. Arthritis Care Res. 2016;10:1547–53.

    Article  Google Scholar 

  17. Hamilton CB, Leese JC, Hoens AM, Li LC. Framework for advancing the reporting of patient engagement in rheumatology research projects. Curr Rheumatol Rep. 2017;19:38.

    Article  Google Scholar 

  18. Gillis C, Gill M, Marlett N, MacKean G, GermAnn K, Gilmour L, Nelson G, Wasylak T, Nguyen S, Araujo E, Zelinsky S, Gramlich L. Patients as partners in enhanced recovery after surgery: a qualitative patient-led study. BMJ Open. 2017;6:e017002.

    Article  Google Scholar 

  19. Lopatina E, Miller JL, Teare SR, Marlett N, Patel JN, Barber CEH, Mosher DP, Wasylak T, Woodhouse LJ, Marshall DA. The voice of patients in system design: a case study of designing a centralized system for intake of referrals from primary care to rheumatologists for patients with suspected rheumatoid arthritis. Health Expect. 2018. https://0-doi-org.brum.beds.ac.uk/10.1111/hex.12855.

    Article  Google Scholar 

  20. Barnabe C, Miller J, Teare S, Eaglespeaker C, Roland B, Eshkakogan N, Crowshoe L, Lopatina E, Marshall DA. Solution model for enhancing the experiences of urban first nations and Métis patients accessing and navigating the health system for inflammatory arthritis care. Int Indig Policy J. 2019;10(2). https://0-doi-org.brum.beds.ac.uk/10.18584/iipj.2019.10.2.3.

  21. Barber CE, Patel JN, Woodhouse L, Smith C, Weiss S, Homik J, LeClercq S, Mosher D, Christiansen T, Howden JS, Wasylak T, Greenwood-Lee J, Emrick A, Suter E, Kathol B, Khodyakov D, Grant S, Campbell-Scherer D, Phillips L, Hendricks J, Marshall DA. Development of key performance indicators to evaluate centralized intake for patients with osteoarthritis and rheumatoid arthritis. Arthritis Res Ther. 2015;17:322.

    Article  Google Scholar 

  22. Health Quality Council of Alberta. The Alberta quality matrix for health. Health quality Council of Alberta. 2005. http://hqca.ca/about/how-we-work/the-alberta-quality-matrix-for-health-1/. Accessed 20 Mar 2019.

    Google Scholar 

  23. Barber CE, Marshall DA, Mosher DP, Akhavan P, Tucker L, Houghton K, Batthish M, Levy DM, Schmeling H, Ellsworth J, Tibollo H, Grant S, Khodyakov D, Lacaille D. Development of system level performance measures for evaluation of models of care for inflammatory arthritis in Canada. J Rheumatol. 2016;43:530–40.

    Article  Google Scholar 

  24. Dalal S, Khodyakov D, Srinivasan R, Straus S, Adams J. ExpertLens: a system for eliciting opinions from a large pool of non-collocated experts with diverse knowledge. Technol Forecast Soc Chang. 2011;78:1426–44.

    Article  Google Scholar 

  25. Staniszewska S, Brett J, Mockford C, Barber R. The GRIPP checklist: strengthening the quality of patient and public involvement reporting in research. Int J Technol Assess Health Care. 2011;27:391–9.

    Article  Google Scholar 

  26. Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, Altman DG, Moher D, Barber R, Denegri S, Entwistle A, Littlejohns P, Morris C, Suleman R, Thomas V, Tysall C. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. Res Involv Engagem. 2017;358:j3453.

    CAS  Google Scholar 

  27. Esmail L, Moore E, Rein A. Evaluating patient and stakeholder engagement in research: moving from theory to practice. J Comp Eff Res. 2015;4:133–45.

    Article  Google Scholar 

  28. Guba EG. The Paradigm dialog. Newbury Park: Sage Publications; 1990.

    Google Scholar 

  29. Bate P, Robert G. Experience-based design: from redesigning the system around the patient to co designing services with the patient. Qual Saf Health Care. 2006;15:307–10.

    Article  Google Scholar 

  30. Clarke D, Jones F, Harris R, Robert G. What outcomes are associated with developing and implementing co-produced interventions in acute healthcare settings? A rapid evidence synthesis. BMJ Open. 2017;7:e014650.

    Article  Google Scholar 

  31. Scholle SH, Vuong O, Ding L, Fry S, Gallagher P, Brown JA, Hays RD, Cleary PD. Development of and field-test results for the CAHPS PCMH survey. Med Care. 2012;50:S2–10.

    Article  Google Scholar 

  32. Wong ST, Haggerty J. Measuring patient experiences in primary health care: a review and classification of items and scales used in publicly-available questionnaires. Centre for Health Services and Policy Research (CHSPR); 2013. https://0-doi-org.brum.beds.ac.uk/10.14288/14281.0048528.

    Book  Google Scholar 

  33. Tijhuis GJ, Kooiman KG, Zwinderman AH, Hazes JM, Breedveld FC, Vliet Vlieland TP. Validation of a novel satisfaction questionnaire for patients with rheumatoid arthritis receiving outpatient clinical nurse specialist care, inpatient care, or day patient team care. Arthritis Rheum. 2003;49:193–9.

    Article  Google Scholar 

  34. The Picker Institute Europe. Outpatient survey 2011: The Royal Bournemouth and Christchurch Hospitals NHS Foundation Trust: The Picker Institute Europe; 2011. http://www.rbch.nhs.uk/assets/templates/rbch/documents/patient_visitors/have_your_say/outpatient_survey_2011.pdf. Accessed 01 Apr 2018.

  35. First Nations Health Authority. Traditional wellness strategic framework. First nations health Authority. 2014. http://www.fnha.ca/wellnesscontent/wellness/fnha_traditionalwellnessstrategicframework.pdf. Accessed 20 Mar 2019.

    Google Scholar 

  36. Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise JM, Santa J, Conway PH, Daudelin D, Morrato EH, Leslie LK. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J Gen Intern Med. 2012;27:985–91.

    Article  Google Scholar 

  37. Carr EC, Babione JN, Marshall D. Translating research into practice through user-centered design: an application for osteoarthritis healthcare planning. Int J Med Inform. 2017;104:31–7.

    Article  Google Scholar 

  38. Singer SJ, Burgers J, Friedberg M, Rosenthal MB, Leape L, Schneider E. Defining and measuring integrated patient care: promoting the next frontier in health care delivery. Med Care Res Rev. 2011;68:112–27.

    Article  Google Scholar 

  39. Walker KO, Labat A, Choi J, Schmittdiel J, Stewart AL, Grumbach K. Patient perceptions of integrated care: confused by the term, clear on the concept. Int J Integr Care. 2013;13:e004.

    PubMed  PubMed Central  Google Scholar 

  40. Pollard LC, Graves H, Scott DL, Kingsley GH, Lempp H. Perceived barriers to integrated care in rheumatoid arthritis: views of recipients and providers of care in an inner-city setting. BMC Musculoskelet Disord. 2011;12:19.

    Article  Google Scholar 

  41. Government of the United Kingdom - Department of Health and Social Care. National Collaboration for Integrated Care and Support. Integrated care: our shared commitment. Government of the United Kingdom - Department of Health and Social Care. 2013. FINAL_VERSION_Integrated_Care_and_Support__Our_Shared_Commitment_2013-05-13.pdf. Accessed 20 Mar 2019.

  42. Eger K, Gleichweit S, Rieder A, Stein KV. Prioritising integrated care initiatives on a national level. Experiences from Austria. Int J Integr Care. 2009;9:e91.

    Article  Google Scholar 

  43. Huber CA, Reich O, Früh M, Rosemann T. Effects of integrated care on disease-related hospitalisation and healthcare costs in patients with diabetes, cardiovascular diseases and respiratory illnesses: a propensity-matched cohort study in Switzerland. Int J Integr Care. 2016;16:11.

    Article  Google Scholar 

  44. Connor M, Cooper H, McMurray A. The Gold Coast integrated care model. Int J Integr Care. 2016;16:2.

    Article  Google Scholar 

  45. Breton M, Gray CS, Sheridan N, Shaw J, Parsons J, Wankah P, Kenealy T, Baker R, Belzile L, Couturier Y, Denis JL, Wodchis WP. Implementing community based primary healthcare for older adults with complex needs in Quebec, Ontario and new-Zealand: describing nine cases. Int J Integr Care. 2017;17:12.

    Article  Google Scholar 

  46. Mitchell GK, Burridge L, Zhang J, Donald M, Scott IA, Dart J, Jackson CL. Systematic review of integrated models of health care delivered at the primary-secondary interface: how effective is it and what determines effectiveness? Aust J Prim Health. 2015;21:391–408.

    Article  Google Scholar 

  47. Desmedt M, Vertriest S, Hellings J, Bergs J, Dessers E, Vankrunkelsven P, Vrijhoef H, Annemans L, Verhaeghe N, Petrovic M, Vandijck D. Economic impact of integrated care models for patients with chronic diseases: a systematic review. Value Health. 2016;19:892–902.

    Article  Google Scholar 

  48. Barnabe C, Lockerbie S, Erasmus E, Crowshoe L. Facilitated access to an integrated model of care for arthritis in an urban aboriginal population. Can Fam Physician. 2017;63:699–706.

    PubMed  PubMed Central  Google Scholar 

  49. Alberta Health Services. The 2017-2020 health plan and business plan: Alberta health services; 2017. https://www.albertahealthservices.ca/assets/about/org/ahs-org-hpbp-2017-2020.pdf. Accessed 20 Mar 2019

  50. Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, Brito JP, Boehmer K, Hasan R, Firwana B, Erwin P, Eton D, Sloan J, Montori V, Asi N, Abu Dabrh AM, Murad MH. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14:89.

    Article  Google Scholar 

  51. Haywood K, Brett J, Salek S, Marlett N, Penman C, Shklarov S, Norris C, Santana MJ, Staniszewska S. Patient and public engagement in health-related quality of life and patient-reported outcomes research: what is important and why should we care? Findings from the first ISOQOL patient engagement symposium. Qual Life Res. 2015;24:1069–76.

    Article  Google Scholar 

  52. Forsythe LP, Ellis LE, Edmundson L, Sabharwal R, Rein A, Konopka K, Frank L. Patient and stakeholder engagement in the PCORI pilot projects: description and lessons learned. J Gen Intern Med. 2016;31:13–21.

    Article  Google Scholar 

  53. Woolf SH, Purnell JQ, Simon SM, Zimmerman EB, Camberos GJ, Haley A, Fields RP. Translating evidence into population health improvement: strategies and barriers. Annu Rev Public Health. 2015;36:463–82.

    Article  Google Scholar 

  54. Agency for Healthcare Research & Quality. Strategy 2: the communicating to improve quality strategy and its tools help facilitate this communication. Agency for Healthcare Research & Quality. 2017. http://www.ahrq.gov/professionals/systems/hospital/engagingfamilies/strategy2/index.html. Accessed 20 Mar 2019.

    Google Scholar 

  55. Cooper A, Gray J, Willson A, Lines C, McCannon J, McHardy K. Exploring the role of communications in quality improvement: a case study of the 1000 lives campaign in NHS Wales. J Commun Healthc. 2015;8:76–84.

    Article  Google Scholar 

  56. Oppenheim AN. Questionnaire design, interviewing and attitude measurement. 2nd ed. London: Bloomsbury Academic; 2000.

    Google Scholar 

  57. Boynton PM, Greenhalgh T. Selecting, designing, and developing your questionnaire. BMJ. 2004;328:1312–5.

    Article  Google Scholar 

Download references

Acknowledgments

We would like to acknowledge Dr. Esther Suter who assisted in developing the patient experience survey. We acknowledge and thank the patients who generously shared their experiences.

Funding

Alberta Innovates Health Solutions, Partnership for Research and Innovation in the Health System (PRIHS) Grant: “Optimizing Centralized Intake to Improve Arthritis Care for Albertans”; The Arthritis Society, Models of Care Catalyst grant: “Creating an optimal model of care for the efficient delivery of appropriate and effective arthritis care”; Canadian Institute for Health Research (CIHR) Planning Grant (funded through Priority Announcement Health Services and Policy Research): “Evidence based planning of an optimal triaging strategy for arthritis care in Alberta”; Alberta Health Services, Research Grant (through the Bone and Joint Health Strategic Clinical Network): “Optimizing Centralized Intake to Improve Arthritis Care for Albertans.” The funding bodies had no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

Author information

Authors and Affiliations

  1. Faculty of Nursing, University of Calgary, PF2237, 2500 University Drive NW, Calgary, Alberta, T2N 1N4, Canada

    Eloise C. J. Carr & Mia M. Ortiz

  2. Pan-SCN Manager, Strategic Clinical Networks™, Alberta Health Services, 10030 – 107 Street NW, Edmonton, Alberta, T5J 3E4, Canada

    Jatin N. Patel

  3. O’Brien Institute for Public Health, University of Calgary, 3280, Hospital Dr. NW, Calgary, Alberta, T2N 4Z6, Canada

    Jean L. Miller & Sylvia R. Teare

  4. Arthritis Research Center, University of Calgary, HRIC 3AA20, 3280, Hospital Dr. NW, Calgary, Alberta, T2N 4Z6, Canada

    Claire E. H. Barber

  5. Cumming School of Medicine, University of Calgary, Health Research Innovation Centre (HRIC) – 3C56, 3280 Hospital Drive NW, Calgary, Alberta, T2N 4Z6, Canada

    Deborah A. Marshall

Authors
  1. Eloise C. J. Carr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jatin N. Patel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mia M. Ortiz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jean L. Miller
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sylvia R. Teare
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Claire E. H. Barber
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Deborah A. Marshall
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

EC contributed to the methodological evaluation of the study and patient experience survey, the analysis and led the development of the manuscript. JP executed the study protocol, assisted in developing the patient experience survey and contributed to the manuscript development. MO coordinated the data collection and analysis. JM and ST participated in the design of the study methodology, developing the patient experience survey and co-led the PaCER study. CB assisted in developing the study methodology and the patient experience survey. DM conceived the study, designed the study methodology, coordinated the study, and contributed to data collection and analysis. All authors were involved in the development of the manuscript and commented on the final version. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Eloise C. J. Carr.

Ethics declarations

Ethics approval and consent to participate

Institutional ethical review and approval were obtained for each of the phases (University of Calgary Conjoint Health Research Ethics Board ID: REB13–0822). All participants provided written informed consent.

Consent for publication

Not applicable.

Competing interests

Dr. Eloise C. J. Carr is a Co-investigator on PRIHS grant, Optimizing Centralized Intake to.

Improve Arthritis Care for Albertans. Dr. Deborah A. Marshall is a Professor at the University of Calgary, Canada Research Chair and Arthur J.E. Child Chair. Dr. Marshall is also a recipient of CIHR Emerging Team Grant.

The authors Jatin N. Patel, Mia Maris Ortiz, Dr. Jean Louise Miller, Sylvia Ruth Teare and Dr. Claire E.H. Barber declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:

Patient Experience Survey. (DOCX 37 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Carr, E.C.J., Patel, J.N., Ortiz, M.M. et al. Co-design of a patient experience survey for arthritis central intake: an example of meaningful patient engagement in healthcare design. BMC Health Serv Res 19, 355 (2019). https://0-doi-org.brum.beds.ac.uk/10.1186/s12913-019-4196-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12913-019-4196-9

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us