Skip to main content
Download PDF

Use and usability of the dr. Bart app and its relation with health care utilisation and clinical outcomes in people with knee and/or hip osteoarthritis

BMC Health Services Research volume 21, Article number: 444 (2021) Cite this article

Abstract

Background

Self-management is of paramount importance in the non-surgical treatment of knee/hip osteoarthritis (OA). Modern technologies offer the possibility of 24/7 self-management support. We developed an e-self-management application (dr. Bart app) for people with knee/hip OA.

The aim of this study was to document the use and usability of the dr. Bart app and its relation with health care utilisation and clinical outcomes in people with knee/hip OA.

Methods

For this study we used backend data for the first 26 weeks of use by the intervention group (N = 214) of an RCT examining the effectiveness of the dr. Bart app. A central element of the dr. Bart app is that it proposes a selection of 72 preformulated goals for health behaviours based on the ‘tiny habits method’ (e.g. after lunch I rise 12 times from my chair to train my leg muscles). The usability of the app was measured using the System Usability Scale questionnaire (SUS), on a scale of 0–100. To assess the association between the intensity of use of the app and health care utilisation (i.e., consultations in primary or secondary health care) and clinical outcomes (i.e., self-management behaviour, physical activity, health-related quality of life, illness perceptions, symptoms, pain, activities of daily living) we calculated Spearman rank correlation coefficients.

Results

Of the 214 participants, 171 (80%) logged in at least once with 151 (71%) choosing at least one goal and 114 (53%) completing at least one goal during the 26 weeks. Of those who chose at least one goal, 56 participants (37%) continued to log in for up to 26 weeks, 12 (8%) continued to select new goals from the offered goals and 37 (25%) continued to complete goals. Preformulated goals in the themes of physical activity (e.g., performing an exercise from the exercises library in the app) and nutrition (e.g., ‘eat two pieces of fruit today’) were found to be most popular with users. The mean usability scores (standard deviation) at the three and six month follow-ups were 65.9 (16.9) and 64.5 (17.5), respectively. The vast majority of associations between the intensity of use of the dr. Bart app and target outcomes were weak at ρ < (−) 0.25.

Conclusions

More than one-third of people with knee/hip OA who started using the app, continued to use it up to 26 weeks, though usability could be improved. Patients appear to have preferences for goals related to physical activity and nutrition, rather than for goals related to vitality and education. We found weak/no associations between the intensity of use of the dr. Bart app and health care utilisation and clinical outcomes.

Trial registration

(21 September 2017): Dutch Trial Register (Trial Number NTR6693/NL6505)

Peer Review reports

Background

Osteoarthritis (OA) of the knee/hip is the most common form of disability of movement and is characterised by pain, stiffness, and a decline in daily functioning. The primary approaches for non-surgical treatment in early stages of knee/hip OA are lifestyle education, exercise therapy, weight management and pain medication [1,2,3,4]. As OA is a chronic disease, a cornerstone of its non-surgical treatment is self-management. Self-management interventions offer patients guidance in improving their skills to take better care of themselves and take an active role in their disease management [5, 6], including skills navigating the health care system (i.e., making optimal use of primary and secondary health care options).

Compared with usual care, traditional self-management interventions (e.g. face-to-face education) show small benefits in the self-management skills, pain, and function of people with knee/hip OA [7]. Mobile health applications have the potential to enhance self-management 24/7 (i.e., 24 h a day, 7 days a week). Their growing and emerging popularity of eHealth applications have resulted in a proliferation of applications in the health domain; however, the majority of mobile health applications have not proven their effectiveness in clinical trials [8,9,10,11,12].

Given the potential of these modern technologies, we developed a fully automated stand-alone mobile health application (the ‘dr. Bart app’) to enhance self-management in people with knee/hip OA. The content of the dr. Bart app is based on the Fogg model for behavioural change, augmented with reminders, rewards and self-monitoring to reinforce app engagement [13, 23]. In a randomized evaluation, we found that the dr. Bart app did not impact health care utilisation (HCU), but resulted in small positive effects on pain, symptoms and activities of daily living [14]. However, a fundamental issue in eHealth research is non-usage attrition; a proportion of participants do not use the intervention at all, or use it sparsely [15].

Although the issue regarding non-usage attrition is well-known, most eHealth studies do not provide information regarding use, despite the possible diminishing of the effects of the intervention by low exposure rates [16,17,18]. Studies on stand-alone eHealth tools in other chronic diseases (e.g., diabetes and chronic pulmonary disease) showed that after a month the applications were used by fewer than 50% of participants [19]. Two studies evaluating stand-alone eHealth interventions assisting patients in their preparation for the first consultation with an orthopaedic surgeon, which could be used either one or 2 weeks prior to the consultation, found relatively high user rates (i.e. 70%) in people with OA [20, 21]. So far, there is little insight into whether the actual usage of stand-alone eHealth applications in people with OA enhances self-management and its determinants. Furthermore, it is likely that there is a dose response relationship between the (intensity of) use of an eHealth intervention and the clinical outcomes [22]; however, there has been little insight into the association between the extent of the use of (different components of) an app and its effects on target outcomes. The aim of this explorative study was therefore to quantify the use, identify the patterns of use and assess the usability of the dr. Bart app over half a year. Furthermore, we explored the association between the intensity of use of the dr. Bart app and its relation with HCU and clinical outcomes in people with knee/hip OA over half a year. Finally, we aimed to gain insight into the demographic and clinical characteristics of various types of users.

Methods

Design and setting

The data in the present study were collected as part of a randomised controlled trial (RCT), evaluating the effectiveness of the dr. Bart app on health care use and clinical outcomes, which was conducted by the Sint Maartenskliniek Nijmegen, the Netherlands, from 24 January 2018 to 7 January 2019. The original study is registered in the Dutch Trial Register (trial number NTR6693/NL6505) (https://www.trialregister.nl/trial/6505) as of 21 September 2017. All participants provided digital informed consent for participation. The ethical approval for this study was waived by the Medical Research Ethics Committee of the Radboud University Medical Centre, Nijmegen, the Netherlands (CMO Arnhem-Nijmegen, protocol number: 2017–3625) because the study was considered outside the remit of the law (Medical Research Involving Human Subjects Act). This study is reported according to the CONSORT guidelines.

Participants and procedure

Participants were recruited through advertisements in local newspapers (i.e., the region of Nijmegen, the Netherlands), and throughout the Netherlands via campaigns on social media (i.e., Facebook, Twitter, and LinkedIn). Participants willing to participate were invited to the website (https://www.drbart.eu/) to check their eligibility. The inclusion criteria were: 1) having self-reported OA of the knee and/or hip (i.e., having a painful knee and/or hip, knee and/or hip pain > 15 days of the past month, morning stiffness < 30 min (knee) and/or < 60 min (hip)); 2) ≥ 50 years; 3) having an e-mail address; 4) possession of a smartphone or tablet and willing to download the dr. Bart application on one or more devices; and 5) able to read, write and sufficiently communicate in Dutch.

Exclusion criteria were as follows: 1) being wheelchair-bound; 2) diagnosis of (other) inflammatory rheumatic disease; 3) knee and/or hip replacements; and 4) scheduled for knee and/or hip joint arthroplasty in the next 6 months. Eligible participants were asked to provide their e-mail address and subsequently received a baseline assessment via CastorEDC, an electronic software application for data collection and management (https://www.castoredc.com/). Baseline and follow-up data at three and 6 months were taken from the intervention group (N = 214) of an RCT examining the effectiveness of the dr. Bart app, together with backend data (i.e., the technology component responsible for processing and storing the data) of the app over 26 weeks and used for the present analysis [13].

Intervention

The dr. Bart app is a stand-alone mobile health application that was designed to enhance self-management and actively involve people with knee/hip OA in the management of their disease. This mobile health application is based on the Fogg model for behavioural change [23], augmented with other motivation-enhancing techniques such as reminders, rewards and self-monitoring, to reinforce app engagement and health behaviour. Users receive a daily push notification from dr. Bart. Additionally, the app automatically sends a push notification stating: “We have not seen you in a while. Do you think of your goals?” when a user has not opened the app for more than 7 days. The Fogg model, also known as the ‘tiny habits method’, utilises the concept of accumulating small goals to structurally change health behavior, and in the long run health outcomes. Machine learning techniques are used to propose tailored goals based on data collected in a personal profile and on previously selected and discarded goals. For the dr. Bart app, the machine learning comprised a dynamic model (contextual multi-armed bandit approach), proposing goals that are challenging, achievable and tailored for that specific user. The content and functionalities of the dr. Bart app were frozen during the study period (version 1.3.7), although bug fixes (e.g., failure to log in) and system failures were resolved. Further details on the theoretical framework, development and functionalities of the dr. Bart app have been published elsewhere [13]. Screenshots of the dr. Bart app are presented in Additional file 1: Appendix 1.

Assessments

At the baseline, and three and 6 months after inclusion, participants received online questionnaires via CastorEDC (https://www.castoredc.com).

Use of the dr. Bart app

Prior to start of the study, we decided which parameters of use should be logged and extracted from the backend of the app to quantitatively measure its use. These data were collected for the 26-week study period. The parameters of use were automatically logged and extracted for each participant. ‘Non-users’ were those participants who never logged in. To elaborate on the nature and extent of use of the app, we further classified use of the app as:

  1. 1)

    active with logins, but no further activity

  2. 2)

    active with choosing goals, but without completing goals

  3. 3)

    active with completing ≥ 1 goals

Users can choose more than one of the proposed goals simultaneously and goals can be completed more than once by the same user. The following indicators of use were extracted from the backend of the app: number of logins, number of unique chosen goals, number of unique goals completed, and total number of completed goals. Moreover, we quantified the use of information as the number of paragraphs read of the educational library (range 0–108), which indicates exposure to information.

For participants who chose at least one goal, we constructed Kaplan-Meier curves to illustrate the percentage of persons who used the app over time, based on the aforementioned indicators of use.

Usability

We assessed the usability of the dr. Bart app with the System Usability Scale (SUS) at three and six months [24, 25]. The SUS is a 10-item questionnaire scored on a five-point Likert scale (“Strongly agree” to “Strongly disagree”. We calculated a total score ranging from 0 to 100, with a higher score indicating better usability. Additionally, we provided a free-text option after each question, so participants could elaborate on their given answers.

Demographic and clinical characteristics

Demographic data were collected at the baseline. We assessed pain, symptoms, activities of daily living, quality of life, and physical functioning in sports and recreation, with subscales of either the Knee injury or Hip disability Osteoarthritis Outcome Score (KOOS or HOOS), ranging from 0 to 100, where a higher score indicates fewer problems in that domain [26, 27]. We assessed health-related quality of life using the EQ-5D-3L (0–1; with a higher score reflecting better health) [28]. Physical activity was assessed with the Short Questionnaire to Assess Health-Enhancing Physical Activity (SQUASH) [29]. Knowledge, skills and confidence to cope with one’s health were assessed with the Patient Activation Measure (PAM-13) questionnaire [30, 31]. We used the Illness Perception Questionnaire (IPQ) to assess the patient’s cognitive and emotional perception regarding their OA (0–80; higher score indicating more concerning views of OA) [32]. Moreover, we assessed both positive and negative treatment beliefs regarding various treatment modalities (i.e., physical activities, pain medication, physical therapy, injections, and joint replacement surgery) in knee and hip OA with the Treatment Beliefs in Osteoarthritis (TOA) questionnaire [33]. Psychometric properties are satisfactory to good. However, minimal clinically important difference is not yet available. We calculated mean sub scale scores ranging from 1 to 5 for the TOA.

Statistical analysis

Data analysis

All statistical analyses were performed using Stata 13.1 [34]. The (missing) data were handled according to the recommendations of the specific questionnaire. For the PAM, we also calculated a total score when a maximum of two items of the questionnaire were missing, though the PAM recommends to only calculate a total score if no single item is missing. For the SUS questionnaire, we did not calculate a total score when two or more items were missing. Descriptive statistics were used to describe participant characteristics and parameters of use. In all analyses, we considered p < 0.05 to be statistically significant. Since this is an explorative study, we refrained from correcting for multiple testing.

Subgroup characteristics

In order to determine whether baseline characteristics could be used to predict the use of the app by subgroups of participants, parameters of use were taken as the dependent variable in univariate regression analyses, with the baseline characteristic as the independent variable.

Association between use and clinical outcomes

To assess the association between the intensity of (different indicators of) use of the app and changes in HCU and clinical outcomes over 6 months of follow-up, we calculated Spearman rank correlation coefficients. Additionally, we classified users into six groups for the four separate indicators of use (i.e., number of logins, number of chosen goals, number of completed goals and number of read paragraphs) based on backend data; non-users and a population split into five equal groups (i.e., quintiles). Subsequently, boxplots of relative differences in clinical outcomes were created for those six groups for the four different indicators of use.

Results

The original randomised controlled trial included 427 participants with knee/hip OA [14]. Of those, 214 participants were randomised into the intervention group and subsequently included in the current explorative study. The mean age of these participants was 62.1 years (SD 7.7), with the majority being female (68.7%). Most participants had symptoms predominantly in their knee(s) (73.4%), and 60 % had OA symptoms for less than 5 years (Table 1).

Table 1 Baseline and clinical characteristics of the participants in the study
Full size table

Use of the dr. Bart app

Among the participants, 171 (79.9%) were active with logins, 151 (70.6%) were active with choosing goals and 113 (52.8%) were active with completing goals. The remaining 20.1% of participants did not log in to the app over the course of the study. We did not find relevant differences in the baseline characteristics of those who were active with the dr. Bart app and those who were less active or inactive (Additional file 1: Appendix 2).

In total, participants logged in 7006 times, chose 1062 goals, completed 884 unique goals and completed 9229 goals over the 26 weeks (Table 2). The median number of paragraphs read of the educational library was 42 (interquartile range (IQR): 18–84) for participants who were active with completing goals (Table 2), while 50% of participants who were active and chose goals read more than 30 paragraphs in the educational library. Paragraphs about OA and its complaints, progression, conservative treatment options, as well as the exercise library, were read more often than paragraphs related to the pathogenesis of OA, pharmacological care, assistive technologies and surgical treatments (Additional file 1: Appendix 3).

Table 2 Indicators of use over 26 weeks, presented for three pre-specified groups of users based on their usage activity (N = 171). Of note, non-users are not included in this Table
Full size table

Fig. 1 shows the proportion of participants actively using the dr. Bart app over half a year, with separate lines for the three indicators of use. Of the participants who chose at least one goal, 38% were active with logins after 26 weeks, 8% were active with choosing goals and 25% were active with completing goals. Appendices 4 and 5 show the median and IQR of the number of cumulative logins and completed goals over time of users who chose at least one goal, respectively.

Fig. 1
figure 1

Percentage of users that are active with logins, choosing goals or completing goals over time as a fraction of the users that chose at least one goal during the study (N = 151)

Full size image

Goals related to physical activity were relatively most often chosen (times chosen divided by times proposed) and completed (times completed divided by times chosen), and goals related to nutrition were more popular than goals related to vitality or education. Appendices 6 and 7 show the five relatively most and least often chosen and completed goals.

Usability

After three and three months of follow-up, the mean usability scores were 65.9 (SD 16.9) and 64.5 (SD 17.5), respectively. People who more intensively used the app, gave higher usability scores (Table 2). Participants rated the main features (i.e. education and exercise library) of the app positively in the additional free-text option of the SUS (Additional file 1: Appendix 8). Moreover, the ability to access this information at any given time and place was considered important, as well as to incorporate the ‘tiny habits’ easily in daily life. On the other hand, 14 participants stated that the app did not provide any new information or exercises or that they did not always carry their mobile phone. Moreover, 13 participants did not see the benefits of using such an application (Additional file 1: Appendix 8).

Subgroup characteristics

We could not (clearly) distinguish between non-users and different intensities of users, based on age, gender, body mass index (BMI), living situation, level of education, or main OA location (Additional file 1: Appendix 9).

Association between use and clinical outcomes

We found Spearman’s rank correlation coefficients ranging from − 0.20 to 0.30 between different indicators of use and HCU and clinical outcomes (Table 3). We found a statistically significant correlation between visiting a general practitioner (yes/no) and the number of unique chosen goals (ρ = 0.25) and unique goals completed (ρ = 0.28).

Table 3 Spearman rank correlation coefficients between different indicators of use and health care utilisation and clinical outcomes (relative difference between baseline and 6 month follow-up)
Full size table

Negative beliefs about physical activity, medication and physical therapy were associated with lower app use (Additional file 1: Appendix 10).

A visual inspection of boxplots showed an absence of a dose-response relationship (data not shown).

Discussion

The aim of this study was to document the use and usability of the dr. Bart app and to examine intensity of use of the app and its relation with HCU and clinical outcomes. The results of this study show that more than one-third of participants who were offered the dr. Bart app persistently used it over their first 6 months of access. The two main features of the app, goal setting and the educational library (including the exercise library), were extensively used. More than half of the participants completed at least one goal, with goals related to physical activity and nutrition being the most popular. In the educational library, participants were predominantly interested in general information regarding OA, complaints (specifically fatigue), treatment options, prognosis, and the exercise library. After half a year, a quarter of users still used the app to set and complete goals and two-fifth of users still opened the app. We could not identify differences in the characteristics of non-users and users of the dr. Bart app. Moreover, we were not able to demonstrate a dose-response relationship between different indicators of use and HCU or clinical outcomes.

To our knowledge little is known about the use of stand-alone mobile health applications in OA. Our user rates are in line with two studies evaluating stand-alone mobile health applications assisting people with OA in their preparation for their first consult with an orthopaedic surgeon [20, 21]; however, these applications had relatively short time frames (i.e., one or 2 weeks prior to consultation) and a different focus, which might have resulted in higher user rates in these studies. Stand-alone applications in other chronic diseases (chronic pulmonary disease or diabetes) are not directly comparable, as the use of these applications is more likely to result in short-term benefits (e.g. fewer exacerbations and betters glucose levels), and are consequently more likely to be used than the dr. Bart app. Nevertheless, the user rates over time observed in our study were even higher than those observed in stand-alone eHealth applications for the self-management of other chronic diseases [35,36,37,38]. Additionally, the use of the dr. Bart application is comparable with the use of a self-management application embedded in clinical practice (blended intervention in people with OA) [39], despite the use of blended interventions being more likely to result in higher user rates than stand-alone applications [16]. Taken together, we conclude that the usage rate of the stand-alone dr. Bart application is relatively high. This could be explained by the application of different reinforcement techniques (i.e. rewards, reminders and self-monitoring) to reinforce engagement with the dr. Bart app [13].

The majority of users consulted the educational library on a regular basis and were particularly interested in general information regarding OA, complaints, progression and, (non-pharmacological) conservative treatment options. Interestingly, information about the services provided by health care professionals was less frequently consulted, despite the navigation of the health care system being an important aspect of self-management. A possible explanation is that the majority of our sample is highly educated, which is known to be associated with possessing better skills of navigating the health care system [40]. Also, information with respect to the pathogenesis of OA, pharmacological care, walking aids, assistive technology, and surgical treatments were found to be less popular. This is an interesting finding, as these treatments are also considered important in the treatment of OA; however, topics considered important in the guidelines, do not necessarily reflect the educational needs of patients [41]. Our study provides insights into the educational needs of patients with knee/hip OA and offers starting points for optimising patient education.

Contrary to expectations, we did not find associations between the (intensity of) use of the dr. Bart app and HCU and clinical outcomes. Studies on web-based interventions in patients with mental disorders suggest that certain levels of use result in therapy saturation, and that patients are most likely to obtain benefits from the intervention early on [22]. The lack of any strong relationship between use and HCU and clinical outcomes might be explained by this model; users might have reached a plateau in which they did not benefit from additional use, with most benefits of the intervention coming early on. Given the nature of the data collected (we only have clinical outcomes after three and six months of follow-up), we were not able to assess the relationship between the early use of the app and its relationship with HCU and clinical outcomes. Future studies should assess the exact relationship between usage and HCU and clinical outcomes.

The usability of the app (assessed with the SUS) could be improved, as it does not reach an acceptable score of 70, which in turn might result in higher usage rates. However, a study with a satisfactory to good usability score regarding an application for post-operative self-reporting after colorectal surgery showed that participants did not use the app or used it only once [42]; thus, high usability alone is not sufficient to motivate patients to use eHealth applications. This is in line with the Technology Acceptance Model, which states that actual system use is not only dependent of the perceived ease-of-use (i.e., usability), but also on the perceived usefulness. Perceived usefulness is described as “the degree to which a person believes that using a particular technology would enhance his/her performance”. It is therefore important that participants see the necessity and benefits of using an application, in addition to the usability of the technology itself. Qualitative information about the usability of the app, not covered by the SUS, could help to identify targets to improve the usability [43]. For future studies, we recommend performing qualitative studies in parallel with the evaluation of a new intervention as this is essential to derive useful insights from end-users. Moreover, qualitative studies allow the elicitation of suggestions for improvement, which can be incorporated in new versions as well as the identification of facilitators and barriers for the use of the intervention.

Strength and limitations

The current study has several limitations that should be taken into account when interpreting the results. First, sample selection bias is likely to be apparent, as participants were willing to participate in research on a mobile health application, which in turn might induce the Hawthorne effect (i.e., a change in behaviour of participants due to their awareness of being observed). Moreover, it is unknown why 20% of participants did not open the app. It is therefore conceivable that user rates are lower and non-use is higher when the application is applied in clinical practice. Furthermore, the results of this exploratory study were based on four different indicators of use, since a consensus on a definition of use and how to measure it is lacking. Nevertheless, we think that applying four different indicators of use, resulted in a thorough understanding of the use of the app. Since this is an explorative study, we chose not to formulate particular hypotheses in advance, which should be taken into account when interpreting the results. As said, the current study was exploratory in nature and we did not define the intended usage prior to the start of the study. Since intended use of a technology forms an important element of the definition of adherence. In future investigations, it might be useful to operationalise the intended use of eHealth technologies based on theory, evidence or rationale as the intended use of a technology forms an important element of the definition of adherence.

Conclusions

This study is the first to elucidate the use and usability of a stand-alone mobile health application in people with knee/hip OA. A considerable proportion of participants persistently used the dr. Bart app for up to half a year, confirming that a stand-alone mobile health application has the potential to reach the target population. However, we were not able to demonstrate a dose-response relationship between use of the app and HCU and clinical outcomes. Additionally, this study provides insights into the educational needs of patients with knee/hip OA. In conclusion, we think the dr. Bart app has the potential to serve as a trustworthy tool to provide education and facilitate goal setting in people with knee/hip OA.

Availability of data and materials

The datasets used and/or analysed in the present study are available from the corresponding author upon reasonable request.

Abbreviations

OA:

Osteoarthritis

SUS:

System Usability Scale

app:

application

RCT:

Randomized Controlled Trial

SD:

Standard Deviation

KOOS or HOOS:

Knee injury/Hip disability Osteoarthritis Outcome Score

PA:

Physical Acitivity

PAM:

Patient Activitation Measure

IPQ:

Illness Perception Questionnaire

HCU:

health care utilization

IQR:

Interquartile Range

BMI:

Body Mass Index

TAM:

Technology Acceptance Model

References

  1. Bannuru RR, Osani MC, Vaysbrot EE, Arden NK, Bennell K, Bierma-Zeinstra SMA, et al. OARSI guidelines for the non-surgical management of knee, hip, and polyarticular osteoarthritis. Osteoarthr Cartil. 2019;27(11):1578–89. https://0-doi-org.brum.beds.ac.uk/10.1016/j.joca.2019.06.011.

    Article  CAS  Google Scholar 

  2. Fernandes L, Hagen KB, Bijlsma JWJ, Andreassen O, Christensen P, Conaghan PG, et al. EULAR recommendations for the non-pharmacological core management of hip and knee osteoarthritis. Ann Rheum Dis. 2013;72(7):1125–35. https://0-doi-org.brum.beds.ac.uk/10.1136/annrheumdis-2012-202745.

    Article  PubMed  Google Scholar 

  3. Smink AJ, van den Ende CHM, Vliet Vlieland TPM, Swierstra BA, Kortland JH, Bijlsma JWJ, et al. “Beating osteoARThritis”: development of a stepped care strategy to optimize utilization and timing of non-surgical treatment modalities for patients with hip or knee osteoarthritis. Clin Rheumatol. 2011;30(12):1623–9. https://0-doi-org.brum.beds.ac.uk/10.1007/s10067-011-1835-x.

    Article  PubMed  Google Scholar 

  4. Altman RD. Early management of osteoarthritis. Am J Manag Care. 2010;16 Suppl M(2):S41–7 http://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/pubmed/20297876.

    Google Scholar 

  5. Barlow J, Wright C, Sheasby J, Turner A, Hainsworth J. Self-management approaches for people with chronic conditions: a review. Patient Educ Couns. 2002;48(2):177–87. https://0-doi-org.brum.beds.ac.uk/10.1016/s0738-3991(02)00032-0.

    Article  PubMed  Google Scholar 

  6. Lorig KR, Holman H. Self-management education: history, definition, outcomes, and mechanisms. Ann Behav Med. 2003;26(1):1–7. https://0-doi-org.brum.beds.ac.uk/10.1207/S15324796ABM2601_01.

    Article  PubMed  Google Scholar 

  7. Kroon FP, van der Burg LR, Buchbinder R, Osborne RH, Johnston RV, Pitt V. Self-management education programmes for osteoarthritis. Buchbinder R, ed. Cochrane Database Syst Rev. 2014;(1):10–3. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.CD008963.pub2.

  8. Marcolino MS, Oliveira JAQ, D’Agostino M, Ribeiro AL, Alkmim MBM, Novillo-Ortiz D. The impact of mHealth interventions: systematic review of systematic reviews. JMIR mHealth uHealth. 2018;6(1):e23. https://0-doi-org.brum.beds.ac.uk/10.2196/mhealth.8873.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Byambasuren O, Sanders S, Beller E, Glasziou P. Prescribable mHealth apps identified from an overview of systematic reviews. npj Digit Med. 2018;1(1):12. https://0-doi-org.brum.beds.ac.uk/10.1038/s41746-018-0021-9.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Brindal E, Hendrie G, Freyne J, Coombe M, Berkovsky S, Noakes M. Design and pilot results of a mobile phone weight-loss application for women starting a meal replacement programme. J Telemed Telecare. 2013;19(3):166–74. https://0-doi-org.brum.beds.ac.uk/10.1177/1357633X13479702.

    Article  PubMed  Google Scholar 

  11. King AC, Hekler EB, Grieco LA, et al. Harnessing Different Motivational Frames via Mobile Phones to Promote Daily Physical Activity and Reduce Sedentary Behavior in Aging Adults. Lucia A, ed. PLoS One. 2013;8(4):e62613. https://0-doi-org.brum.beds.ac.uk/10.1371/journal.pone.0062613.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Cottrell MA, Galea OA, O’Leary SP, Hill AJ, Russell TG. Real-time telerehabilitation for the treatment of musculoskeletal conditions is effective and comparable to standard practice: a systematic review and meta-analysis. Clin Rehabil. 2017;31(5):625–38. https://0-doi-org.brum.beds.ac.uk/10.1177/0269215516645148.

    Article  PubMed  Google Scholar 

  13. Pelle T, Bevers K, van der Palen J, van den Hoogen FHJ, van den Ende CHM. Development and evaluation of a tailored e-self-management intervention (dr. Bart app) for knee and/or hip osteoarthritis: study protocol. BMC Musculoskelet Disord. 2019;20(1):398. doi:https://0-doi-org.brum.beds.ac.uk/10.1186/s12891-019-2768-9

  14. Pelle T, Bevers K, van der Palen J, van den Hoogen FHJ, van den Ende CHM. Effect of the dr. Bart application on healthcare use and clinical outcomes in people with osteoarthritis of the knee and/or hip in the Netherlands; a randomized controlled trial. Osteoarthr Cartil. 2020;28(4):418–27. https://0-doi-org.brum.beds.ac.uk/10.1016/j.joca.2020.02.831.

    Article  CAS  Google Scholar 

  15. Eysenbach G. The law of attrition. J Med Internet Res. 2005;7(1):e11. https://0-doi-org.brum.beds.ac.uk/10.2196/jmir.7.1.e11.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JEWC. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res. 2012;14(6):1–24. https://0-doi-org.brum.beds.ac.uk/10.2196/jmir.2104.

    Article  Google Scholar 

  17. Manwaring JL, Bryson SW, Goldschmidt AB, Winzelberg AJ, Luce KH, Cunning D, et al. Do adherence variables predict outcome in an online program for the prevention of eating disorders? J Consult Clin Psychol. 2008;76(2):341–6. https://0-doi-org.brum.beds.ac.uk/10.1037/0022-006X.76.2.341.

    Article  PubMed  Google Scholar 

  18. Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res. 2011;13(3):e52. https://0-doi-org.brum.beds.ac.uk/10.2196/jmir.1772.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Hui CY. A mixed method observational study of strategies to promote adoption and usage of an application to support asthma self-management. J Innov Heal Informatics. 2019;25(4):243. https://0-doi-org.brum.beds.ac.uk/10.14236/jhi.v25i4.1056.

    Article  Google Scholar 

  20. Claassen AAOM, Vliet Vlieland TPM, Busch VJJF, Schers HJ, van den Hoogen FHJ, van den Ende CHM. An electronic health tool to prepare for the first orthopedic consultation: use and usability study. JMIR Form Res. 2019;3(4):e13577. https://0-doi-org.brum.beds.ac.uk/10.2196/13577.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Timmers T, Janssen L, Pronk Y, van der Zwaard BC, Koëter S, van Oostveen D, et al. Assessing the efficacy of an educational smartphone or tablet app with subdivided and interactive content to increase patients’ medical knowledge: randomized controlled trial. JMIR mHealth uHealth. 2018;6(12):e10742. https://0-doi-org.brum.beds.ac.uk/10.2196/10742.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Donkin L, Hickie IB, Christensen H, Naismith SL, Neal B, Cockayne NL, et al. Rethinking the dose-response relationship between usage and outcome in an online intervention for depression: randomized controlled trial. J Med Internet Res. 2013;15(10):e231. https://0-doi-org.brum.beds.ac.uk/10.2196/jmir.2771.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Fogg BJ. A behavior model for persuasive design. In: Proceedings of the 4th International Conference on Persuasive Technology - Persuasive ‘09. New York: ACM Press; 2009. p. 1. https://0-doi-org.brum.beds.ac.uk/10.1145/1541948.1541999.

    Chapter  Google Scholar 

  24. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Hum Comput Interact. 2008;24(6):574–94. https://0-doi-org.brum.beds.ac.uk/10.1080/10447310802205776.

    Article  Google Scholar 

  25. Brooke J. SUS: a “quick and dirty” usability scale. In: Usability Evaluation in Industry; 1996. p. 189–95.

    Google Scholar 

  26. de Groot IB, Reijman M, Terwee CB, Bierma-Zeinstra SMA, Favejee M, Roos EM, et al. Validation of the Dutch version of the hip disability and osteoarthritis outcome score. Osteoarthr Cartil. 2007;15(1):104–9. https://0-doi-org.brum.beds.ac.uk/10.1016/j.joca.2006.06.014.

    Article  Google Scholar 

  27. de Groot IB, Favejee MM, Reijman M, Verhaar JA, Terwee CB. The Dutch version of the knee injury and osteoarthritis outcome score: a validation study. Health Qual Life Outcomes. 2008;6(1):16. https://0-doi-org.brum.beds.ac.uk/10.1186/1477-7525-6-16.

    Article  PubMed  PubMed Central  Google Scholar 

  28. EuroQol Group. EuroQol--a new facility for the measurement of health-related quality of life. Health Policy. 1990;16(3):199–208. https://0-doi-org.brum.beds.ac.uk/10.1016/0168-8510(90)90421-9.

    Article  Google Scholar 

  29. Wagenmakers R, van den Akker-Scheek I, Groothoff JW, et al. Reliability and validity of the short questionnaire to assess health-enhancing physical activity (SQUASH) in patients after total hip arthroplasty. BMC Musculoskelet Disord. 2008;9(1):141. https://0-doi-org.brum.beds.ac.uk/10.1186/1471-2474-9-141.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Skolasky RL, Green AF, Scharfstein D, Boult C, Reider L, Wegener ST. Psychometric properties of the patient activation measure among multimorbid older adults. Health Serv Res. 2011;46(2):457–78. https://0-doi-org.brum.beds.ac.uk/10.1111/j.1475-6773.2010.01210.x.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Rademakers J, Nijman J, van der Hoek L, Heijmans M, Rijken M. Measuring patient activation in the Netherlands: translation and validation of the American short form patient activation measure (PAM13). BMC Public Health. 2012;12(1):577. https://0-doi-org.brum.beds.ac.uk/10.1186/1471-2458-12-577.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Broadbent E, Petrie KJ, Main J, Weinman J. The brief illness perception questionnaire. J Psychosom Res. 2006;60(6):631–7. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jpsychores.2005.10.020.

    Article  PubMed  Google Scholar 

  33. Selten EMH, Vriezekolk JE, Schers HJ, Nijhof MW, van der Laan WH, van der Meulen-Dilling RG, et al. Development of the “treatment beliefs in knee and hip OsteoArthritis (TOA)” questionnaire. BMC Musculoskelet Disord. 2017;18(1):402. https://0-doi-org.brum.beds.ac.uk/10.1186/s12891-017-1762-3.

    Article  PubMed  PubMed Central  Google Scholar 

  34. StataCorp. Stata statistical software: release 13. College Station: StataCorp LP; 2013.

    Google Scholar 

  35. Yamaguchi S, Waki K, Nannya Y, Nangaku M, Kadowaki T, Ohe K. Usage patterns of GlucoNote, a self-management smartphone app, based on ResearchKit for patients with type 2 diabetes and Prediabetes. JMIR mHealth uHealth. 2019;7(4):e13204. https://0-doi-org.brum.beds.ac.uk/10.2196/13204.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Hui CY, Walton R, McKinstry B, Jackson T, Parker R, Pinnock H. The use of mobile applications to support self-management for people with asthma: a systematic review of controlled studies to identify features associated with clinical effectiveness and adherence. J Am Med Inform Assoc. 2017;24(3):619–32. https://0-doi-org.brum.beds.ac.uk/10.1093/jamia/ocw143.

    Article  PubMed  Google Scholar 

  37. Murphy LA, Harrington P, Taylor SJ, et al. Clinical-effectiveness of self-management interventions in chronic obstructive pulmonary disease: an overview of reviews. Chron Respir Dis. 2017;14(3):276–88. https://0-doi-org.brum.beds.ac.uk/10.1177/1479972316687208.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Short CE, DeSmet A, Woods C, Williams SL, Maher C, Middelweerd A, et al. Measuring engagement in eHealth and mHealth behavior change interventions: viewpoint of methodologies. J Med Internet Res. 2018;20(11):e292. https://0-doi-org.brum.beds.ac.uk/10.2196/jmir.9397.

    Article  PubMed  PubMed Central  Google Scholar 

  39. de Vries HJ, Kloek CJJ, de Bakker DH, Dekker J, Bossen D, Veenhof C. Determinants of adherence to the online component of a blended intervention for patients with hip and/or knee osteoarthritis: a mixed methods study embedded in the e-exercise trial. Telemed J E Health. 2017;23(12):1002–10. https://0-doi-org.brum.beds.ac.uk/10.1089/tmj.2016.0264.

    Article  PubMed  Google Scholar 

  40. Jansen T, Rademakers J, Waverijn G, Verheij R, Osborne R, Heijmans M. The role of health literacy in explaining the association between educational attainment and the use of out-of-hours primary care services in chronically ill people: a survey study. BMC Health Serv Res. 2018;18(1):394. https://0-doi-org.brum.beds.ac.uk/10.1186/s12913-018-3197-4.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Claassen AAOM, Kremers-van de Hei KCALC, van den Hoogen FHJ, et al. Most important frequently asked questions from patients with hip or knee osteoarthritis: a best-worst scaling exercise. Arthritis Care Res (Hoboken). 2019;71(7):885–92. https://0-doi-org.brum.beds.ac.uk/10.1002/acr.23719.

    Article  Google Scholar 

  42. Scott AR, Alore EA, Naik AD, Berger DH, Suliburk JW. Mixed-methods analysis of factors impacting use of a postoperative mHealth app. JMIR mHealth uHealth. 2017;5(2):e11. https://0-doi-org.brum.beds.ac.uk/10.2196/mhealth.6728.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Broekhuis M, van Velsen L, Hermens H. Assessing usability of eHealth technology: a comparison of usability benchmarking instruments. Int J Med Inform. 2019;128:24–31. https://0-doi-org.brum.beds.ac.uk/10.1016/j.ijmedinf.2019.05.001.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

A special thanks to all participants for their contribution in this study.

We would like to thank Symax B.V. who helped to develop the dr. Bart app.

Funding

This project is funded within the INTERREG-programme and receives financial support by the European Union, the Ministry of Economic Affairs, Innovation, Digitalisation and Energy of the State of North Rhine-Westphalia, the Ministry of Economic Affairs and Climate Policy of the Netherlands and the Dutch Provinces of Gelderland and Limburg. The funding body had and will not have any role in the design of the study and collection, analysis, and interpretation of the data and writing and submitting manuscripts.

Author information

Authors and Affiliations

  1. Department of Rheumatology, Sint Maartenskliniek, Nijmegen, the Netherlands

    Tim Pelle, Frank H. J. van den Hoogen, Karen Bevers & Cornelia H. M. van den Ende

  2. Department of Rheumatic Diseases, Radboud University Medical Center, PO Box 9011, 6500, GM, Nijmegen, the Netherlands

    Tim Pelle, Frank H. J. van den Hoogen & Cornelia H. M. van den Ende

  3. Department of Research Methodology, Measurement, and Data-Analysis, Behavioural, Management and Social Sciences, University of Twente, Enschede, The Netherlands

    Job van der Palen

  4. Medical School Twente, Medisch Spectrum Twente, Enschede, the Netherlands

    Job van der Palen

  5. Orikami B.V, Nijmegen, the Netherlands

    Frank de Graaf

Authors
  1. Tim Pelle
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Job van der Palen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Frank de Graaf
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Frank H. J. van den Hoogen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Karen Bevers
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Cornelia H. M. van den Ende
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

TP, KB, JvdP, FHJvdH and CHMvdE participated in the design of the study. TP was responsible for inclusion and data collection. TP, JvdP, FdG, and CHMvdE were responsible for the data analysis, tables and figures. All authors were responsible for the interpretation of the data. TP and CHME were responsible for drafting the manuscript. All other authors critically reviewed the manuscript. Furthermore, all authors approved the final version of the manuscript.

Corresponding author

Correspondence to Tim Pelle.

Ethics declarations

Ethics approval and consent to participate

Ethical approval for this study was waived by the local Medical Research Ethics Committee of the Radboud University Medical Centre, Nijmegen, the Netherlands (protocol number: 2017–3625). The study fell outside the remit of the law for Medical Research Involving Human Subjects Act and was approved by the local ethical committee. This study is registered in the Dutch trial register (trial number: NTR 6693) as of 21 September 2017. All participants gave digital (written) consent to participate in the present study.

Consent for publication

Not applicable.

Competing interests

Frank de Graaf receives salary from Orikami B.V., the organisation that helped to develop the dr. Bart app. All other authors declare they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Appendix 1.

Screenshots of the dr. Bart app. Appendix 2. Baseline characteristics of participants in the study per indicator of use and non-users. Appendix 3. The 5 most and least often read paragraphs. Appendix 4. Median and interquartile range of the number of cumulative completed goals over time of the active users (n = 151). Appendix 5. Median and interquartile range of the number of cumulative logins over time of the active users (n = 151). Appendix 6. The 5 relatively most and least often completed goals (i.e., times completed / times chosen). Appendix 7. The 5 relatively most and least often chosen goals (i.e., times chosen / times proposed). Appendix 8. Some responses to the free-text option of the SUS. Appendix 9. Regression coefficient and 95% confidence interval of the relation between baseline characteristics and different parameters of use. Appendix 10. Spearman rank correlation coefficients between different indicators of use and beliefs regarding 5 treatment modalities in knee/hip OA as measured with the treatment beliefs in osteoarthritis questionnaire (TOA) (relative difference between baseline and six month follow-up).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pelle, T., van der Palen, J., de Graaf, F. et al. Use and usability of the dr. Bart app and its relation with health care utilisation and clinical outcomes in people with knee and/or hip osteoarthritis. BMC Health Serv Res 21, 444 (2021). https://0-doi-org.brum.beds.ac.uk/10.1186/s12913-021-06440-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12913-021-06440-1

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us