Table 3 Inter-rater reliability of AGREE instrument domain scores
Scope & Purpose | Stakeholder Involvement | Rigor of Development | Clarity & Presentation | Applicability | Editorial Independence | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
Single Rater ICC (95% CI) | Average of Raters ICC- all 6 raters or 3 pairs of 2 raters? (95% CI) | Single Rater ICC (95% CI) | Average of Raters ICC (95% CI) | Single Rater ICC (95% CI) | Average of Raters ICC (95% CI) | Single Rater ICC (95% CI) | Average of Raters ICC (95% CI) | Single Rater ICC (95% CI) | Average of Raters ICC (95% CI) | Single Rater ICC (95% CI) | Average of Raters ICC (95% CI) | |
Generalists | 0.37 (0.11–0.59) | 0.54 (0.19–0.74) | 0.71 (0.54–0.83) | 0.83 (0.70–0.90) | 0.81 (0.68–0.89) | 0.89 (0.81–0.94) | 0.41 (0.15–0.62) | 0.58 (0.26–0.76) | 0.65 (0.46–0.79) | 0.79 (0.63–0.88) | 0.60 (0.39–0.75) | 0.75 (0.56–0.86) |
Clinical Specialists | 0.35 (0.10–0.56) | 0.52 (0.18–0.72) | 0.59 (0.39–0.74) | 0.74 (0.56–0.85) | 0.65 (0.47–0.78) | 0.79 (0.64–0.88) | 0.51 (0.29–0.69) | 0.68 (0.45–0.81) | 0.43 (0.19–0.63) | 0.61 (0.32–0.77) | 0.32 (0.06–0.54) | 0.49 (0.11–0.70) |
Researchers | 0.17 (0.14–0.46) | 0.30 (-0.32–0.63) | 0.73 (0.54–0.85) | 0.84 (0.70–0.92) | 0.77 (0.61–0.87) | 0.87 (0.76–0.93) | 0.47 (0.18–0.69) | 0.64 (0.31–0.81) | 0.47 (0.19–0.68) | 0.64 (0.32–0.81) | 0.59 (0.34–0.77) | 0.75 (0.51–0.87) |
Overall | 0.40 (0.25–0.58) | 0.80 (0.66–0.89) | 0.67 (0.54–0.80) | 0.93 (0.87–0.96) | 0.79 (0.68–0.88) | 0.96 (0.93–0.98) | 0.55 (0.40–0.72) | 0.88 (0.80–0.94) | 0.50 (0.35–0.68) | 0.86 (0.76–0.93) | 0.35 (0.19–0.54) | 0.76 (0.59–0.88) |