Estimating the effectiveness of self-help groups on the adoption of secondary preventive measures by people living with HIV in Central America, 2012

Abstract

Background

According to UNAIDS, the HIV epidemic has stabilized. This as a result of increased condom use and greater access to coverage for antiretroviral therapy (ART). In Central America, civil society organizations work with self-help groups (SHGs) organized in conjunction with public health services to implement interventions seeking to increase condom use and ART adherence for people living with HIV (PLH).

Method

To analyze the effectiveness of SHGs in Central America aimed on increasing condom use and ART adherence in PLH, We conducted a cross-sectional study using a questionnaire and a random sample of 3024 intervention group and 1166 control group. Based on propensity scoring and one-to-one matching (with replacement), we formed a comparison group to help estimate the effectiveness of the above-mentioned intervention on two outcome variables (condom use and ART adherence). The internal consistency of the results was tested through weighted least squares (WLS) and instrumental variable (IV) regression.

Results

Although bivariate comparisons yielded differences between intervention and control group, we found no evidence that the intervention was effective; nor did we find evidence of a heterogeneous impact among countries after adjusting for propensity scoring and the IV model. The impact observed after performing raw comparisons of the indicators may be attributable to self-selection on the part of PLH rather than to the SHGs strategy. Our results demonstrate that it is imperative to use rigorous intervention evaluation methodology to validate the consistency of results.

Conclusions

The intervention had no impact on the outcome indicators measured. We recommend prioritizing the allocation of economic resources for the implementation of interventions with previously proven effectiveness. We also recommend that future studies explore why the intervention failed to produce the expected impact on condom use and ART adherence.

Background

According to The Joint United Nations Programme on HIV/AIDS (UNAIDS), the HIV epidemic has stabilized in recent years [1]. This news has been circulating in UNAIDS documents over the past 5 years [1,2,3]; the main reason adduced for this achievement is increased access to Antiretroviral Therapy (ART) and the adoption of preventive measures - primarily safe sex practices - redounding in fewer new infections [3].

However, increased access to ART will be reflected in improved incidence and mortality indicators only if people living with HIV (PLH) achieve therapeutic adherence. To encourage ART adherence, the World Health Organization (WHO) has recommended the implementation of self-help groups (SHGs) aimed at offering PLH medical follow-up and psychological support [4].

SHGs, often initiated and sponsored by community and civil society organizations (CSOs), generally gather in health units, CSOs installations or other community spaces [5]. Those linked to health units and CSOs are tasked with disseminating information on secondary HIV prevention, strengthening the self-esteem and social skills of PLH, reducing HIV-related stigma and discrimination, promoting the retention of PLH by health services and increasing ART adherence [6, 7].

In a meta-analysis of studies on the effectiveness of SHGs from 1995 to 2014, Bateganya et al. (2015) found a number of benefits including greater PLH retention by health services, improved quality of life and lower rates of HIV-related morbidity, mortality and transmission; nonetheless, the authors concluded that evidence was insufficient to draw decisive conclusions on the subject [5].

In Central America, CSOs have been striving to find a response to HIV for more than a decade. International agencies have provided technical and financial cooperation support to them for implementing strategies to reduce the damage and improve the quality of life of PLH. Deserving special mention is an intervention titled Developing and strengthening the technical and professional capacities of PLHA (Acronym used in this proposal to designate people living with HIV/AIDS) in order that they may effectively contribute to the HIV/AIDS response, quality of life advancement and poverty reduction in the region, 2008–2012 [8]. Its objectives included increasing condom use and ART adherence among PLH in all seven Central American countries. Henceforth, we will refer to this initiative as “the intervention.”

For the implementation of the intervention in Central America, CSOs have relied on SHGs for PLH, most of them located within or close to health units. Considering that evidence on the effectiveness of HIV/AIDS initiatives based on SHGs is virtually non-existent, particularly in Central America, we decided to evaluate the effectiveness of the intervention. We were interested in ascertaining its potential for increasing secondary preventive measures among PLH [9], specifically condom use at last sexual encounter and ART adherence.

Methods

This is a secondary data analysis of the cross-sectional survey Elaboration of the risk profile of people living with HIV in the beneficiary countries of the REDCA+ Regional Program [10]. The survey was administered to PLH in the seven Central American countries between June and October of 2012. Its aim was to explore the status of this population in terms of socio-demographic characteristics, prevalence of condom use, ART adherence, sexual practices, predominance of HIV-related stigma and discrimination and AIDS-related morbidity. The questionnaire was designed based on the USAID’s Guidelines for Repeated Behavior Surveys in Populations at HIV Risk (see additional files section). Our analysis centered on socio-demographic characteristics, condom use and ART adherence.

For the survey results to be nationally representative, the seven countries were regarded as strata with similar sample sizes. The SHGs were defined as clusters, 103 SHGs were selected from the lists provided by the health authorities of participating countries. The sample size by country was proportional to the number of clusters. It was calculated taking into account that the parameter to be estimated was the proportion of people diagnosed with HIV who had used a condom during their last sexual encounter, which was assumed to be between 40 and 70% regardless of the country. The sampling strategy resulted in a design effect (DEFF) of 1.74.

SHGs were randomly selected and visited; every third individual who arrived at the sessions was interviewed. Based on the records of local health services and references from users PLH who were non-SHGs users were identified and visited at them home where pollsters applied the questionnaires. Data collection in each country lasted 66 days on average, that is, until reaching the required size. The sample collected was 4940 individuals. Unable to make home interviews in Nicaragua, this country was excluded from our study. Final data analyzed pertained to 4190 individuals: 3024 interviewed at 88 SHGs and 1166 at home.

Following data collection, the effectiveness of the intervention was evaluated. Our objective was to assess whether it had contributed to increased condom use and ART adherence among PLH in six Central American countries: El Salvador, Belize, Costa Rica, Guatemala, Honduras and Panama.

The intervention was developed according to a Behavior Change Communication strategy known as Because I am capable, I act (the original version was registered by the LLAVES Foundation in Pedro Sula, Honduras, under the following title: Initiative Based on the Integrated Behavior Development and/or Change Communication Strategy). The CSOs in charge of implementation at the regional level trained PLH - designated as “focal points” - to offer peer counseling to PLH attending SHGs.

For the purposes of our study, the intervention and control groups was defined as follows:

• Intervention group were PLH users of SHGs who were interviewed at the sites of their sessions, and

• Control group (those who did not form part the intervention) were PLH who did not attend SHGs and were interviewed at home.

Intervention group received medical care and follow-up as well as ART and behavior change counseling under the Because I am capable, I act model, with the latter provided permanently by trained personnel. SHGs attendance by participants was estimated at one in-person session per month lasting from 60 to 120 min.

In-person sessions included games designed to promote behavior changes, safe-sex practices and ART adherence. Time was provided at the end of each session for group members to discuss their experiences. For the most part, participants attended sessions following their monthly medical consultation, with the number of attendees ranging from five to ten. During the intervention, which lasted 4 years, PLH were offered medical follow-up and ART at their health units and behavior change counseling in their SHGs.

Because of its potential benefits, the intervention was made available for the entire PLH population attending SHGs, not was a randomly assigned individuals. It is therefore important to consider the presence of a self-selection bias in our research and, hence, the need to isolate factors extraneous to the intervention which might have affected its expected results [11], namely individual characteristics which may have rendered PLH un/likely to self-select themselves for intervention.

Given that the intervention was not randomized, its estimated effects on our outcome variables may have been affected by confounding factors. Nonetheless, in order to minimize bias from observable characteristics, we used propensity score (PS) matching [12,13,14]. In other words, by setting up a control group (PLH who did not attend SHGs sessions), we were able to contrast our group of participants (intervention) against a subgroup of non-participants (control) exhibiting highly similar covariables or a comparable group.

Insofar as the people in the intervention and control groups pertained to a population of similar individuals as regards eligibility to receive intervention, PS matching proved a valid method for establishing a control group with comparable characteristics. This facilitated our evaluation of the effectiveness of the intervention [12,13,14]. In order to ensure consistency, we verified that our matching procedure met the required criteria of conditional independence and region of common support [13].

We assumed that the following factors may have influenced the inclusion or likelihood of participating in the intervention: individual characteristics such as demographic profiles [15]; availability and accessibility of medical services [16], depending on the country of residence; a propensity to engage in risky behavior; a history of drug use [17,18,19]; and resilience factors related to adaption to be living with HIV [20].

The characteristics used in PS estimation were the following:

• (1) Country of residence: El Salvador, Belize, Costa Rica, Guatemala, Honduras and Panama. We included this information because SHG availability can differ by country;

• (2) Age: 18–24, 25–44, 45–59 and ≥ 60 years. We grouped participants by generational stage;

• (3) Schooling: none, literate, 6, 12 or ≥ 16 years of schooling;

• (4) Sex: male or female;

• (5) Sexual orientation: heterosexual, bisexual or homosexual. We included these categories because it has been reported that homosexuals and bisexuals exhibit a higher rate of SHGs attendance than do heterosexuals [21];

• (6) Currently in a stable relationship: yes or no. This characteristic referred to individuals with whom interviewees had sex without payment and maintained an affectionate/constant/regular bond, for example, a partner, boyfriend/girlfriend or spouse, even if not living in the same house. Couple counseling has been recommended by UNAIDS as an effective strategy [1] which may correlate positively with SHGs use;

• (7) Average monthly income in US dollars: lack of money has been reported as an obstacle to SHGs use [15, 22]. As 13.6% (n = 572) of interviewees did not disclose their monthly income, we constructed a zero-inflated Poisson regression model to estimate the missing data on the basis of sex, age, schooling, type of employment and country of residence. With the values obtained we were able to estimate and assign monthly income figures for 86% of the missing values. A 0.65 correlation was obtained between the values predicted by the model and those reported;

• (8) Family support for dealing with HIV-related disease: yes or no. This item was considered an indicator for family and social support which, in turn, has been reported as a promoter of health-care use [23, 24];

• (9) Having economic dependents: yes or no. We included this item as an indicator for family structure [23, 24], assuming that PLH who were responsible for economic dependents were more likely to take better care of their health and participate in SHGs;

• (10) Prior diagnosis of tuberculosis: yes or no. It has been documented that PLH attending SHGs are referred by health personnel working with HIV-related diseases [25];

• (11) Having received counseling when tested for HIV: yes or no. In the event of a positive result, laboratory staff are required to refer individuals to health services and SHGs [5, 22];

• (12) History of drug use: yes or no. Use of drugs has been associated with reduced SHGs attendance [1]; and

• (13) Future Time Perspective (FTP): with values ranging from zero to ten, the FTP scale was used to estimate the resilience of PLH in confronting their condition. FTP is considered a protective factor that promotes resilience, especially among the most vulnerable groups [26]. A negative correlation has been shown to exist between FTP and high-risk sexual practices [27]. People with a highly developed FTP are expected to make better present decisions that will allow them to achieve their future plans. FTP was measured using the Consideration of Future Consequences Scale proposed by Strathman et al. [28] and validated in 2003 by Petrocelli [29].

The Average Treatment Effect on the Treated (ATT) [12] was estimated as a measure of the impact of intervention; that is, the average effect of the intervention solely among intervention group as this was the only part of the sample selected randomly. The ATT was estimated using two indicators: (1) use of a condom at last sexual encounter and (2) ART suspension by own admission, as a means of estimating therapeutic adherence in cases where therapy was not suspended. The two outcome variables were measured through the following questions:

• (1) Condom use: Section 11 of the questionnaire inquired about the last sexual encounter. Participants were asked to recall their last sexual encounter and the type of partner involved (stable, client, occasional, commercial or stranger). They were then asked, “Did you use a condom the last time you had sexual relations?”

• (2) Adherence to ART: Section 05 of the questionnaire explored whether participants were currently attending ART, and how long ago they had begun (71% had been attending for less than 5 years). They were then asked if they had suspended ART: “Since you began attending ART, have you ever suspended treatment by your own choice? That is, by your own will and not for reasons of medical prescription?”

The robustness and consistency of the ATT results were tested using four matched algorithms: second nearest neighbor (without specifying caliper value), Kernel (epanechnikov type), one-to-one match (one nearest neighbor) and Weighted Least Squares (WLS) regression. The weights were estimated as follows: value 1 for those who received the intervention and for those who are in the control group the assigned value was 1 / (1-pscore). We used other models to analyze the countries separately (one at a time) in order to identify any heterogeneous effects of the intervention on the two outcome variables.

With a view to reducing the bias generated by omission of explanatory variables, we tested the consistency of the results through a two-stage WLS method incorporating an instrumental variable (IV) which was tested for overidentification, endogeneity and weakness [30].

In order to use this variable as an instrument in IV modeling, we performed a geospatial analysis considering the distance between the places of residence of the PLH and the SHGs as a variable which may have influenced SHGs attendance. The IV employed was the squared Euclidean (straight-line) distance in kilometers between the geographic location of the PLH (locality) and the geographic location of the SHGs [31]. We used the Google Earth Pro 7.7.1 Program to identify the geographic location and georeferenciation of the localities in relation to the SHGs. We also used Qgis 2.18 software to estimate the distances: 578 localities and 89 SHGs were georeferenced, resulting in an estimated 864 distances of interest for the study. For impact estimates, we used the Stata v.13.0 statistical package.

Results

Propensity-score (PS) estimation

We estimated the PS of an initial sample of 4190 observations by adjusting a probit model: 1166 observations pertained to control group and 3024 to intervention group. Based on propensity scoring and one-to-one matching (with replacement), we formed a comparison group. As a result, we identified 1032 PLH in control group and 2629 PLH in intervention group on the region of common support, representing an adequate sample size for both groups. Figure 1 shows the histogram of the estimated PS including the region of common support, while Figs. 2 and 3 show sample distribution before and after matching.

Fig. 1

Histogram of the propensity score (region of common support) estimated for the non/participant matching process

Fig. 2

Distribution of observable characteristics in the non/participant groups prior to matching

Fig. 3

Distribution of observable characteristics in the non/participant groups after matching

We analyzed PS and matching using the Kernel method to explore the reduction of the differences and the bias of the variables following matching; the general model yielded a bias reduction from 15.6 to 6.2% (p < 0.001). Table 1 details the information for each variable.

Table 1 Bias reduction tests before and after matching based on the Kernel algorithm

Socio-demographic characteristics, sexual behavior and health

The sample analyzed was composed largely of young adults (25–44 years old) with 6–12 years of schooling and an average monthly income of 193 US dollars. In the majority of cases, they were responsible for economic dependents. Six out of ten mentioned having no stable partner at the moment of the interview.

Eight out of ten interviewees reported having received counseling when they learned they were seropositive, and only 12% of interviewees mentioned having a history of pulmonary tuberculosis. The majority related that they could count on their families for care in the event of hospitalization. Table 2 details population characteristics before and after sample matching.

Table 2 General characteristics of the analyzed sample

Impact of intervention on condom use at last sex

In analyzing the average impact of the intervention on condom use at last sex among those being treated (ATT), no significant impact emerged from any of the four methods. Kernel matching yielded an estimator of 4.4 positive percentage points (95% CI -0.043/0.050) and WLS one of 4.3 negative percentage points (95% CI -0.091/0.004). The other methods generated estimators in the range of − 0.006 to 0.073, as shown in Fig. 4. No heterogeneous effects were observed as a result of the intervention in the sampled countries (Table 3).

Fig. 4

Effectiveness of the intervention according to outcome indicators

Table 3 Average effect of treatment for those treated (ATT) on the outcomes measured by country

The IV method generated the most imprecise estimator with a 95% CI of 1.296 to 0.792. This method yielded a significant squared Euclidean distance association between the geographic location of the place of residence of the individual and the geographic location of the SHG attended, confirming that PLH used nearby SHGs less than distant ones. This made it possible to verify the validity of the instrument employed (Table 4).

Table 4 Ordinary Least Squares model weighted in two stages with instrumental variables incorporated for condom use

Impact of the intervention on ART adherence

Voluntary suspension of ART, which was employed as both a proxy for measuring ART adherence and the second outcome variable, did not yield statistically significant estimators. Kernel matching produced an estimated impact of 3.6 percentage points (95% CI-0.006/0.079), while the WLS yielded an estimated impact of 3.5 percentage points (95% CI-0.008/0.078). The other methods produced results ranging from − 0.007 to 0.072 (Fig. 4). No heterogeneous effects from the intervention were identified in the sampled countries (Table 3).

When modeling the outcome indicator using an IV, we also found a significant association of the instrument employed. The values resulting from the tests performed on the IV model for adherence (squared Euclidean distance) are shown in Table 5. The consistency of the parameters for both outcome indicators was tested by performing 5000 bootstrap replications in ATT estimation (Table 6).

Table 5 Ordinary Least Squares model weighted in two stages with instrumental variables incorporated for therapeutic adherence

Table 6 Average effect of treatment for those treated (ATT)

Because temporal measurement of adherence is imprecise, we performed a second analysis. In this case, we used solely the subsample of PLH who had attended ART for less than 5 years (2027), the approximate duration of the intervention. No statistical difference was observed in the estimates for intervention vs. control group. The ATTK obtained through Kernel matching was 0.024 with a p value of 0.434, while that obtained through nearest neighbor matching was 0.003 with a p value of 0.925.

Discussion

Our results indicated that the intervention under study did not exert an impact on the outcome indicators measured. This finding contradicts the results that would have ensued from a simple comparative analysis of indicators; such an analysis would have yielded positive impacts of at least five percentage points on condom use and seven percentage points on ART adherence.

Because SHGs participation was voluntary and therefore not random, the results above may be explained more in terms of an association with individual characteristics such as FTP and self-selection for intervention than in terms of a direct impact from the strategy under evaluation. In other words, it is possible that those who decided to participate in the intervention had greater interest in caring for their health and were therefore more inclined to use condoms and adhere to ART than those who refrained from participating.

Our results differ from those presented in a study published by Lung Vu et al. in 2015 [34]. This study found that interventions based on the Behavior Change Communication (BCC) model were effective in increasing condom use up to 2.4 times. This degree of effectiveness may be a result of having employed only one matching methodology and not considering the importance of the specific PLH involved as well as other variables including educational and income levels when forming the control group. It is thus possible that self-selection bias was overlooked by these authors, something we took into account in formulating the methodological approach for our study.

The main limitation of our study was that no baseline measurement was taken prior to the intervention, with the only data available dating from a one time-point measures taken in 2012, 4 years after initiating the intervention. It is for this reason that we used several complementary methods for estimating indicators as a way of reducing biases related to the lack of an experimental design, the absence of a baseline measurement and the possibility of self-selection. In this respect, we controlled extensively for observable characteristics that, in theory, could have influenced self-selection. We also used methodology appropriate for cross-sectional studies, achieving a substantial reduction by forming a control group. In all cases, we obtained consistent estimators but observed no significant impact from the intervention under any of the methods utilized. Neither did we find any heterogeneous effects upon analyzing the countries separately.

Although the variable employed as an instrument – Euclidean distance – was weak, it proved significant in both cases. This means that PLH do not always use the nearest health services but are willing to travel a bit farther to receive services [31]. Other studies have yielded similar results, explaining them by reference to the desire of PLH to preserve their anonymity, obtain specialized services, guarantee the availability of medications or receive care in top-level facilities [32, 35, 36]. According to Cook et al. [37], more than half of the population do not use the nearest health facility, and people at higher socioeconomic levels tend to travel greater distances to seek care.

Civil society organizations (CSOs) play a predominant role in the response to HIV. It is important to consider that the intervention analyzed had objectives beyond those evaluated in this study, namely the participation of PLH in the regional response to HIV/AIDS, social mobilization and the enforcement of the rights and responsibilities of PLH (objectives extracted from the funding proposal document). It would be useful to evaluate these objectives in order to determine which ones have achieved positive results and support their continuity with scientific evidence.

UNAIDS suggests that reducing the number of new HIV infections by 2020 will require investing at least a quarter of the total HIV budget in prevention, and has stressed the importance of allocating funds only for interventions of proven effectiveness [38]. As a consequence, international financial organizations should support shifting financing to proposals for interventions whose effectiveness has been scientifically proven.

Given potential bias from omitted (not observable) variables, we used an IV model to analyze the possible impact of intervention by controlling for geographic distance. Even if Euclidean distance does not consider environmental factors such as means of communication and transportation, it is nonetheless a valid way of identifying general differences resulting from geographic proximity to health units [31].

It is likely that the outcome variable, therapeutic adherence, represents a measurement error as a result of self-reporting. However, this would not be expected to constitute a differential in the intervention and control groups, as participating was not conditioned on a response to this variable. Moreover, evidence indicates that self-reporting overestimates real adherence by as much as 25% [33]; if a significant impact had been found, appropriate corrections could have been made. Furthermore, we recognize the problems of temporality associated with measuring this variable; for this reason we undertook an additional analysis with a subsample of PLH who had been receiving ART less than 5 years, and still found no significant impact.

Finally, we recognize that lack of a baseline measurement rendered it impossible to determine either the parameter of the effectiveness indicator prior to the intervention or what occurred between 2008 and 2012. As a consequence, we only analyzed the final results observed in 2012.

Conclusions

Our study results contribute empirical evidence regarding the effectiveness of an HIV/AIDS intervention undertaken in Central America, while highlighting the importance of employing a strict methodology for evaluating the impact of interventions. Little published evidence exists concerning evaluations of interventions of this type in the region.

The intervention under study did not exert an impact on the outcome indicators measured. We suggest evaluating the effectiveness of the interventions already implemented or that international agencies grant financing to implement interventions that have already been proven effective. We also recommend that future studies explore the reasons why the intervention failed to produce the expected impact on condom use and ART adherence.