Skip to main content
  • Research article
  • Open access
  • Published:

Assessing the sustainability of the Nigerian urban reproductive health initiative facility-level programming: longitudinal analysis of service quality

Abstract

Background

To date, there is little information on the sustainability of family planning (FP) service quality after completion of a donor-funded program. This paper examines the sustainability of the Nigerian Urban Reproductive Health Initiative (NURHI) program on quality of FP services in two cities: Ilorin, where the program ended in March 2015 and Kaduna where the program continued.

Methods

Data come from three time periods: 2011, before program implementation; 2014, near Phase 1 completion; and 2017, two-years post Phase 1. In 2011, we undertook a facility audit and provider surveys in all public sector facilities in each city as well as all private facilities mentioned as the source for FP or maternal, newborn, and child health services in a 2010 women’s household survey. In 2014 and 2017, we returned to the same facilities to undertake the facility audit and provider surveys. Quality is measured from principal component analyses of 30 items from the facility audit and provider surveys. Service use outcomes are measured as the ratio of FP clients (total and new) to the number of reproductive health staff members. Multivariate random effect models are estimated to examine changes in the outcomes over time, between NURHI and non-NURHI facilities and by city.

Results

We demonstrate that NURHI facilities had better quality and higher service use than non-NURHI facilities. Further, while quality of services was higher in Ilorin in 2011, by 2014 and three years later (2017), the quality was better in Kaduna where the program continued. In addition, while no difference was found in service utilization between Ilorin and Kaduna in 2014, by 2017, Kaduna had significantly more new FP users than Ilorin.

Conclusions

In Ilorin, quality of services did not continue its strong upward trend after the program ended. Programs need to consider long-term strategies that support continuation of program components post program implementation. This may include ensuring continued training of providers and addressing equipment and commodity stock-outs through system changes rather than specific facility-level changes. The findings from this study can be used to inform future programs seeking to improve quality of FP services in a sustainable manner.

Peer Review reports

Background

Increasing modern contraceptive use requires providing quality family planning services, where quality is fairly broadly defined [1, 2]. For instance, per the Family Planning 2020 Initiative, quality includes, among other things, access to a full range of methods, providing clear and medically accurate information to clients, and having providers who are well equipped, technically competent and respectful [3].

To date, much of the literature on quality of care in family planning services in sub-Saharan Africa has focused on cross-sectional or descriptive studies demonstrating the level of quality at a certain point in time and/or comparing quality between the public and private sectors [4,5,6,7]. One recent exception by Winston and colleagues [8] examines if program components that sought to increase the quality of services led to increases in service use. To do this the authors use longitudinal facility data from urban areas in Kenya, Nigeria and Senegal. The authors demonstrate that facilities with bigger improvements in service quality had increased service use in the follow-up period [8]. No studies were found that examine whether strategies to improve family planning quality of care sustain following the conclusion of program implementation.

What happens to service quality once a program ends or funding is terminated remains a critical open question. Recently there has been a lively discussion in the literature on the importance of measuring the sustainability of program activities after initial or intensive funding ends [9,10,11]. Sustainability in these studies focuses on the continuation of program activities or the institutionalization of a program in the long-term [9, 10]. Four main reasons have been proposed to strive for sustainability: to maintain program effects; to capture effects that may only appear following a certain period of time; to reap the benefits of investments; and to prevent the disillusionment of community members and clients, as can happen when programs are discontinued [10]. Reviews of program sustainability, mostly from developed countries, have demonstrated that programs that are more sustainable have certain common characteristics including: a) the program is modifiable; b) there is a program champion to keep the program going; c) there is a good fit between the program and the organization implementing it; d) the benefits of the program are recognized by those implementing it; and e) community-level stakeholders endorse the program [9]. In more recent reviews of program sustainability across program areas and contexts, similar factors that affect whether programs sustain have been identified including: a) characteristics related to the innovation itself (such as the fit, ability to be modified, and its effectiveness or perceived benefit); the organizational context where the program is implemented (climate, culture, and leadership); capacity to continue the program (internal, such as the presence of a champion, and external, such as community involvement); and processes (such as training, collaboration and partnership, and relationship building) [9, 12, 13]. That said, there is evidence that the effect of these factors on sustainability varies by program area [12].

To date, only a few studies focusing on sexual and reproductive health in developing countries have examined program sustainability at the beneficiary or the program level. Studies from Asia on maternal and child health have shown sustainability of use of skilled birth attendants by women [14, 15] and two studies from India examining a family planning program demonstrated continued family planning awareness and use among women exposed to the program after program implementation concluded [16, 17]. More recently, a study from Nigeria demonstrated that those persons exposed to a family planning program were more likely to use family planning post-implementation, but overall exposure declined in the two-years since the program ended [18].

At the program level, sustainability may be measured as continued intervention activities, such as outreach programs or trained providers still practicing their skills or implementing activities after program completion. For example, a project in northwest Tanzania that focused on training village health workers about maternal health showed continuation of the community-supported transport system and community workers providing education and referrals up to five years after the end of the project in communities where these activities were incorporated into the village political systems [19]. A literature review on the sustainability of health interventions in sub-Saharan Africa found 41 studies of sustainability [20]. The main program areas of these studies were HIV/AIDS and malaria and none of the studies specifically focused on family planning programming. The authors found that community ownership and community mobilization, including resource mobilization, were key components of programs that sustained [20].

Methods

In 2011, a survey of facilities in NURHI project cities (Abuja, Benin City, Ibadan, Ilorin, Kaduna, and Zaria) was undertaken to provide a baseline profile of the family planning (FP) supply environment prior to the launch of the NURHI program. At endline (2014), another survey was undertaken to examine changes in facility level indicators in the same six NURHI study cities. In 2017, a follow-up survey of facilities was undertaken in two cities: Ilorin and Kaduna.Footnote 1 These two study cities were intentionally selected to provide differing perspectives of program activities. In particular, the NURHI program continued in Kaduna in the second phase of NURHI (called NURHI 2) while the program ended in March of 2015 in Ilorin.

The initial set of facilities interviewed in this study was determined at baseline. It included all public sector facilities and all private sector facilities where NURHI was working (or intended to work at baseline) as well as any additional facilities named by female household survey respondents in the baseline household survey from 2010/2011 as locations where they obtained FP or maternal and child health services. At baseline, interviews were undertaken in 72 facilities in Ilorin and 92 in Kaduna. At endline (2014) the project team sought to undertake interviews in the same facilities from baseline. A total of 71 facilities from Ilorin and 83 from Kaduna were included at endline. By the time of the follow-up sustainability study (2017), interviews were undertaken in 70 facilities from Ilorin and 86 facilities from Kaduna. The numbers of facilities change over time primarily due to a small number of refusals at the time of follow-up data collection.

In each study facility at each round of data collection, interviewers implemented a facility audit and provider surveys with up to four providers, depending on the size of the facility. The facility audit interviewed a manager or administrator in the facility and included detailed questions about services offered, number of clients, provider characteristics, quality of services, and equipment in the facility. A copy of the facility audit used in 2017 is included as supplementary materials to this paper (Additional file 1); the tool used in the earlier phases is similar to this tool. Providers were asked questions about their training in FP, services they offer and potential biases by age or marital status with provision of FP. A copy of the provider survey used in 2017 is included as supplementary materials to this paper (Additional file 2); the tool used in the earlier phases is similar to this tool. Facility surveys also included exit interviews with clients obtaining sexual and reproductive health services in a subset of higher volume facilities; exit interview data are not used in this analysis.

Among the facilities included in the study, none were “NURHI supported” facilities at baseline whereas by 2014, NURHI was working in 22 facilities in Ilorin and 22 facilities in Kaduna; this represents 31 and 27% of study facilities in Ilorin and Kaduna, respectively (see Table 1). For the analyses presented below, those facilities identified as “NURHI supported” in 2014 remain in this classification in 2017 for examination of long-term sustainability, even though the NURHI program was no longer working in Ilorin in 2017. We exploit the facility-level data that includes information on the same facilities across multiple time periods, by undertaking multivariate analyses of the quality of services over time, allowing estimation of how quality varies by city, time period, and whether the facility was NURHI supported.

Table 1 Characteristics of health facilities by city and time period, MLE facility surveys from Kaduna and Ilorin, Nigeria

The main service delivery measure used in this paper is quality of services. As done in earlier studies [4, 8, 21], we create a quality index to measure the multiple, correlated components of service quality. The quality index is created from principal components analysis (PCA) of the facility data from the three time periods in one PCA using the first component of the score. This component captures about 22% of the total variance for the 30 items that make up the measure. The quality index incorporates the following measures: method availability (10 methods are captured); stockout of each method at the facility (8 methods); information, education and communications (IEC) materials available (8 types of IEC materials measured); outreach activities (3 types of activities); and the mean number of providers trained in the last year (see Table 2 for a description of these variables). We create an overall quality index which is examined by city and time period. This quality index is used as an outcome variable as well as a key independent variable for the analysis of FP service use. In addition, related to each of the components of quality (method availability, stockouts, IEC, outreach, and provider training), we create indexes for more detailed analyses. Notably, except for training, we use PCA to create the component indexes. Training is calculated as the average number of providers trained.

Table 2 Inputs for quality measures and service delivery outcomes included in the facility surveys

Two FP service use measures are used as outcomes as well: the ratio of the number of new FP clients in the last year to the number of FP and reproductive health (RH) staff, and the ratio of the total number of FP clients in the last year to the number of FP and RH staff. The ratio adjusts for the fact that larger facilities have more staff and will have more clients. The explanatory variables in the model are all dummy indicator variables: indicators for survey round, city, NURHI supported facility, and whether the facility was private. In addition, interactions are examined between time point (e.g., 2014 and 2017) and the city and whether the facility was a NURHI facility. Three clinics were dropped for the new client analyses and four from the total client analyses because they had extremely high and unrealistic values for the dependent variables and had an undue influence on the regression results.

Descriptive analyses are used to examine the characteristics of the study facilities at each time point. In addition, we examine the overall quality measure as well as the components of quality and service use across the two cities and at each time point. Because the data are longitudinal at the facility level, we used longitudinal regression methods. In particular, we specified a random effects model and used maximum likelihood estimation methods. Since the model is linear, the coefficient estimates from these models are also the marginal effects; all multivariate results are presented with the marginal effects and their 95% confidence intervals. Random effects estimation estimates the standard deviation of the two components of the error term in addition to the regression coefficients. The standard deviation of the time invariant error is labeled “Sigma mu” and the standard deviation of the time varying error is labeled “Sigma epsilon” in the tables. In all instances except one, both error components were highly significant indicating the importance of correcting for both sources of error. However, also in all cases, the time varying error component was considerably larger than the time invariant component indicating that time varying unobservables were relatively more important factors. Another assumption for the random effects model is that the explanatory variables are not correlated with the time invariant error. We tested this null hypothesis using a robust version of the standard Hausman test (see, for example. Wooldridge, 2010, page 132 [22]). In all cases, we failed to reject the null hypothesis which again is supportive of the validity of our estimation strategy. All multivariate analyses were performed using the xtreg program from the Stata statistical software package and the robust version of the Hausman test used the user created xtoverid Stata add on.

Results

Table 1 presents the characteristics of the facilities included in the study. As mentioned earlier, no facilities were classified as NURHI facilities at baseline since the program had not yet begun at that time. By the time of endline data collection (2014), 31 and 26.5% of study facilities in, respectively, Ilorin and Kaduna were NURHI facilities. Table 1 also shows that about two-fifths of study facilities in Ilorin were public sector facilities while only about a third of facilities in Kaduna were public sector facilities. In both cities, most of the facilities included were hospitals. The next most common type of facility was health centers followed by clinics.

Table 3 presents the results of the principal component analyses of the overall quality index as well as for the component indexes of the overall quality (method provision index, IEC index, stock out index, and outreach index) and the average number of providers interviewed that were trained by city and time period. Because these measures are from principal components analysis, the interpretation is not straightforward, so we focus on the signs and trends. The overall quality index in both cities was lowest at baseline (2011). By endline (2014), the overall quality improved in both cities. By the sustainability study (2017), overall quality remained positive and the values were larger than in 2014 in both cities. Availability of methods in the facilities over time and by city has a similar pattern to that of the overall quality – it was negative and increased over the follow-up periods. Stockouts are not considered good and thus the sign of the index is in the opposite direction such that there were more stockouts in Ilorin at baseline, these reduced by endline but increased again by 2017. For Kaduna, there were fewer stockouts between baseline and endline and stockouts increased by 2017 as well. The outreach index improved in both cities between baseline and endline but declined somewhat between endline and 2017. Finally, provider training in Ilorin increased between baseline and endline but declined again by 2017 whereas in Kaduna, where the program continued, provider training increases across the three time periods.

Table 3 Description of service outcomes at baseline (2011), endline (2014), and sustainability study (2017) in the analysis sample by city, MLE facility surveys from Kaduna and Ilorin, Nigeria

Also shown in Table 3 are the service use outcomes measured as the number of new FP acceptors in the last 12 months per reproductive health staff member and the total number of family planning users in the last 12 months per reproductive health staff member. In Ilorin, both service delivery outcomes increase between baseline and endline and then decline somewhat by 2017 whereas in Kaduna, both outcomes continue to increase over time with large increases in total FP users per staff member between endline and 2017.

Multivariate random effects models were estimated to examine whether the quality of services is related to NURHI program support and if there were changes in quality over time and across the two cities. These results are presented in Table 4 (overall quality index) and Table 5 (components of quality). Table 4 demonstrates that NURHI supported facilities were of higher quality than non-NURHI supported facilities (marginal effect: 2.32; 95% CI: 1.58; 3.06; p < 0.001). In 2011, Ilorin facilities were of higher quality than Kaduna facilities (the Ilorin marginal effect is positive and significant), however, the interaction between Ilorin and the two time periods (2014 and 2017) indicates that by 2014, Ilorin facilities had lower overall quality than Kaduna facilities and this continues into 2017. That said, over time in both cities the quality of services is significantly better than at baseline (positive and significant marginal effects on the 2014 and 2017 variables). There is no difference in NURHI facilities in 2017 as compared to the earlier time period (2014). Finally, private facilities have lower quality across all time periods and cities.

Table 4 Random effects results predicting quality of services in facilities, MLE facility surveys from Kaduna and Ilorin, Nigeria
Table 5 Random effects results predicting program components scores among facilities, MLE facility surveys from Kaduna and Ilorin, Nigeria

Table 5 presents similar models for each of the component indexes from the overall quality index. The results for the method provision index are the same as those presented for overall quality – that is NURHI facilities perform better than non-NURHI facilities, Ilorin had better method provision at baseline but it is not better than Kaduna by endline and 2017, and overall, method provision improved over time. For the training index, Ilorin and Kaduna were similar at baseline and by endline, Ilorin was doing better than Kaduna. That said, by 2017, Kaduna was doing better on provider training than Ilorin (marginal effect: -0.20; 95% CI: − 0.29; − 0.11; p < 0.001). Improvements in availability of IEC materials are observed over time and NURHI facilities do better than non-NURHI facilities. In addition, Kaduna facilities in 2014 have better IEC than Ilorin facilities but there is no difference by 2017. For outreach, Ilorin was doing better at baseline and while improvements happened over time, by endline (2014) and 2017, Ilorin is doing worse on outreach (p < 0.10). Further, private facilities are doing worse than public facilities on outreach. The last index shown in Table 4 is stockouts. There are fewer factors related to stockouts and stockouts are found to increase by 2017 relative to 2011. This likely reflects state-level rather than city-level commodity distribution systems that are not as controlled by specific facilities and the NURHI program.

Table 6 presents the multivariate random effects models of the effect of quality, city and NURHI facilities on the total number of family planning users in the last year per reproductive health staff member. Two models are shown, the first using the overall quality index and the second using the individual component indexes of quality. Both models control for the facility and time characteristics shown previously to be related to the quality of services. Model 1 shows that higher quality facilities have a higher ratio of clients to providers, although the effects do not attain significance (p = 0.120). Between baseline and endline, there is not a significant increase in the number of family planning clients to providers; however, by the time of the sustainability study (2017), all facilities generally had more clients to providers (p < 0.001) and NURHI facilities had more clients to providers than non-NURHI facilities (p = 0.009) (and private facilities had fewer – p < 0.001). The interaction between time and Ilorin was only marginally significant in 2017 such that there was no difference in service delivery between the two cities in 2011 and 2014 when the program ended. However, by 2017, Kaduna facilities had a slightly higher client load (p = 0.081). The results of Model 2 demonstrate that those facilities with more methods available have more family planning clients to providers and those with fewer stockouts also have more clients to providers. All the control variables perform similarly in Model 2 with the exception of the Ilorin and 2017 interaction which is not significant.

Table 6 Random effect model results predicting the effect of overall quality and program component scores on the number of FP users in the last year per reproductive health staff member, MLE facility surveys from Kaduna and Ilorin, Nigeria

Table 7 presents the multivariate random effect results for the outcome of new family planning clients per reproductive health staff member. These results are like those presented in the previous table such that quality is positive but does not attain significance (p = 0.196) and the number of new clients to providers is significantly higher over time and in NURHI-supported facilities. In addition, by 2017, the number of new clients to providers is lower in Ilorin as compared to Kaduna (Ilorin and 2017 interaction – marginal effect: -20.57; 95% CI: − 35.42; − 5.72; p = 0.007). Using the specific quality components, we find that method availability (p < 0.10) and stockouts remain important in this model in the same direction as earlier.

Table 7 Random effect model results predicting the effect of overall quality and program component scores on the number of new family planning acceptors in the last year per reproductive health staff member, MLE facility surveys from Kaduna and Ilorin, Nigeria

Discussion

This study seeks to determine the post-program sustainability of the quality of FP services by comparing a city where a program continued to a city where the it ended two years earlier. Determining the level and extent of sustainability is important for informing future programs seeking to improve the quality of FP services in the short and long term.

Our results demonstrate that NURHI facilities tended to have better quality and higher service use than non-NURHI facilities. Further, while quality of services was higher in Ilorin at baseline (2011), by endline (2014) and three years later (2017), the quality was better in Kaduna where the program continued. This was shown for the overall quality index as well as the components of training, outreach, and method availability; each of these were lower in 2017 in Ilorin as compared to Kaduna. In addition, while no difference was found in service utilization between Ilorin and Kaduna at endline, by 2017 Kaduna had significantly more new users than Ilorin. These findings demonstrate that quality of services was maintained in NURHI facilities from Ilorin after the program ended. Further, while quality of services increased in 2014 and 2017 in both cities, quality did not continue to improve at the same pace in Ilorin as it did in Kaduna where the program continued. Two years after program completion, there were fewer new FP clients in the Ilorin facilities.

The literature on sustainability points to reasons why we care about sustainability of program activities including reaping the benefits of investments made and determining longer-term effects that may be missed when measurement ends at program completion [11]. Further, program activities may diffuse or spread, for example by trained providers training others, by outreach workers continuing to give messages after the program ends and spreading messages to locations beyond their normal catchment area, or by friendly competition whereby NURHI improved facilities lead other facilities to improve their service quality. Without an explicit measurement strategy to capture what happens post-implementation, this information goes unknown and cannot be used for design of future programs that are sustainable and scalable.

Our findings are not without limitations. First, there is no standardized measurement of quality of FP services [5] and therefore we used proxy measures based on a facility audit and provider interviews. Second, information was not available from provider client observations and mystery clients to strengthen the measurement of program quality at each time period. Further, while exit interviews were undertaken as part of the study and can often give information on client satisfaction, these were only undertaken in high-volume facilities, limiting their utility for this study. Third, we were not able to examine directly whether quality of services affects reported contraceptive use since linking women from household surveys to facilities is complex, particularly in urban settings [23]. That said, we did show that facility quality is related to service use, measured as the ratio of family planning clients (or new clients) to number of reproductive health staff members. Fourth, the sample of facilities was too small to create all the potentially relevant interactions including the three-way interaction of being a NURHI facility in Ilorin in 2017. Qualitative data would be useful for better understanding detailed changes across various facilities (NURHI and non-NURHI) by 2017 in both Kaduna and Ilorin.

Conclusions

This study demonstrates that the quality of services in the city where the program was terminated did not maintain its upward trend from the earlier period. On the other hand, service quality continued to improve in the city where the program continued. When designing new programs, WHO and ExpandNet recommend “beginning with the end in mind” [24]. This seems particularly relevant for programs that have a short time duration and are seeking to improve quality of services. Identifying long-term strategies that support continuation of the key components of the program is essential. For example, programs need to ensure that a system is developed to address stockouts beyond the finite program timeline; identify strategies for continued training of providers since there is significant provider turn-over in many of these settings; and develop an approach that ensures that facility improvements are maintained including having a strategy to maintain improvement and repair equipment post-program. These types of approaches should lead to sustained quality improvements beyond the life of the program.

Availability of data and materials

The dataset supporting the conclusions of this article is available from the corresponding author and can be requested at this site: https://dataverse.unc.edu/dataverse/mle

Notes

  1. Note that in 2017, facility-level data were also collected in a third city, Jos, where NURHI program activities were never undertaken; data from Jos are not presented in this paper.

Abbreviations

BMGF:

Bill & melinda gates foundation

CCP:

Center for communication programs

FP:

Family planning

IEC:

Information, education, and communication

MLE:

Measurement, Learning & Evaluation project

NURHI:

Nigerian urban reproductive health initiative

PCA:

Principal components analysis

RH:

Reproductive health

References

  1. Jain A. Quality of care in the context of rights-based family planning. New York City: Population Council; 2017 [Accessed 27 Jan 2019]. Available from: http://www.popcouncil.org/uploads/pdfs/2017RH_QoCRightsBasedFP.pdf.

  2. Kumar J. How does quality of care relate to a rights-based approach to family planning programs? New York City: Population Council; 2015 [Accessed 27 Jan 2019]. Available from: https://www.popcouncil.org/uploads/pdfs/2015RH_QOC-RightsBasedFP_wp1.pdf.

  3. FP2020. Rights and empowerment principles for family planning. Washington, DC: Family Planning 2020; 2016 [Accessed 27 Jan 2019]. Available from: http://ec2-54-210-230-186.compute-1.amazonaws.com/wp-content/uploads/2016/08/FP2020_Statement_of_Principles_11x17_EN_092215.pdf.

  4. Tessema GA, Gomersall JS, Mahmood MA, Laurence CO. Factors determining quality of care in family planning services in Africa: a systematic review of mixed evidence. PLoS One. 2016;11:11.

    Article  Google Scholar 

  5. Fruhauf T, Zimmerman L, Kibira SPS, Makumbi F, Gichangi P, Shiferaw S, Seme A, Guiella G, Tsui A. Measuring family planning quality and its link with contraceptive use in public facilities in Burkina Faso, Ethiopia, Kenya and Uganda. Health Policy Plan. 2018;33:828–39.

    Article  Google Scholar 

  6. Agha S, Do M. The quality of family planning services and client satisfaction in the public and private sectors in Kenya. Int J Qual Health Care. 2009;21(2):87–96.

    Article  Google Scholar 

  7. Hutchinson PL, Do M, Agha S. Measuring client satisfaction and the quality of family planning services: a comparative analysis of public and private health facilities in Tanzania, Kenya and Ghana. BMC Health Serv Res. 2011;11:203.

    Article  Google Scholar 

  8. Winston J, Calhoun LM, Corroon M, Guilkey D, Speizer I. Impact of the urban reproductive health initiative on family planning uptake at facilities in Kenya, Nigeria, and Senegal. BMC Womens Health. 2018;18:9.

    Article  Google Scholar 

  9. Scheirer MA. Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am J Eval. 2005;26(3):320–47.

    Article  Google Scholar 

  10. Pluye P, Potvin L, Denis JL. Making public health programs last: conceptualizing sustainability. Eval Program Plann. 2004;27:121–33.

    Article  Google Scholar 

  11. Scheirer MA. Linking sustainability research to intervention types. Am J Public Health. 2013;103(4):e73–80.

    Article  Google Scholar 

  12. Stirman S, Wiltsey JK, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7(1):17.

    Article  Google Scholar 

  13. Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101(11):2059–67.

    Article  Google Scholar 

  14. Sarriot E, Jahan S, Kouletio M, Sardar M, Ali KL, Saha S, Rasul I. The end of magical thinking: sustainability evaluation three years after the end of the Saidpur and Parbatipur urban health project. Bangladesh: Concern Worldwide; 2008 [Accessed 27 Jan 2019]. Available from: https://www.measureevaluation.org/resources/networks/datausenet/dashboards-and-data-use-forum-may-2010/end-magical-thinking.pdf.

  15. Eder C, Khatiwada LK, Schooley J. Sustainability of a community-based CHOICE program to improve the health and nutrition status of mothers and infants in Indonesia. Matern Child Health J. 2018;22(6):903–12.

    Article  Google Scholar 

  16. Jejeebhoy SJ, Prakash R, Acharya R, Singh SK, Daniel E. Meeting contraceptive needs: long-term associations of the PRACHAR project with married women’s awareness and behavior in Bihar. Int Perspect Sex Reprod Health. 2015;41(3):115–25.

    Article  Google Scholar 

  17. Pandey N, Jejeebhoy SJ, Acharya R, Singh SK, Srinivas M. Effects of the PRACHAR project’s reproductive health training programme for adolescents: findings from a longitudinal study. New York City: Population Council; 2016 [Accessed 27 Jan 2019]. Available from: https://www.popcouncil.org/uploads/pdfs/2016PGY_PracharReport.pdf.

  18. Speizer IS, Guilkey DK, Escamilla V, Lance PM, Calhoun LM, Ojogun OT, Fasiku D. On the sustainability of a family planning program in Nigeria when funding ends. Chapel Hill, NC: MLE working paper; 2019.

    Google Scholar 

  19. Ahluwalia IB, Robinson D, Vallely L, Gieseker KE, Kabakama A. Sustainability of community-capacity to promote safer motherhood in northwestern Tanzania: what remains? Glob Health Promot. 2010;17(1):39–49.

    Article  Google Scholar 

  20. Iwelunmor J, Blackstone S, Veira D, Nwaozuru U, Airhihenbuwa C, Munodawafa D, Kalipeni E, Jutal A, Shelley D, Ogedegbe G. Toward the sustainability of health interventions implemented in sub-Saharan Africa: a systematic review and conceptual framework. Implement Sci. 2016;11(1):43.

    Article  Google Scholar 

  21. Tumlinson K, Pense BW, Curtis SL, Marshall SW, Speizer IS. Quality of care and contraceptive use in urban Kenya. Int Perspect Sex Reprod Health. 2015;41(2):69–79. https://0-doi-org.brum.beds.ac.uk/10.1363/4106915.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Wooldridge J. Econometric analysis of cross section and panel data. Cambridge, MA: The MIT Press; 2010.

    Google Scholar 

  23. Cronin CJ, Guilkey DK, Speizer IS. 2017. The effects of health facility access and quality on family planning decisions in urban Senegal. Health Econ. 2017;27(3):576–91.

    Article  Google Scholar 

  24. World Health Organization (WHO) and ExpandNet. 2011. Beginning with the end in mind: planning pilot projects and other programmatic research for successful scaling up. Geneva: WHO; 2011 [Accessed 27 Jan 2019]. Available from: http://apps.who.int/iris/bitstream/handle/10665/44708/9789241502320_eng.pdf;jsessionid=C9CDA96C7A77044F0FA49DBD196D3DE8?sequence=1.

Download references

Acknowledgements

The authors acknowledge those who supported this work at various phases of project design, implementation, and analysis. This includes: the Nigerian Urban Reproductive Health Initiative (NURHI) Team, data collectors, and study participants.

Funding

Funding for this project comes from a grant awarded to the Carolina Population Center (CPC) at the University of North Carolina at Chapel Hill by the Bill & Melinda Gates Foundation. This funder was kept informed of the study design and study progress but played no role in the collection, analysis, interpretation, and writing of this manuscript. We also received general support from the Population Research Infrastructure Program through an award to the Carolina Population Center (P2C HD050924) at The University of North Carolina at Chapel Hill. The contents of this article are solely the responsibility of the authors and do not necessarily represent the official views of CPC or the Bill & Melinda Gates Foundation.

Author information

Authors and Affiliations

Authors

Contributions

ISS is the overall Principal Investigator of the study and led the writing of the paper; LMC performed all of the analyses for the paper and reviewed and provided input into the final draft of the paper; CM worked closely with the data collection team throughout the process, organized the facility lists, and reviewed and provided inputs into the final version of the paper; PML participated in informing the methodological analyses of the paper and reviewed and edited the final version of the paper; CH did a literature review and wrote a first draft of the background section of the paper; DKG was responsible for the methodological approaches to ensure the rigor of the final analyses, he also contributed to writing the numerous drafts of the paper. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Ilene S. Speizer.

Ethics declarations

Ethics approval and consent to participate

All facility respondents were asked to consent to participate in the study. Prior to being surveyed, health providers were asked to sign consent forms that included details on the purpose of the study, potential benefits and potential risk, and clarified that the information provided would not be identifiable. Facility administrators who were asked questions about the services offered at the facility through a facility audit were asked for verbal consent to participate. All study tools including the consent procedures were approved by the Institutional Review Board at the University of North Carolina at Chapel Hill and by the National Health Research Ethics Committee of Nigeria (NHREC) in Nigeria.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Nigeria Sustainability SDP Service Provider Survey.pdf – Provider survey used as part of this study. (PDF 1065 kb)

Additional file 2:

Nigeria Sustainability SDP Facility Audit.pdf – Facility audit used as part of this study. (PDF 1145 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Speizer, I.S., Calhoun, L.M., McGuire, C. et al. Assessing the sustainability of the Nigerian urban reproductive health initiative facility-level programming: longitudinal analysis of service quality. BMC Health Serv Res 19, 559 (2019). https://0-doi-org.brum.beds.ac.uk/10.1186/s12913-019-4388-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12913-019-4388-3

Keywords