COVID-19 and related social restriction policies have created unprecedented challenges for humanitarian and development organizations. These challenges are even more problematic when they appear in contexts with existing hazards and threats. Such is the case in Colombia, where COVID-19 emerged in a dynamic mix involving 1.8 million Venezuelan migrants and refugees, a legacy of long-standing armed conflict and massive internal displacement, and vulnerabilities to multiple natural hazards. This complexity calls for organizational adaptability and agility—starting with efforts to understand the rapidly evolving context.
In March, DevTech was asked by USAID/Colombia to conduct a Rapid Education and Risk Analysis (RERA) of the situation of urban and peri-urban school communities receiving migrant Venezuelans, as well as the needs of the education system working to accommodate them while addressing all school-aged children. The RERA would help USAID support the Government of Colombia to provide flexible and effective models of education for both migrant and receptor communities, to strengthen already stressed education systems, and tailor support for education access, permanence, and school community resilience.
Initial RERA planning assumed in-person primary data collection. However, as COVID-19 infection rates rose and social restriction measures were instituted in both the US and Colombia, DevTech quickly adapted its approach to conduct the RERA remotely. The RERA team would operate virtually from their home locations in Bogotá, New York, Washington, D.C., and Massachusetts throughout the RERA while under lockdown. Primary data collection for the RERA involved a quantitative survey followed by qualitative interviews of survey respondents—all of which would now be carried out by calling the cell phones of respondents, and semi-structured expert interviews.
The RERA team worked with USAID/Colombia, the Ministry of Education and local education secretariats to select 24 primary and secondary schools in 11 cities and peri-urban municipalities. The team collaborated intensively with local education secretariats and school principals by telephone to identify over 600 respondents and obtain their contact information. Ultimately, the survey recorded 457 successful responses, and qualitative interviews were conducted with 201 of those respondents.
Some key lessons have emerged in carrying out the RERA remotely:
Early and clear explanations of adaptation options help agile decision-making.
- DevTech drafted a concept note in March on how to adapt the RERA to remote management, with optional scenarios and budgetary implications, to inform a quick decision by USAID/Colombia and the Ministry of Education.
Intensive collaboration with local education authorities and principals can be even more crucial when working remotely.
- As is customary with an in-person RERA, the RERA Team sought entrées to local education secretariats through the Ministry of Education to begin the process of selecting schools and respondents.
- The team then invested considerable time sending emails and talking by telephone iteratively with local education secretariat officials to explain the RERA’s remote process, discuss sample requirements and school selection, as well as respondent types and our connectivity requirements. This took more time given the crisis that secretariats and schools were managing and getting through by phone was understandably challenging.
- Several principals became very involved and took the initiative to contact all respondents personally and explain the RERA process, and in one area prone to child recruitment, a secretariat required additional assurances of data protection to share student contact information.
**Various technologies are available and effective to support remote fieldwork. **
- The RERA Team used KoboToolbox for the survey, and a mixture of Skype (both Skype-to-Skype and Skype-to-telephone) and a local data collection firm’s own call center facility to administer the survey and conduct interviews.
- Documents were developed, shared and discussed using Google Drive and Google Documents.
**Conducting the quantitative survey and follow-up interview on the same call can avoid lower response rates in follow-up interviews. **
- Response rates for the RERA survey were strong, but dropped off for the second, follow-up interview calls to the same respondents.
- Merging a survey and qualitative interview can be organized within a one-hour call.
Working remotely can constrain efforts to randomize the selection of respondents.
- While this can occur in any in-person RERA, concerns about sharing respondent contact information electronically for the RERA team to randomize emerged as a limiting factor. Ultimately, school principals decided on respondent lists using criteria and guidance provided by the RERA Team.
Remote training can be effective—with sufficient time.
- Asynchronous and synchronous approaches are effective to allow participants time to study materials off-line and use synchronous platforms—such as live written or verbal question-and-answer sessions—strategically and efficiently.
- Plenary, synchronous sessions are important for specific topics which the entire team must discuss, such as ethics.
- Running simulations to assess data collectors and test the protocols themselves are a necessity, especially for open-ended interviews, but sufficient time is needed for corrections and re-runs.
- If a data collection firm is hired, maximizing the firm’s internal quality control and management systems to discuss training content, prioritize questions, and consider running internal question-and-answer sessions can winnow and prioritize the list of questions for plenary.
- The RERA Team needed more time to adequately test survey and interview questions before the launch.
**Remote data collection can build in certain biases. **
- It is a known limitation that requiring respondents to have a cell phone (or internet connectivity) can bias data collection away from the most vulnerable respondents in the sample. Tackling a lack of access to technology is a challenging and complex issue beyond the scope of an assessment, especially rapid ones. Some partners doing face-to-face surveys have issued phones and solar chargers to respondents, but this approach for remote, rapid assessments may not be feasible. Local phone sharing strategies have potential, but these are also complicated by the COVID-19 reality. A common approach is to find ways to supplement their data in order to infer the situation of vulnerable populations.
- In interviews, there were instances where parents accompanied their children on the telephone and directed them what to say. Interviewers can explain to parents the need to let their children speak freely, but this must also be carefully observed to ensure quality control of the data.
Remote management requires less funds—but may also reap less insights.
- Remote management does not require domestic and international travel, lodging and meal expenses.
- However, conducting conversations on sensitive issues by telephone can affect the level of confidence in the respondent. On the one hand, speaking remotely might increase a respondent’s trust and candor. On the other hand, and in contexts where child recruitment and other predatory behavior is common, speaking on the telephone can reduce trust.
- Typically, doing in-person school visits for a RERA offer opportunities for the team to observe the school environment and gain additional insights from being present. This dimension is lost in a remotely managed RERA.
**RERA briefing and validation meetings can and should be held remotely. **
- The RERA process involves two important moments of consultation with stakeholders—a briefing before data collection starts, and a consultation on preliminary results as data collection closes.
- These consultations were held using different on-line platforms and remain vital to the legitimacy of the RERA process and to the quality of the RERA conclusions and recommendations.
**Working under COVID-19 social restrictions affects everyone—including the RERA Team itself. **
- RERA is designed to be a rapid exercise, thus it is important to recognize the pressures and unique stresses that affect not only respondents and partners, but RERA Team members themselves.
- Working remotely under COVID-19 is not the same as working remotely in a normal situation. Understanding, flexibility and creativity all play important roles in managing for quality and results.