Introduction The St. Paul Neighborhood Network (SPNN) has been at the forefront of the community technology movement for over 35 years. We are led and staffed by people who understand the power of technology to build bridges to civic, educational, and economic opportunities. Our AmeriCorps Community Technology Empowerment Project (CTEP) has been a focus of this work since 2004. SPNN is issuing a Request for Proposals for an external evaluator to conduct an impact evaluation of the CTEP program following AmeriCorps requirement CFR 45 §§2522.500 - .540 and .700 - .740. The purpose of this evaluation is to strengthen the CTEP program, foster continuous improvement, and help identify best practices and models that merit replication, as well as programmatic weaknesses that need attention.
Background The COVID-19 pandemic has highlighted the need for digital skills training as more and more facets of education, work, and business have moved online. It also revealed the inequity between those who can adjust to a technology based economy and those who cannot. The National Skills Coalition in 2020 reported that 48 million of the 155 million workers in the US have few or no digital skills. Instead, they have what is called fragmented knowledge, the ability to perhaps send a text message but lacking the ability to edit a document, use a mouse, upload an application, or create a spreadsheet. This disparity tracks along racial lines, where African Americans (12%) and Hispanics (13%) are less likely than Whites (65%) to have high levels of digital readiness (Pew 2016). According to the Minnesota Department of Education's Digital Equity Community Needs Report published in 2020, thirteen percent (13%) of Minneapolis and St. Paul households lack an internet subscription or computer in their home. Access is particularly low among those 65 and older, living in low-income households, involved with the criminal justice system, or the 49.3% of Minnesotans of color who are foreign-born.
Over 80% of Fortune 500 companies require online job applications and most jobs in the United States require digital skills. The inability to access those opportunities contributes to Minnesota's employment disparities, which are among the highest in the nation according to The Minnesota Federal Reserve (2020). The Connectivity Blueprint, a 2021 initiative of the City of St. Paul and Ramsey County found that the digital divide contributed to a Black unemployment rate more than double that of Whites during the pandemic recession of 2020.
Theory of Change and Intervention CTEP's theory of change is that the provision of face-to-face, personalized instruction improves adult learners' technology literacy skills as well as their ability to use technology strategically to find, understand, and use information. Those changes in technology literacy skills are then hypothesized to translate into an increased probability of employment or better employment. CTEP's outside evaluations have provided evidence that digital literacy instruction coupled with assessment and certification positively affects learners' confidence in their digital skills and their ability to use digital skills to obtain or improve employment outcomes. Each year the CTEP AmeriCorps program recruits and trains up to 35 people (referred to as Members) and places them with our nonprofit partner organizations that are located in and serve low-income communities in all seven counties of the Twin Cities Metro Area. CTEP members assess learner's digital skills using the NorthStar Digital Literacy Assessments. Those who score lower than 85% on the test or who cannot complete the assessments receive targeted instruction. AmeriCorps members measure clients' progress through assessments in each of the nine NorthStar competencies, and proctor certification exams. CTEP members coach learners through NorthStar-aligned Portfolio-Based Assessments designed to teach learners to apply digital skills in real-world contexts (e.g., using email, social media, or job search)
In addition, members help host sites build capacity. CTEP members work on at least 3 capacity-building projects in the second half of their service year. Under the guidance of the site supervisor members design new programs and curricula; recruit, train, and supervise volunteers; and secure in-kind donations of hardware and software. Each year these efforts support sites in sustaining or expanding digital skills instruction beyond the members' service year.
Proposed Study The primary outcome of interest in the proposed evaluation will be the degree to which adult participants attain employment within one-quarter of being served by CTEP, or in the case of comparison participants, a workforce training program. In addition, we are interested in the hourly wage for job seekers who are employed one quarter after exit. The proposed study should build on CTEP's past evaluations that examined the link between instruction of technology literacy and digital skills provided to low-knowledge individuals by CTEP AmeriCorps members and their employment and wage outcomes one quarter after having received CTEP services. The results of the study will inform the future design and development of the CTEP program as well as the design of the evaluation for the subsequent grant cycle (******** ).
Data Limitations The target subjects of this study are job seeking adults who:
Were active in the program for a minimum of 4 hours Successfully earned at least one corresponding Northstar certification Obtained a job after exiting CTEP services CTEP acknowledges the challenge of available demographic data on some of these adults due to the privacy policies of certain host sites. In addition, many learners seeking CTEP services over the past 2 years are youth under 18 and therefore below the age of consent for providing their personal information.
Studies with limited data are acceptable provided that a thorough description of the limitations found, their sources, and recommendations for overcoming these limitations for the next evaluation are included in the final report. The Evaluation Plan should build on these recommendations by proposing an evaluation design and data collection process to be utilized for the next impact evaluation in 2028.
Responsibilities CTEP's past evaluations have utilized a quasi-experimental design using propensity score matching. Data for comparison participants will be obtained via a data-sharing agreement with the Minnesota Statewide Longitudinal Education Data System (SLEDS). The consultant will be responsible for developing the final survey design and data collection tools. The consultant will also train the current cohort of AmeriCorps members and the next cohort starting in September 2024. Past Evaluation reports, evaluation plans, and other relevant materials shall be made available to all applicants upon request.
Qualifications We seek consultants well experienced in running statistical analysis for quasi-experimental design evaluations and demonstrated experience with controls for data limitations. The consultant should have a working knowledge of the nonprofit sector, adult basic education, and workforce training programs. Experience with these sectors in Minnesota is important but not required. Experience with AmeriCorps is a plus. While the work for this study may be performed remotely, Minnesota-based consultants are highly desired. Budget up to $25,000 including funds for data collection. Deliverables A final Study Report that includes an introduction, description of study design and methods, findings, and results. An evaluation Plan for the next impact evaluation in 2028. The requirements for an impact evaluation are updated by AmeriCorps every year. Core components of the plan include but are not limited to outcomes of interest, research questions, evaluation design (data collection, analysis plan), and a budget and timeline for implementing the plan over 3 years. Training plan and curricula for the September 2024 AmeriCorps cohort Submission Proposals should be received no later than Friday, May 10, 2024, by 5:00 p.m. CST. Please include the CV of the principal investigator. If the PI will be supported by a team, please include a summary of each member's qualifications. Proposals and attachments must be in PDF format Send via email to Allison Williams, CTEP AmeriCorps Program Director, at . Please put "CTEP AmeriCorps RFP" in the subject line. Questions about the information contained in this RFP should be directed to Allison Williams at the email address above. Phone calls/voicemails will not be responded to. Responses to questions may be provided to all applicants at CTEP's discretion.
