Project Activities
 
The JSI and Mosaica team is carrying out the evaluation through three approaches:

• Document Review and Research, providing a comprehensive review of existing materials related to the Campaign, including guidances, protocols, policies, talking points, memoranda, brochures, reports, web pages, work plans, and all other documents related to the structure, process, delivery, assessment, and evaluation of the Campaign. The review will familiarize the evaluation team with the project and the planned mix and level of activities across the regions.
•  Observation of Events and Activities and Satisfaction Surveys using several different tools:
  
  • Flexible but specific participant satisfaction surveys for use during most Campaign events. The tool will be designed to gather information for process and outcomes evaluation and, during the first six to nine months, formative/design evaluation. The survey tool will be modified for use with different activities, events, messages, and target audiences.
  • An observation checklist to be used by JSI/Mosaica staff attending selected Campaign events or activities to facilitate key data collection. Where the evaluators cannot attend events, the evaluation team will work with the RRCs or local OHAP contacts to provide observations where feasible.
  •  A debriefing instrument to guide the evaluators (or RRCs in their absence) in meeting with presenters, trainers, and community leaders immediately after selected events or activities to review the design, materials, and process, and obtain their perspectives on quality and potential results. The format will ensure that those responsible for the activities/events, including Campaign personnel and partners, will provide feedback on them.
    
    The evaluation team will develop a data collection protocol to outline the procedure by which surveys are administered, collected, and entered into a database. If a JSI/Mosaica evaluator is attending an event or activity, that team member will be responsible for the administration and collection of the survey instruments. In all other cases, the RRCs will be charged with this responsibility. RRC involvement has two direct benefits for the RRCs: increased sense of ownership in the evaluation and increased capacity to conduct evaluation.
    
    To ensure consistent data collection, the evaluation team will conduct a one-half to one-day training session for all RRCs and other key Campaign staff on evaluation, the methodology for the evaluation of the Campaign, and the data collection protocols. This training will be conducted by Mosaica staff and hopefully held in conjunction with a regularly scheduled RRN team meeting in Washington, DC.

• Semi-Structured Interviews, which will be used to gather information for process evaluation (including partner development) and for measuring outcomes, including partnerships developed and resources mobilized, evidence of initial and continuing action as a result of the Campaign’s messages and activities, and individual and community impact. Individuals to be interviewed will include federal agency personnel involved with the Campaign, RRCs, and other key stakeholders such as representatives of faith-based partners, representatives of community-based organization (CBO) partners, Speaker’s Bureau members, public officials, key event participants, and other community stakeholders.

JSI and Mosaica will take advantage of information technology throughout the project to gather, aggregate, and analyze data collected.
 
Mosaica and JSI will develop two reports based on the information gathered:
 

•  A Progress Report at the end of year one of the Campaign, which will include recommendations for substantial changes for year two events and activities.


• A comprehensive Final Report at the end of year two of the Campaign, which will include a report on outcomes, discussion of best practices or models, lessons learned, and recommendations for future Mobilization efforts.

Evaluation Team Background
 
John Snow, Inc. (JSI) is a health care research and consulting organization committed to improving the health of individuals and communities worldwide. Since its inception in 1978, JSI has grown to a staff of more than 400 with expertise in public health, clinical care, research methods, community and organizational development, and program operations. JSI’s commitment to improving HIV/AIDS prevention and care began in the early days of the epidemic. Since then, JSI has worked with both domestic and international organizations at the local, state, and federal levels to develop and advance HIV health services delivery for PLWH and prevention programs for those at risk. JSI’s work began with an assessment of the impact of HIV on federally funded community health centers in 1991 and an evaluation of the HIV Service Demonstration Program, the first federally-funded, coalition-based HIV/AIDS services program.
 
Since that time, JSI’s HIV/AIDS portfolio has expanded and now includes program evaluation, comprehensive planning, prevention and education, needs assessments, HIV service models, clinical trials and training, quality assurance and improvement, and technical assistance (TA). One of JSI’s core strengths is its ability to work with grantees and community-based organizations to develop and provide high-quality HIV/AIDS services that meet federal mandates and guidelines, yet are responsive to local needs and priorities. JSI has developed a rich network of community and government partners, clinicians and advocates, minority leaders, and people living with HIV/AIDS (PLWH) with whom it collaborates regularly.

JSI is the current contractor managing the Office of HIV/AIDS Policy’s The Leadership Campaign on AIDS (TLCA).
 
Mosaica: The Center for Nonprofit Development and Pluralism is a multicultural nonprofit that provides tools to other nonprofits to build just, inclusive, and thriving communities and societies and helps public agencies work effectively with nonprofits and communities. Its special commitment is to strengthen and support entities committed to serving and empowering those whose voices are least likely to be heard when public policies are adopted and resources allocated. 
 
A national organization established in 1994 in the District of Columbia, Mosaica:

• Helps nonprofit organizations, planning bodies, and community associations become stronger and more effective through training and facilitation, individual consultation, evaluation, and easy-to-use “how to” guides.


• Links the nonprofit, public, and private sectors in ways that strengthen local communities and enrich American society.


• Helps individuals and groups develop the skills to bridge racial, ethnic, and economic differences and work effectively in multicultural societies.

Mosaica has been involved in technical assistance efforts related to HIV/AIDS since its inception. It has assisted well over 100 minority CBOs, community planning bodies for HIV care and prevention, and public agencies involved in HIV, providing organizational assessments, evaluations, and a wide range of capacity-building services. For more than 12 years, Mosaica has long been a part of the Ryan White CARE Act Technical Assistance Contract, assisting Ryan White grantees and providers, planning bodies, and the Health Resources and Services Administration’s HIV/AIDS Bureau (HRSA/HAB), which administers the CARE Act.
 
Mosaica has been involved in many HIV-related evaluations and TA efforts with a strong community of color focus. A Mosaica team evaluated the Regional Resource Network program as implemented through September 2006. Mosaica has been a partner in OHAP’s The Leadership Campaign on AIDS (TLCA) for most of its existence. It is currently JSI’s subcontractor for TLCA, responsible for output-focused process evaluation and short-term project outcomes measurement, as well as technical assistance to community and faith-based partners and plain language training to federal staff. For more than six years, Mosaica was also part of a team that evaluated two national minority-focused HIV prevention capacity-building initiatives funded by the Centers for Disease Control and Prevention (CDC) – the National Regional Minority Organizations program and the Capacity Building Assistance (CBA-1) program.
 
Roles: JSI will oversee the project, participate in project design, and provide the data systems for the evaluation. Mosaica will be the lead agency conducting the day-to-day activities of the evaluation and assessment. Mosaica’s experience working in communities of color will inform all aspects of the project, from data collection to reporting.
 
JSI team members working on the evaluation include:

• Jeremy Holman, Senior Consultant, serves as Principal Investigator for the project. He is involved in overall project oversight and management, including completion of reports and deliverables. Dr. Holman has over eight years of HIV/AIDS experience with a focus on TA, capacity building, and community-based organizational management.
•  Rodolfo Vega, Senior Research Scientist, serves as a Technical Advisor for the project. He is involved in the design of the formative, process, and outcome evaluation components. Dr. Vega has extensive experience in program evaluation and in working with communities of color across the country on HIV/AIDS and substance abuse issues, with a focus on TA that helps community-based organizations translate research into practice.
• Thomas Mangione, Senior Research Scientist, serves as a Technical Advisor for the project. He is assisting with evaluation design and survey and evaluation tool development. Dr. Mangione is the Director of JSI’s Survey Research Facility and is an evaluation specialist with over 30 years of survey research experience using in-person, telephone, and self-administered data collection methods.
• Fong Lui, Staff Consultant, serves as Database Programmer for the project. She is involved in the development of a tracking database to produce timely data for evaluating Campaign activities. Ms. Lui has extensive experience with a wide range of programming and database applications that allow her to manage complex databases and reporting processes in a constantly changing health services environment.
•  Melinda Stylos-Allan, Staff Associate, serves as Research Assistant for the project. She will maintain the tracking database and provide administrative, research, and technical support for all project team members and activities.
   

The following Mosaica team members will serve as lead staff for the evaluation:

• Emily Gantz McKay, Mosaica Founder and President, serves as Senior Technical Advisor for the project. She is involved in project design and in all stages of the project, primarily planning, evaluation and instrument design, and report preparation. Ms. McKay has more than 35 years of experience in assisting minority nonprofits and 17 years in HIV/AIDS planning, prevention, and care.
•  Ruben Cantu, Senior Program Specialist, serves as Project Director. He is involved in project management, tasks and deadline monitoring, and report preparation. He is the primary point of contact with OHAP. Mr. Cantu has extensive experience as a technical assistance provider for minority CBOs working in HIV/AIDS. He has broad experience in program management and HIV prevention and care.
•  Nicole Robinson, Evaluation Specialist, serves as Evaluation Specialist, with primary responsibility for data collection and analysis. She will be involved in the development of evaluation measures and instruments, the implementation of the evaluation, including training of RRCs, the management of data collection and analysis, and report development. Ms. Robinson has extensive experience with HIV/AIDS and health promotion programming and evaluation.
• Ana Tellez, Program Assistant, serves as Project Assistant for the project. She provides general support for the project team, including assistance with evaluation data collection, aggregation, and analysis. Ms. Tellez has worked on multiple evaluation projects for local and national organizations while at Mosaica.

Additional Mosaica team members, such as Program Administrator and Senior Project Specialist Hila Berl (who will provide oversight and evaluation expertise) will also serve as part of the project team.
 
For More Information
 
For more information, please contact Ruben Cantu, Project Director, at 202-887-0620 or via e-mail at ruben@mosaica.org.