EHDI 2016 Evaluation Capacity Building RFP Questions and Answers

Eligibility

Are current EHDI grantees eligible to apply or does this affect scoring in any way?

Please note that the purpose of the grant is to fund professional services to build the evaluation capacity of current EHDI community grantees. Funds may not be used to plan or implement a health program or research. If a current EHDI grantee has the qualifications and capacity to provide the needed evaluation services to all other current grantees and their own EHDI program, the grantee is eligible to apply. However, the RFP is not limited to nor was originally intended for current grantees.

Is this RFP a limited competition to only current grantees?  The introduction seems to imply this but the eligibility criteria do not mention anything about it.

Please see response above.

Proposal Contents

Can there be two principal investigators if there is a multiple-partner approach?

A multiple-partner approach is acceptable, but please designate only one lead organization.

Our consulting practice has rates that have been approved in several State contracts, including prior MDH contracts. For the budget, may we use these in the “contractual services” line items rather than the “salary/fringe” line items?

No. The contractual services budget line-item is for subcontracts, or the amount paid to nonemployees for services or products.

History of Grant

What are the most significant changes that were made to this EHDI evaluation capacity building grant over prior years?

The shared outcome measurement system is the most significant change from previous years. The components of the evaluation capacity building have remained largely the same over the years – one-on-one TA on the planning and implementation of their evaluation, reporting, and dissemination; trainings; peer learning in large or small groups; and provision of evaluation resources (e.g., forms, templates, tip sheets, published articles, webinars, etc.).

What were some of the major successes of EHDI evaluation capacity building in prior years?

In surveys conducted with grantees in the past, they reported valuing most the one-on-one technical assistance and the affinity group meetings with fellow grantees working in the same priority area or priority population. The latter, especially, allowed them to learn about successful strategies, share evaluation tools, and collaborate on activities.

What were some of the weaknesses or challenges of EHDI evaluation capacity building in prior years?

• Frequent staff turnover at grantee organizations which diminished capacity building gains
• Small size of grantee organizations meant limited capacity to create a culture of evaluation within the organization despite earnest efforts to learn
• Absence of a shared outcome measurement system allowed grantees to specify outcomes that were most appropriate and most relevant for their program; however, it also meant there was no collective interest in what they were accomplishing (or not accomplishing) together as grantees working in same priority health area or with the same population, or how as a group they were contributing to the elimination of disparities in their health area. On MDH’s part, this also limited its ability to report in aggregate the progress made by EHDI grantees in reducing health disparities in each of the priority health areas.
• Inactivity in the community of practice due to (1), (2) and (3), thus grantees sometimes found participating in or leading activities burdensome.
• Underutilization of MDH content experts who can provide insight into the priorities and outcome measures tracked statewide or nationally in the priority health areas, possibly also due to (1), (2) and (3).
• Limited participation by grantees’ external evaluators, possibly due to the set number of hours in their contracts.

Who created the materials for and delivered the training described on p. 8, including templates for progress reports, evaluation templates, and available tip sheets; webinar on building a logic model and evaluation plan?

These materials were created by both MDH staff and a previous evaluation capacity building provider and revised over the years by current MDH staff. The CHE evaluation coordinator delivered the recent training on building a logic model and evaluation plan.

Who was the most recent evaluation vendor for the EHDI? What other external (outside of MDH) consultants, if any, are working with EHDI grantees on other capacity building activities?

Rainbow Research Inc. was the evaluation provider until July 2014. There has been no evaluation provider since then. At this time the only information we have is that approximately two-thirds of grantees have indicated that they plan to hire an external evaluator to do all or part of their EHDI program evaluation. We do not know yet how many of these are currently working with an external evaluator.

Grant Responsibilities and Scope of Work

What opportunities will the selected vendor have for evaluation-related activities in the half-day EHDI Grantee Meetings?

In past grantee meetings, the evaluation capacity building provider conducted trainings or led small group discussions on selected evaluation topics, or had mini consulting sessions with grantees.

Where will these meetings be held?

Whenever possible, the Center for Health Equity (CHE) holds grantee meetings at a community site. All have been in the Twin Cities area. Neighborhood House, Wilder Foundation, and Concordia College are examples of venues for past grantee meetings. Smaller meetings also have been hosted by EHDI grantees at their own sites.

Who from MDH will we be working with most closely? What are their expectations for reviewing deliverables, participating in planning, or other contributions to EHDI evaluation capacity building?

The evaluation capacity building provider will work with the CHE evaluation coordinator and grant managers. To a lesser extent, the provider will work with the CHE director, MDH director of American Indian Health, and MDH content experts. These MDH staff members will be involved in ensuring evaluation capacity building goals and objectives are met, ensuring that activities align with Grantee Guidelines, and providing feedback into the planning of evaluation activities. Once on board, the evaluation capacity building provider is expected to take the lead, with MDH staff playing a supporting role.

Will the EHDI staff continue to play a technical assistance role, as described in the fourth bullet on p. 8 of the RFP? If so, how do you envision the roles of EHDI staff vs. the selected contractor?

Please see response to previous question.

Who from MDH will be working with the contracted vendor and what roles will they have?

Please see response to previous question.

Are evaluation requirements the same for grantees, regardless of whether their grant is for $50,000 or $200,000?

Yes. All grantees are required to use at least 10 percent of their grant-funded program budget for evaluation purposes, and all grantees are required to evaluate their programs and report their findings in reports to MDH.

May grantees pool resources for evaluation? For example, collaborate to hire an external evaluator that implements shared instruments on their behalf?

Grantees are free to contract with whomever they choose with MDH approval. In the past there were two or three external evaluators who each worked with two grantees. An external evaluator working with two or more grantees may design the same instrument for those grantees if there are commonalities, such as the use of the same curriculum or similar strategies. As long as each grantee separately contracts with the evaluator, and separately invoices MDH for the services provided by the evaluator, they may pool their resources.

What language or communication preferences should the vendor be aware of when working with the grantees?

None. Grantees have shown openness to using the English language medium and various modes of communication.

Beyond the funding requirements for evaluation noted in the RFP, what information have grantees received to set expectations for working with the selected vendor?

None. Grantees were told at the two grantee meetings held thus far that MDH would be contracting with an evaluation capacity building provider sometime in January who would be able to assist them in designing and implementing their evaluation, and that this was an option available to them. The evaluation capacity building provider scope of work was not yet available at the time.

We see the list of grantee profiles for EHDI grantees in 2012-13 and the current list for 2016-17. Were there EHDI grantees in the interim years (2014 or 2015)?

Yes, the 2012-2013 EHDI grantee contracts were amended to extend the grant period for the following years.

On p. 9, under work item 1, can the in-person meetings to provide technical assistance (d) occur during in-person site visits (c)? Or is it intended that these are separate encounters?

They are intended to be separate encounters. The one-on-one meetings to provide technical assistance are a must, while the in-person site visits are on an as-needed basis only. According to MDH Policy 238.01 on Grantee Monitoring, the purpose of a site visit is “to review and ensure progress against the grants’ goals, to address any problems or issues before the end of the grant period and to build rapport between the state agency and the grantees. This visit may cover topics such as statutory compliance, challenges faced by the grantee, modifications made to the grant program, program outcomes, grantee policies and procedures, grantee governance, and training and technical assistance needs.” For these reasons, CHE grant managers may invite the evaluation capacity building provider to site visits when appropriate, but participation in all site visits is not expected.

How would you describe the differences and areas of potential overlap between technical assistance provided to grantees from the selected vendor and a grantee’s contracted external evaluator?

They should not overlap. A grantee contracting with an external evaluator may choose not to work with the evaluation capacity building provider outside of EHDI meetings, community of practice, and adopting a shared measurement system. If they choose to accept technical assistance from the evaluation capacity building provider, MDH expects the provider to work in close collaboration with the external evaluator to avoid duplication of services. In the past, the evaluation capacity building provider would check in regularly with the grantee and their evaluator to make sure they were clear on what deliverables were due and when, or would offer to review their deliverables and work with the external evaluator to improve them. It is important to note that the evaluation capacity building provider must not conduct data collection and analysis or report writing on behalf of EHDI grantees, which are tasks that a grantee’s contracted evaluator may perform. The evaluation capacity building provider’s role is to provide technical assistance to improve grantees’ evaluation work and so that EHDI grantees are able to conduct these activities without assistance in the future.

Community of Practice

What online content from prior grants will be maintained? In particular, is there an online community of practice that will continue to be live and able to be accessed? What role do you hope the new contractor takes on in maintaining existing content vs. building new resources for the community of practice?

There is currently an EHDI Grantee Portal webpage on the MDH website where grantees can access resources, forms, and other documents that have been provided to them. This webpage is maintained by CHE grant managers. Outside of the grantee meetings and site visits, all communications regarding grant updates, announcements, and requirements are currently sent via group email. There is no existing online community of practice that the evaluation capacity building provider will maintain; the provider is expected to create new content and/or platforms that would build an active community of practice among EHDI grantees.

What has MDH already heard from grantees on what they would like to see out of a community of practice?

Having an EHDI Community of Practice was discussed briefly at one of the grantee meetings. It is our intention that grantees will be able to provide input in its development, and we expect the evaluation capacity building provider to incorporate the feedback of grantees as appropriate and reasonable. The results from the two meeting evaluations may also provide some insight into grantees’ needs and wants. When asked what they would like to learn or happen in future meetings, the responses included:

• How to make a good logic model great (one that reflects reality)
• Data on specific health areas and specific populations (e.g., data on children's health issues in the South Asian community in Minnesota)
• Success stories or creative ways grantees are providing services or reporting findings
• Communities of practice around specific health issues
• Networking opportunities organized by target population
• Creating spaces for more communication and collaboration
• Getting to know other grantees better

Is the web-based community of practice structure suggested in the RFP the preferred mode communication and networking among grantees?

Current grantees have not given input on the mode of communication and networking. However, grantees in previous EHDI grant cycles expressed their appreciation for webinars and conference calls due to busy schedules, but also acknowledged that there was no substitute for in-person interactions. The community of practice does not necessarily have to be web-based; it could be a mix of online and in-person activities.

Shared Outcome Measurement System

Has a shared outcome measurement system been developed in previous iterations of the EHDI grants?

No.

To what extent is the shared measurement system expected to be used beyond the duration of this EHDI funding cycle? What other non-EHDI programs will also use the shared measurement system?

The intent is for the shared measurement system to be used exclusively by EHDI grantees. It is expected to be used at least until June 30, 2019.

Given the grantees have already started their work plans and their current grant ends in 7 months, when does the shared measurement system need to be operational?

The anticipated end date for current grantees is June 30, 2019, contingent on funding availability and satisfactory performance. We do not anticipate that the system will be fully operational within the next seven months, but at the very least we expect to be able to provide grantees some information on what to expect in fiscal years 2017-2019.

How will the ideal mode (e.g., interactive web-based, submission of standard written/electronic forms) of the shared measurement system be determined?

The grant applicant should propose how the shared outcome measurement system will be deployed. Applicants should keep in mind that the system should not be so sophisticated that grantees who do not have the requisite technology or skills for such a system will not be able to use it.

Who (the vendor or MDH) will have responsibility for maintaining the shared measurement system (e.g. monitoring data submission quality, responding to grantee questions, extracting data for EHDI and other programs)?

The evaluation capacity building provider will have all these responsibilities. MDH will own the data, and expects the provider to provide MDH with an analysis-ready database. From time to time, EHDI staff may ask the vendor to provide summary results from the grantee data, but this will not require extensive analysis.

To what extent can the EHDI grantees be involved in the development of the shared measurement system?

MDH expects grantees to be involved in the development of the system to the extent that they will provide feedback on what shared outcomes would be useful to them and which outcomes they would be capable of reporting year after year, in addition to the reporting system format (paper or online). Input from MDH content experts on the shared outcomes must also be incorporated.

Grantee Selection Process

Would the selected vendor also be able work as the contracted external evaluator of EHDI grantees?

MDH reserves the right to approve any potential subcontractors of an EHDI grantee before that grantee enters into a binding agreement to fulfill any of its obligations or duties under the grant agreement. MDH does not anticipate approving any subcontracts between the evaluation capacity building provider and a current EHDI grantee.

Would the selected vendor be able to subcontract with other organizations/consultants for specialized TA support that might be identified through the initial capacity assessment?

Yes. However, MDH reserves the right to approve any potential subcontractors of a grantee before entering into a binding agreement to fulfill any of the obligations or duties under the grant agreement.

Other

What, if any, additional legislative mandates must be considered in the evaluation beyond those listed in the statue included in the RFP? Will the vendor have any responsibility for preparing the final legislative report?

There are no other legislative mandates. MDH’s Center for Health Equity is responsible for preparing the EHDI legislative report.

Would MDH consider changing the January 25 timeline for grantees to submit logic models and evaluation plans so that the vendor can interact with and support the grantees before these deliverables are developed?

No. However, the logic model and evaluation plans submitted on January 25, 2017 are considered drafts. Grantees have been encouraged to work on refining their logic model and evaluation plan when the evaluation capacity building provider comes on board. If grantees refine their logic models and evaluation plans after the deadline, they should be resubmitted to MDH as necessary.

M