Evaluation Process for Adopting
New Content Areas
MinnesotaTracking Program uses a hierarchical evaluation process to guide development and adoption of new state-specific content areas for Tracking. This process provides a systematic and transparent approach to help focus valuable Tracking resources on areas that are actionable, feasible, and useful for protecting public health. The Tracking Program uses this process to develop Tracking content areas that focus on measures of environmental health hazards, people’s exposure to hazards, and health effects or diseases. For questions or more information, contact us.
Information on this page:
- Why use an evaluation process?
- What are the phases of the process? What criteria are used?
- Who provides input?
- How was the evaluation process developed?
- What is the estimated timeline?
- Where can I find more information?
This evaluation process helps to ensure that valuable Tracking Programresources are used wisely, and aimed at areas that are actionable and that protect public health. The process requires a sound, defensible rationale to support new content areas before they can be adopted.
Developing and updating new content areas for Tracking is resource- and time-intensive. Data and measures are developed and piloted, and then updated and maintained over time. These activities require ongoing dedication of a highly trained, multi-disciplinary team with expertise in epidemiology, data and spatial analysis (GIS), and health risk communication (in addition to web, application, and database development and support).
This evaluation process also helps to build and sustain partnerships with data stewards and other Tracking Program partners by establishing a clear and transparent process. Opportunities to provide input along the way help build a strong rationale and broad support for sustaining Tracking content areas over time.
The Tracking Program uses a four-phase, hierarchical process to ensure that new Tracking content areas meet minimum (required) criteria at key decision points. The four phases are:
- Exploration: explore rationale and public health importance; determine if resources are available; find available datasets
- Feasibility: pilot the data and measures; evaluate quality of datasets
- Recommendation: final assessment of overall content area; explore stakeholder interest, economic impact, and utility of measures for informing public health
- Implementation: incorporate content area as a continuing Tracking measure; develop data and measures, messaging, and how-to guides
The evaluation process may halt or suspend at any phase, depending on whether criteria for moving ahead are deemed to be met. Tracking Program documents each phase of the process using the Evaluation Process Template (PDF: 681KB/3 pages).
Phase 1 Criteria:
- Available Resources: are financial and technical resources available?
- Prevalence: is a high proportion of the population exposed to the environmental health hazard OR is there a high estimated prevalence of disease or outcome associated with the hazard?
- Causality: is there evidence that exposure to the environmental hazard(s) is associated with the adverse health outcome(s)?
- Public Health Impact: what is the public health impact of exposure or severity of disease(s)?
- Actionability: can the level of exposure or disease be modified through policy, regulatory, or personal actions? Can these new data and measures be used to develop new initiatives?
- (Initial) Feasibility: does a data source exist to create a Tracking measure?
Phase 2 Criteria:
- Detailed Feasibility: is the data of high quality (e.g. population-based, representative)? Is data collection continuous and timely?
- Pilot the new measures: how would the data source(s) be used to create new Tracking measures?
Phase 3 Criteria:
- Emerging Issues: is the level of exposure or disease changing?
- Information Building: is the prevalence or level of exposure unknown?
- Outside Interest: is there a high level of state and local agency interest?
- Balance: would there be balance among the types of content areas tracked (i.e., hazard vs. disease) and among age groups affected?
- Economic Impact: what is the cost to the Tracking Program and partners (e.g., healthcare systems, industry) of adding this content area?
Throughout the evaluation process, Tracking Program staff consult the interagency Technical Team (composed of content area experts and data stewards) and the external Advisory Panel for input on criteria and rationale. Using this input, Tracking Program Management decides whether the criteria support further exploration in the next phase.
Ideas for Minnesota-specific content areas and measures may come from a variety of sources, including staff, Advisory Panel members, the public, non-governmental organizations, local public health officials, or legislators and policymakers. The Tracking Program's Advisory Panel is informed and consulted after each phase is completed. Work may halt if the rationale for pursuing a content area is judged not robust.
In the final step, Program Management decides whether or not to adopt a new content area considering the strength of the rationale, resources, and program goals/priorities.
The Tracking Program reviewed existing models and criteria, developed a list of evaluation criteria, and then placed criteria into a hierarchical evaluation process. The Tracking Program obtained input from the Technical Team and Advisory Panel, and then developed the Evaluation Process Template (PDF: 681KB/3 pages).
The Tracking Program also piloted the process with Environmental Tobacco Smoke (ETS), a new state-specific content area (see Summary of Evaluation Criteria for ETS (PDF: 695KB/8 pages)), developed a how-to guide and implemented new measures for display on Minnesota Public Health Data Access in February 2012.
The evaluation process may take approximately 4-12 months, using the following estimates: Phase 1 (1-4 months), Phase 2 (1-6 months), and Phase 3 (1-2 months).The timeline depends on staff workload and the complexity of issues or challenges encountered during the evaluation process. For example, delays between phases may occur so the appropriate input can be provided and revisions can be made.
- Poster: A hierarchical evaluation process to adopt new content areas for Tracking in Minnesota (PDF: 155KB/1 page)
- Summary of Evaluation Criteria for ETS Exposure (PDF: 695KB/8 pages)
- Evaluation Process Template (PDF: 681KB/3 pages)