Play 4: Prioritize
Last updated
Was this helpful?
Last updated
Was this helpful?
Preparing your data for sharing with other departments requires staff time and resources; this step focuses and prioritizes data-sharing efforts on data that yields the most benefits to your department and data consumers from other organizations. Starting with a single priority dataset delivers benefits more quickly than an expansive effort across all your department’s data and better utilizes staff resources. Additionally, an incremental approach allows your team to “inspect and adapt” with each iteration to improve efficiency.
This Play provides general guidance to:
Establish a prioritization rubric.
Gather information to prioritize data-sharing efforts.
Obtain executive approval to launch data-sharing improvements.
Establishing your prioritization rubric before initiating data prioritization research helps you collect the information needed to rank your department’s datasets during dataset analysis and discovery sessions.
The dataset is provided under a current BUCP or other data-sharing agreement. The dataset is being requested through a BUCP that is under review.
Data recipients will immediately benefit from your enhanced metadata, such as business definitions.
Security classifications and statute citations help address Specialized Security and Privacy fields.
The dataset is commonly requested by other departments. The program supported by the dataset is interconnected with programs in other departments. The program supported by the dataset interconnected with programs within your department. The program supported by the dataset is related to agency-level initiatives.
Indicates future demand to provide the dataset.
Your department heavily uses the dataset in reports and analytics.
The improvements to data-sharing benefits report and analytics creation by your department.
The system that maintains the dataset is a candidate for modernization.
The resulting data architecture created by improved data-sharing supports the Project Approval Lifecycle (PAL) and new system design.
The CalHHS Data Playbook also contains several plays to help you prioritize your data listed below:
Your prioritization criteria will have different levels of importance. For example, a dataset already provided under a BUCP may have more significance than a dataset potentially supplied in the future. Additionally, datasets shared between programs in your department provide a direct return on investment in data sharing.
To help score datasets, you can establish weights to your prioritization criteria. This approach creates a numerical score to help assess data-related priorities.
Consider Creating Data Subsets
Not all the data in a database may be a priority for data sharing. For example, data requestors may only need a portion of the data stored in a database related to participants, programs, and participation. To reduce your data-sharing improvement effort, consider creating curated datasets focused on commonly requested data.
Past BUCPs and other data-sharing agreements are a source to identify specific tables or files to focus your efforts. Review the BUCPs you collected in Play 1: Establish Data-Sharing Metrics and BUCP Tracking to identify data previously requested to identify data subsets.
After receiving executive approval and launching a parallel effort to create the CDW metadata repository, Sally assembles a working group to create the department’s data prioritization rubric. The department executives selected the following staff members to participate in creating the data prioritization rubric:
Ann represents the Healthy Habits program.
Angelica represents the Walk 2 Work (W2W) program.
Pankaj represents the Your Environment program.
Carly is the CDW Chief Information Officer (CIO).
Danny is the manager of the CDW data analysis unit.
These participants represent a distribution of program, data, and information technology interests and needs.
The working group meets to identify the criteria and weighting to prioritize the department’s data. After the discussion, the group determined that the Play’s example criteria are a good starting point. The group plans to reevaluate the criteria for their effectiveness after prioritizing the CDW’s data.
Next, the working group defines weights for each criterion. They adopt a numeric scale from one to five as weights for the criterion. The weights are then totaled to create a ranking of the priority for each data set. The workgroup creates the following weights:
The W2W and Your Environment programs have plans to share data to identify cross- program impacts. Angelica and Pankaj work with the group. They advocate that datasets used across CDW programs should receive a score of five as the effort directly benefits the department. The workgroup agrees to this weight factor.
The CDW is increasingly receiving data requests from other departments. The group agrees to assign a weight of four if other departments frequently request the dataset.
Carly, the CDW CIO, notes the department has several forthcoming modernization efforts. She requests that datasets created by systems with scheduled modernizations receive a score of three.
The remaining factors will be evaluated during the final decision-making process by executive management.
Sally documents the criteria and weighting factors in the data inventory the team created in Play 2: Identify Your Datasets. During data prioritization analysis, the team will refer to criteria during research and discovery sessions.
Your approach to gathering information to prioritize your data will vary depending on the selected criteria. The list below provides examples based on the example criteria provided in Play 4.1: Establish a Data Prioritization Rubric:
Review program initiatives and strategic plans to identify cross-program relationships. You want to validate prioritization provided by document reviews by interviewing department Executives and Program Managers. Discovery sessions provide an effective mechanism to help further your understanding of priorities and achieve executive buy-in.
that identifies datasets that are frequently requested or part of a current BUCP.
Review your department’s project portfolio to identify systems that have future modernization efforts.
As you gather information, retain your notes and supporting documents for later reference. Summarize your notes into key points for future reviews by the data prioritization workgroup.
While collecting information on the CDW dataset, Sally took some initial notes to start the prioritization process. These notes are a starting point, but additional information is required.
Sally decides to use discovery sessions with the primary CDW program teams and executives to gather additional information to prioritize the CDW datasets. Sally acts as the facilitator while Carlos takes notes on the meeting. Dividing facilitation and note-taking allows Sally to focus on questions that elaborate additional information. Carlos benefits from note-taking by learning more about the CWD’s programs. She schedules individual discovery sessions with the Walk 2 Work, Healthy Habits, and Your Environment. Sally uses the notes she took during the data inventory effort and program documentation to develop questions that will illuminate the value of program data, including:
The importance of data to the program’s mission.
The relationship of the program to:
The department’s strategic plans.
Other CDW programs.
Other departments and organizations.
The frequency by which data is shared with other departments.
Sally packages the meeting notes and sends them to the respective program teams. This allows the teams to make corrections, ask questions, and further clarify ideas.
Sally then reviews the BUCP tracking repository her team created earlier that year. She runs a report and identifies that the Walk to Work program has received the most data requests. She records this information in the data inventory.
Once you have gathered prioritization information, perform a test of the criterion created in Play
4.1: Establish a Data Prioritization Rubric and the dataset information gathered in Play 4.2: Gather Information to Prioritize Your Data. The test of the criterion identifies potential adjustments to suggest to the stakeholders that approve the data prioritization scoring.
Send the summary of key points from your research to the data prioritization workgroup for review ahead of any meetings. Sending information ahead of meetings ensures participants are informed and prioritization meetings are effective. You may also want to provide access to your detailed notes to give workgroup participants more insights about the findings.
Schedule a meeting with your data prioritization workgroup. Designate a facilitator who will not participate in creating recommendations. Use the criteria and weights to make an initial prioritization ranking. The workgroup can also include qualitative factors in its recommendation to supplement your ranking system.
With the data prioritization recommendation complete, it’s time to present it to executive management. Create a PowerPoint presentation that contains the following information:
Purpose of the data prioritization effort.
Members of the data prioritization workgroup to remind leadership who was allocated to the effort.
Overview of the rubric to provide the basis of the recommendation.
Anticipated data-sharing benefits from the highest priority dataset.
Summary of the data prioritization recommendation.
List of staff resources required to move forward with the effort.
Ask executive leadership to review the data prioritization recommendation and seek their approval to move forward. You may get approval during your review meeting or need to schedule a follow-up to allow time to review the team’s recommendation.
You can start executing Play 5: Establish Your Metadata Repository while executive leadership reviews the data prioritization recommendation and data-sharing improvement request.
Sally and Carlos distribute summaries of their notes to the data prioritization workgroup. They schedule a meeting with sufficient lead time for the workgroup to review notes and ask questions.
During the data prioritization meeting, the workgroup uses the rubric to evaluate each dataset using the criterion and weights to rank the department’s datasets. The weights result in the following dataset priority ranking:
Walk 2 Work
Healthy Habits
Your Environment
The workgroup discusses the agreed-upon criterion to validate that the ranking accurately reflects the department’s priorities. The team documents the recommendation to use the W2W as the first program for data-sharing improvements based on the following findings:
A current BUCP is under review to supply the W2W data to the Department of Mass Transit.
Data sharing with the Healthy Habits program will identify the positive impacts of W2W on physical activity programs.
Other departments frequently request the data.
The resulting data dictionary helps create some of the PAL Stage 2 artifacts including input for Mid-Level Requirements (MLR), reference architecture, and data conversion plan.
Sally schedules a meeting with the data prioritization workgroup and executive management to present the data prioritization recommendations. Sally presents the following to executive leadership:
Overview of the rubric used for data prioritization.
Ranking of the data based on the criterion and weights.
Relationship of the dataset across CDW and CalHHS programs.
Program impacts from data sharing captured during discovery sessions.
After review, the CDW executive management agrees with the workgroup’s recommendation to prioritize data efforts on the W2W program.
With W2W selected as the priority dataset, Sally and her team start to collect detailed metadata to improve data understanding and assist with BUCP agreements.
With your goals and strategy successfully outlined, you can now think about what data or measurements you need to collect to answer your guiding questions, as well as the data you need to determine if you are ready to proceed with data collection. If you’re a manager, you’ll also need to define your outcome measures and performance/self-assessment metrics to maintain the integrity of your project and ensure you’re supporting your team and stakeholders as best you can.
Before proceeding, you should go through a Readiness Checklist to ensure you’ve considered your own strengths, weaknesses, and that of your manager and team. Get the support or learning you need now to prevent misunderstandings or frustrations later in the process.
Ask yourself: Do you have the Support, Knowledge, and Resources to Complete your Project?
Do my managers/directors have the bandwidth to support me?
Do I/my team have enough expertise to complete this project?
Do I have access to the data I need to complete the project?
Do I know the statistical methods required to analyze my data?
Who is my department’s Data Coordinator? (The individual responsible for knowing the data assets held by your department)
Contact CHHS@osi.ca.gov to find your Departments Data Coordinator
Your program data is the core data of this project — it’s the specific measurements that you need to collect in order to answer the project’s guiding questions. As a review, your guiding questions are the purpose of this project as a whole, and spending some time thinking about your project’s purpose statements will help you determine what data you need and how you should collect it.
Example Purpose Statements:
I need to decide how to allocate resources to different programs based on which is the most successful
I want to improve or refine an existing program or model to be more effective
I want to create product or service that positively impacts a community
I want to look at existing data to find trends and patterns that people care about
It can be useful to review all your data assets with these questions in mind. Contact your department’s data coordinator for more information about the types of program data you collect in your department by emailing CHHS@osi.ca.gov.
Managing a team at CHHS is challenging — in addition to setting and working toward your program goals, you must also assess the performance of your team and support their continuing learning; set the broader goals that guide larger initiatives, programs, or departments; and work toward capacity building in analytics, data literacy/governance, and much more. The following section is written for a wide range of manager roles, including the larger cohort of managers who supervise analysts and technical employees (SSM1s) to the smaller cohort of branch-level directors or managers working on capacity, vision, and strategy of their department.
As a manager, you may be in charge of managing the overall performance and strategy of the project or program; you also may need to assess the performance of the team itself, and the department’s resources. This requires defining and measuring outcome data, monitoring your team’s or program’s performance, and assessing your department’s current data assets and analytic capabilities.
The following section contains a number of frameworks and resources to assess your Team’s Capabilities… …related to projects and programs
Assessing Readiness: considering the scope, risks, limitations of your data project
Measuring Performance: Setting Key Performance Indicators (KPIs) for the project and your team
Determining Outcome Measures: benchmark, baseline, and comparative data …at the department level
Strategic Use of Data: how effectively does the department utilize data to inform decisions and strategy?
Capacity Building: Improving internal capacity, assessing management strategy & organization
Data Governance & Management: Management & Security of Data, Improving Data Literacy, data de-identification guidelines
Before planning your data collection, go through the following readiness checklist to ensure you are capable of successfully carrying out this data project. You should catalog your assets and resources regularly throughout your project to identify areas of weakness or gaps in resources.
How do programs or stakeholders use data currently? What do they do with it? How do they use it to make decisions or produce products for external stakeholders?
What are limits to either the data or the implementation solution?
What are the risks and issues with the current data? What value is not being realized?
Identify the current workflow for collecting, processing, and publishing data. Are there dependencies to collecting, processing, and publishing the data?
Remember, if you do not have the resources you need, you and your team will likely encounter problems in your data project. Address weaknesses early and be on the lookout for areas you can improve throughout your project.
Effective Budgeting and Financial Planning practices driven by data
Assessing organizational strategy and goal-setting
Measuring accountability at all levels of your team
You may also be tasked with assessing the quality of your department’s data management and data governance, or working on capacity-building frameworks to improve data literacy and analysis skills.
Harvard’s Strategic Use of Data Self-Assessment Guide has specific questions to identify where departments can better use data at the organizational and strategic level
Your dataset inventory created in is the foundation to prioritize data-sharing improvement efforts.
We recommend executing Play 4 in parallel with to have capabilities for an immediate launch of after completion of your data prioritization effort.
A data prioritization rubric provides a standard set of characteristics to create consensus among department stakeholders about where to focus data-sharing improvement efforts. In , we recommended obtaining approval from executive management to collaborate with data stakeholders to establish a data prioritization rubric. This approach creates an organizational-level perspective to prioritize your department’s data.
The table below provides sample characteristics to incorporate into your data prioritization rubric. Collaborate with the departmental staff selected to prioritize your department’s data in to identify additional criteria and rankings. Collaboration improves your rubric and helps establish buy-in during the data prioritization ranking selection conducted later in this Play.
One option is to create curated datasets to reduce the scope of work while receiving data- sharing benefits. A curated dataset is a collection of datasets that are selected and managed to address specific data consumer needs and business questions. The Medium article introduces data curation. The supplemental section, Additional Training and Reference Materials references online training materials that provide training on data curation.
Use your BUCP tracking system and repository created in .
Sally and Carlos met with the CDW Project Management Office to get a list of the department’s forthcoming modernization efforts. The Walk 2 Work system is currently in the California Department of Technology (CDT) (PAL) Stage 1 and is scheduled to enter Stage 2 later this fiscal year. Carlos meets with Carly, the CDW CIO, and Andrea, the IT manager. He learns that the improved data dictionary will help with Stage 2 deliverables and be useful for the new system design.
This is the data you need to collect after deploying your product or service to determine whether or not it met your goals and was successful. A useful framework to reference is the Key Performance Indicators (KPIs) framework described . KPIs measure your performance relative to your goals.
Check out to learn all about KPIs: what they are, why they work, and how to set them effectively.
It is imperative for managers to regularly assess and improve how effectively they use their data assets to inform their strategic planning and organizational structure, as well as improve their offered programs and services. We will root our assessment in Harvard’s , a useful framework for understanding how strategically your department uses data and how to improve. A few examples from the guide:
For managers interested in these types of assessments, check out additional resources on building Capability and Capacity in your department (such as the .)
Note: For more concrete recommendations to build analytic capacity, check out this . It will cover:
Harvard’s has specific questions to identify where departments can better use data at the organizational and strategic levelHarvard Assessment 1Harvard’s Strategic Use of Data Self-Assessment Guide has specific questions to identify where departments can better use data at the organizational and strategic level
