STATE OF WISCONSIN
This memo provides an explanation of the local assessment process for county child welfare programs that will be conducted in spring 2001, including the purpose of the assessment, a description of the process, and the expected timeframes for completing the process. The memo provides an overview of the child welfare assessment process so counties can begin planning how they will conduct their local process. Specific instructions for the process will be issued in a subsequent numbered memo.
The child welfare assessment process is a comprehensive analysis of local child welfare program operations and program outcomes by county child welfare agencies with broad participation by community stakeholders. The outcomes include key aspects of child safety, permanence and child and family well- being. The local assessment process begins the implementation of a continuous improvement approach by local child welfare programs with the Division Children and Family Services (DCFS) to assure the quality of services and improve outcome achievement at both the local and state level.
The local assessment process includes a program overview to obtain information from child welfare agencies on local program operations and uses community discussions with key stakeholders to examine program outcomes and factors affecting those outcomes. The program overview will allow for the development of a comprehensive summary of child welfare program operations that identifies how local practices affect service delivery. The community discussions will provide a mechanism to obtain greater community participation and responsibility in the delivery of child welfare services. DCFS will provide individual county data profiles using abuse and neglect and out-of-home care data submitted by counties so the assessment process can focus on county-specific program outcomes. The assessment process covers the broad scope of the child welfare program, including both child protective service and juvenile justice use of out-of-home care. The assessment process was piloted in 10 counties in fall 2000 and the experience of the pilot counties is being used to design the final assessment process.
The child welfare assessment process will begin in March 2001 and be completed by June 2001. DCFS will issue instructions for the process and the data profiles in March and hold several regional roundtable sessions to provide guidance on how to conduct the local process. Extensive communication efforts are being undertaken to make community stakeholders aware of the process and obtain their active participation. Counties are encouraged to begin planning for the process, including establishing their local leadership team to provide direction for the community discussions. Counties will be given considerable flexibility in how to conduct the process so that the assessment can be individualized based on how counties operate their child welfare programs. Counties can also incorporate items of local interest into the assessment and should take ownership of the process so it meets local as well as state needs.
Once the process is completed, the assessment results should be used for local program management purposes to improve service delivery and local program outcomes. Counties will be expected to use the assessment results in developing their local program plans for state funds in future years. The assessment results will be used to design training, technical assistance and program monitoring strategies. DCFS will also use the assessment results in state-level strategic planning, establishing budget priorities, and setting the legislative agenda.
Purpose and Benefits of the Assessment Process
The child welfare assessment process is being conducted for several purposes, including outcome-based program planning, engaging community stakeholders and preparing for a federal review, with each of these purposes having immediate and continuing beneficial impacts for the child welfare program.
Good program planning requires the examination of program outcomes to identify whether services are working and to set goals for improving the effectiveness of program services. For the child welfare program, it is important to evaluate outcomes in the areas of child safety, permanence for children in out-of-home care and child and family well-being using data on key program indicators. The assessment process will encourage outcome-focused planning practices at the local level and DCFS will provide county data profiles on an annual basis so that local agencies can see year-to-year changes and analyze the effect of services strategies on program outcomes. The outcome focus for planning will lead to improved services to program clients and more effective use of program resources.
The child welfare program involves a number of organizations and individuals in achieving safety, permanence and well-being for children and families. While county child welfare agencies play the lead role, the program has extensive involvement by courts, prosecutors, tribes, law enforcement, service providers and other organizations and individuals to deliver program services and meet program requirements. Maintaining effective working relationships with these stakeholders depends on educating the stakeholders about child welfare program services and obtaining their active engagement in service delivery and collaboration activities. The assessment process will provide an opportunity to educate stakeholders about their impact on child welfare program outcomes, engage stakeholders more and improve working relationships with county child welfare agencies.
The federal government is conducting comprehensive children and family service reviews of state child welfare programs, examining state outcomes and evaluating the service delivery capacity of states. The first federal review of Wisconsins child welfare program is scheduled for 2003, and since child welfare services are operated primarily by county agencies, it is essential to understand local program operations and have local agencies assess their program outcomes for the state to prepare for the federal review. DCFS will use the assessment process results to analyze the factors affecting program outcomes and design statewide program improvement strategies, including ways to better support local agencies through training, technical assistance and improved program data reporting. DCFS will continue to review local program outcomes and work with counties to make continuous improvement in the child welfare program.
Pilot Experience with Assessment Process
The child welfare assessment process was piloted in 10 counties during the period of October through December 2000. The pilot counties included Green Lake, Jefferson, La Crosse, Marathon, Outagamie, Racine, Rusk, Sauk, Waukesha, and Waushara. The pilot counties completed an agency overview and conducted community discussions to test the assessment process, including providing feedback on the outcome data, assessment tool and how to conduct discussions. This feedback is being used by DCFS to design the final assessment process, including the data profiles, assessment tool and instructions for the community discussions.
The pilot counties found the assessment process an effective way to educate community stakeholders, establish or enhance collaboration mechanisms, and understand how local program structure and service delivery strategies may affect program outcomes. Based on feedback from the pilot counties, DCFS is clearly defining and packaging the data profiles to more easily show county-specific outcomes and facilitate comparisons between counties. DCFS is also separating the agency overview and community discussion portions of the assessment process, redesigning the assessment tool, and providing additional guidance on how to conduct the community discussions. The efforts of the pilot counties in testing the process is sincerely appreciated by DCFS and will make the process more beneficial for all counties statewide.
County Data Profiles and Data Sources
The data profiles will include county-specific information on child safety and out-of-home care program activity, the nature of clients served and program outcomes. The profiles will contain caseload and other context information showing the volume of child welfare program activity for both protective services and juvenile justice (excluding placements to juvenile corrections) clients. The profiles will contain program outcome information relating to child safety, permanence, and out-of-home care utilization computed in accordance with how the federal government is computing state program outcomes for federal performance measurement purposes. The profiles will also contain demographic and economic information about counties to facilitate comparisons.
The county data profiles are designed to give counties information on how county-specific outcomes compare with state averages and allow the community stakeholders to participate in discussions on how the demographics of population served, the structure of child welfare program operations and local service strategies might affect local program outcomes. The local assessment results will assist DCFS to analyze the factors influencing program outcomes statewide and develop strategies for improving program outcomes in those areas where the state may not meet federal performance standards. The county data profiles will be produced on an annual basis so that counties can continue to analyze local program trends and use outcome data in their local program plans for state funds provided by DCFS.
The primary data sources for the data profiles will be child abuse and neglect (CAN) information reported to DCFS using CFS-40 forms and out-of-home care information reported in the Substitute Care Module of the Human Services Reporting System (HSRS). Information from other sources may also be included in the profiles. Counties will be given discretion on how to use the data profiles for the assessment process and can use local program data to supplement or substitute for the state data. The data will be for calendar year 1999 with several years of out-of-home care caseload data provided and out-of-home care outcomes computed for 1997-1999 since outcomes can fluctuate from year to year. DCFS recognizes there are issues with the quality of CAN and HSRS data and will include caveats in the data profiles. However, the timeliness and accuracy of local data reporting is becoming more critical as local program outcomes will be computed by DCFS on an annual basis.
Child Welfare Agency Overview
The agency overview portion of the assessment process is designed to capture information about how counties operate their child welfare program and deliver services, including child welfare agency structure, local policies and practices, and service delivery strategies. The results of the agency overview results will be used by DCFS for state-level strategic planning and to prepare for the federal review. The results will be shared with the University of Wisconsin Training Partnerships to identify training needs and develop training curriculum for local agency staff. The results will also be used to develop technical assistance and program monitoring activities by DCFS, Area Administration and other sources. Following the completion of the assessment process, DCFS will produce a comprehensive report on local child welfare program operations that will be shared broadly with local agencies to provide information on child welfare program performance and how local programs are operated statewide.
In addition to questions about child welfare agency structure and operations, the agency overview is also designed to address specific aspects of child welfare program services. These areas include how certain federal and state child welfare program requirements are implemented at the local level along with working relationships between the child welfare agency and key organizations such as the courts. Based on feedback from the pilot counties, some questions pertaining to detailed outcome data will be addressed in the agency overview rather than the community discussions portion of the assessment to allow the community discussions to concentrate on issues of broad interest. While child welfare agency staff can complete much of the agency overview on their own, parts of the overview will require consultation with other organizations or individuals. Counties are encouraged to work on the agency overview at the beginning of the assessment process as there may be some aspects of the overview that may be beneficial to share with stakeholders in the community discussions or otherwise be useful in framing how the discussions should be conducted.
Community Discussions with Stakeholders
The community discussions are designed to engage key stakeholders in examining child welfare program outcomes and talking about factors which influence program outcomes in the county. These factors can include the demographics of the population, the structure of the local child welfare program, the degree of collaboration between agencies, local service strategies, the availability of service providers, and a number of other factors. The discussions are intended to involve stakeholders in an open, constructive dialogue about the safety, permanence and well-being objectives of the child welfare program and help stakeholders realize what impact their activities have on child welfare program outcomes. The discussions should lead to increased awareness of child welfare program issues by stakeholders, greater interest in collaboration mechanisms, and stronger cross-system efforts to improve service quality and outcome achievement.
Counties will be given considerable flexibility in conducting the community discussions in terms of the type and number of stakeholders to participate in the discussions and the manner in which the discussions are held. Many counties may have existing advisory groups that can be used for the community discussions or ad hoc groups can be established. These groups often include the key stakeholders and are typically already involved in local program planning. Each county will establish a leadership team to determine how the discussions will be conducted.
DCFS will provide an assessment tool with suggestions on how to organize the discussion groups and a standard list of questions for groups to discuss. County child welfare agencies will need to provide the stakeholder participants with background information about the local child welfare program, an orientation to the assessment process, and interpretation of the county data profiles and any local data regarding program activity and program outcomes. It is also recommended that county child welfare agency staff actively participate in and facilitate the discussions to respond to questions regarding terminology, child welfare program requirements and other questions that stakeholders may have.
Once the community discussions are completed, child welfare agencies will be expected to submit a written response to the items or questions in the assessment tool. This assessment response should be shared broadly at the local level to help educate the community about child welfare program issues and guide local agencies in planning for their child welfare programs. DCFS will expect counties to use the assessment results in planning for the use of funds received from the state, including setting priorities for the use of funds and establishing local outcome measures to determine the effectiveness of child welfare services. DCFS will also use the assessment results to develop program improvement strategies at the state level, including setting targets for program outcome achievement. The assessment results will be made available at the state level by posting county responses to the Department Web site.
Child Welfare Outcome Areas
The assessment process will examine outcomes relating to child safety, permanency, utilization of out-of-home care, and child and family well being. These outcome areas parallel the areas that will be considered in the federal children and family services review. The outcomes considered, however, will not be limited to only federal performance outcomes as there are a number of areas of state interest. Counties can add items of local interest to the assessment process as well.
The following are examples of issues that will be explored in the assessment:
Counties can establish discussion groups on each of these outcome areas or have discussion groups cover multiple topics depending on which stakeholders will be involved and local preference on how to conduct the discussions. The data profiles include caseload and outcome data that can be considered by the discussion groups to discuss specific questions in the assessment tool. Other questions, such as those questions relating to well being, are more open-ended in nature and are not linked to information in the data profiles. While the assessment process is designed to cover both child protective services and juvenile justice (excluding placements to juvenile corrections), each county will need to define the scope of their local child welfare program in order to interpret their program outcomes.
Using the data profiles, counties will be encouraged to compare their local program outcomes with state averages and outcomes in other counties. While counties will not be expected to justify the level of their outcomes in the assessment response, the use of comparisons will help to identify factors that influence the level of the local program outcomes and suggest other counties that agencies could contact for information on their child welfare program practices.
Role of the Leadership Team
Each county will need to establish a leadership group to provide local direction for the assessment process. At a minimum, the leadership group should include the county human/social services director, a childrens or juvenile court judge, a local district attorney or corporation counsel who handles child welfare cases, or their designees. The leadership team can also include other key individuals, such as county board members, chief law enforcement officers and school superintendents, who can provide leadership to the local assessment process. County directors are encouraged to talk with these key individuals about the assessment process so the leadership team is ready when the process starts in March.
The leadership team will determine the format for the community discussions, recruit stakeholders to participate in the discussions and set the parameters for the discussions. The leadership team will determine who will respond to the agency overview portion of the assessment process. The leadership team will also be responsible for approving the final written assessment response for submission to DCFS.
Role of Child Welfare Agency
County child welfare agency staff will play a critical role in the assessment process by completing the agency overview, reviewing and supplementing the data profiles, preparing and facilitating the discussion groups, and completing the outcomes portion of the assessment tool. While the assessment process will create significant demands on staff time, the active and willing participation by staff is crucial to the success of the assessment process. The long-term benefits of greater community engagement in the child welfare program and improved understanding of program outcomes for local planning purposes will make the effort worthwhile. The assessment process also provides an ideal opportunity for staff to address issues of local interest with the stakeholder groups.
In recognition of the workload associated with the assessment process, DCFS will limit other activities during the assessment period that would create competing demands on county staff time. The data profiles, assessment instructions and assessment tool will be available for downloading from the Department Web site or distributed via e-mail to simplify the assessment response.
Timeframes for Assessment Activities
The assessment process is expected to begin in approximately mid March with the release of a numbered action memo providing instructions for the assessment process and the assessment tool along with release of the county data profiles. Counties will have the period of mid March through mid June 2001 to conduct the local assessment process, including completing the agency overview and the community discussions. Assessment responses will be due to the Division by the end of June.
The Division is holding a series of regional roundtable meetings in mid to late March timed with the release of the action memo to explain the assessment process to leadership teams, child welfare agency staff and other interested persons and respond to questions on how to best conduct the process. Meeting dates and locations are attached. In addition, Area Administration staff from the Departments Office of Strategic Finance will be available to assist counties to begin their assessment process and those staff will periodically check on progress by counties in their region over the course of the assessment period. DCFS staff will also be available throughout the assessment period to respond to questions and provide technical assistance on the use of the data profiles and assessment tool.
Extensive communication efforts are being made at the state level to make key stakeholders, particularly those needed on the leadership team, aware of the child welfare assessment process. Information about the assessment process will be shared with judges and other court staff statewide through the Director of State Courts and district court administrators. Information will also be shared with district attorneys and corporation counsels. Efforts will be made to share information with other stakeholders through their newsletters and other means.
DCFS has worked closely with the Wisconsin County Human Service Association (WCHSA) in developing the assessment process and will continue to share materials with WCHSA for review prior to issuing the assessment materials in final form.
DCFS is establishing a state steering committee consisting of the same stakeholder groups who will be involved at the local level to advise the Division on how to use the results of the assessment process for state-level program planning, including setting goals for program improvement.
Recommended Preparations and Next Steps
Counties are encouraged to make the following preparations for the child welfare assessment process: