If you want to know how well welfare programs work, ask welfare agency administrators how they measure success. This was suggested by Randy Hicks, President and CEO of the Georgia Center for Opportunity (GCO), years ago. Almost invariably these administrators will answer that they measure success by how many people they serve. When the total number of people they serve goes up, the programs are more successful. Or are they?

To the contrary, program participation does not measure success. Furthermore, the chances are that welfare agency administrators lack the metrics to tell us how successful the programs truly are.

Program participation can measure demand for the program, or it might indicate the number of people in need. In these cases, program participation is useful information. But does it actually measure success? 

The more important goal of welfare programs is to help people overcome their financial difficulties and escape poverty. This enables them to live more fulfilling lives. Public policy should not encourage them to languish on assistance for years on end but rather help them improve their circumstances until they no longer need assistance, or their reliance on assistance becomes lessened. Welfare agencies generally lack metrics to effectively measure this important goal.

Which revises our original question slightly: How can you measure success?

Dependency Metrics

One potential way to measure success is to use dependency metrics that evaluate the percent of the population who are dependent on major welfare programs. This is partially done at the federal level but not at all at the state level.

In 1994, Congress passed the Welfare Indicators Act. It focuses on food stamps, Temporary Assistance for Needy Families (TANF) cash grants, and Supplemental Security Income (SSI). Every year, the U.S. Secretary of Health and Human Services is required to file a report with Congress showing dependency on those three welfare programs.

The most recent report was released in 2018. The pie chart below comes from page eight of that report, showing for the year 2015 the percentages of the national population according to their proportion of their total income dependent on the value of food stamps, TANF cash grants, and SSI. The higher the proportion of an individual’s income that comes from these three assistance programs, the worse off the person probably is. For example, if the value of food stamps constitutes more than 50 percent of an individual’s income, that person cannot be well off financially. In comparison, when food stamps constitute 25 percent to 50 percent of an individual’s income, it means the person has more additional income and is better off than when food stamps comprise more than 50 percent  of total income. And having less than 25 percent of total income coming from food stamps is better than having 25 percent to 50 percent of total income on food stamps.

Georgia has the ability to generate dependency metrics through the Georgia Gateway, including TANF cash grants, food stamps, medical assistance, and two other programs. These are means-tested programs, meaning the Department of Human Services has not only participation numbers but also income information of the applicants and recipients. The Department could relatively easily have its I.T. crew write scripts to spit out reports periodically showing the number of individuals and families by dependency on their income on those programs captured through the Gateway. Coupled with Census data, the Department could produce periodic reports showing how dependency changes over time and further break down the data by demographic groups. 

Furthermore, because every individual has a unique identifier, the I.T. crew could produce additional scripts to follow people over time. This would allow for more sophisticated analytics showing the financial progress of people and families in the system. 

Dependency metrics are not perfect. They do not capture persons who would be eligible for the program but do not participate. However, the number of these individuals are regularly estimated and could be presented as additional information in the analysis. 

Ideally, it would be best if the dependency metrics captured all assistance programs. Currently, this is not possible.

Exactly How Many Programs Do People Benefit From? 

Often people qualify for multiple assistance programs. Their children might be on Medicaid and receiving free school lunches. At the same time, the household may be receiving food stamps. Additionally, if the parent or parents work, they may be receiving the Earned Income Tax Credit (EITC) and Additional Child Tax Credit. We just listed five programs that welfare families typically receive. 

And there are more programs. If the family has young children under five, they could receive food packages from the Women, Infants, and Children (WIC) program. Additionally, the family may be receiving childcare assistance, Section 8 rental assistance, and/or energy assistance.

Now you might think that we have a dataset somewhere telling us the total number of welfare programs families are benefiting from. If you assumed that we do, you would be wrong. No such dataset exists.

The reason? First, the welfare system is disjointed. There is no single agency or dataset that can tell us the total number of programs people are on. Even Georgia’s award winning Gateway, which is one of the better integrated eligibility systems in the country, cannot tell you. While the Gateway can tell us about food stamps, Medicaid, WIC, TANF, and subsidized childcare services, it is missing the refundable tax credits, free school lunches breakfasts, Section 8 rental assistance, and other welfare programs not listed. 

Second, statistical sources do not include all welfare programs in their questionnaires and have other limitations, such as serious time lags. For example, the American Community Survey asks about food stamps, Medicaid, and Supplemental Security Income but practically none of the other programs, making a statistical inference for the complete picture impossible. 

The Survey of Income and Program Participation gets us closer, giving us childcare assistance, WIC, energy assistance, and public housing, among others. However, it is still missing the refundable tax credits, including the EITC which is one of the big three welfare programs. Worse, SIPP is structured for longitudinal studies that makes the survey totally impractical for monitoring program participation on a regular and timely basis.

Adopting Dependency Metrics in Georgia

Dependency metrics would improve our ability to measure success, and state leaders should consider implementing them in Georgia. 

Georgia would do a better job than the federal government with dependency metrics. The Gateway houses the data for critical programs, enabling Georgia to produce monthly estimates, more timely estimates, and for more programs. In contrast, the Feds apparently cannot meet its obligation in producing annual reports, provides only national data for only three programs, and there are significant time lags. The most recent Federal report came out on May 4, 2018, with 2015 and some 2016 data.

Once implemented at the state level, dependency metrics will improve over time. If and when further integration, consolidation, and streamlining of eligibility systems occur, as recommended by GCO, dependency metrics will become more complete and more useful.

However, they are not the sole answer. There is another way to measure success that would complement well dependency metrics. This will be the topic of my next blog.

In the meantime, do you have ideas on how we can measure success in welfare programs? We would love to hear them. Be sure to put them down in the comments below.

DISINCENTIVES FOR WORK AND MARRIAGE IN GEORGIA’S WELFARE SYSTEM

Based on the most recent 2015 data, this report provides an in-depth look at the welfare cliffs across the state of Georgia. A computer model was created to demonstrate how welfare programs, alone or in combination with other programs, create multiple welfare cliffs for recipients that punish work. In addition to covering a dozen programs – more than any previous model – the tool used to produce the following report allows users to see how the welfare cliff affects individuals and families with very specific characteristics, including the age and sex of the parent, number of children, age of children, income, and other variables. Welfare reform conversations often lack a complete understanding of just how means-tested programs actually inflict harm on some of the neediest within our state’s communities.