FINDING THE PULSE OF THE PANDEMIC

BY: BRAD EDMONDSON

COVID-19 is forcing everyone to come up with new ways to operate, and that includes the Census Bureau.

The Census Bureau has created a program that could be a real game-changer for business users and others who need data that is reliable, timely, and free. The agency has collaborated with five federal agencies to produce data on the social and economic effects of the COVID-19 pandemic in the United States. It’s called the Household Pulse Survey and its results are released one week after each wave of the survey is collected. It’s an experiment, but so far it seems to be working very well indeed.

Disaster relief is far more effective when it is informed by data that is from trustworthy sources available on a basis that’s as contemporaneous as possible. The US government has a stellar record of producing reliable data. But until now, it has taken a long time to release those numbers. For example, the Current Population Survey’s annual estimates of household income are the gold standard for understanding household economics—but they are collected in March and released in September, and they measure conditions for the previous calendar year.

“We did the Pulse survey to provide needed data, but also as a proof of concept,” says Victoria Velkoff, the Bureau’s associate director for demographic programs. “We wanted to show that the federal statistical system could react quickly.” The federal statistical system is spread across 125 different agencies and employs tens of thousands of people. The Pulse surveys are teaching this elephant how to dance.

The Pulse survey gets results online from about 100,000 users each week. Its 30 questions describe Americans’ experiences with income and employment, emergency federal assistance, food security, mental and physical health, access to health care and insurance, ability to meet rent and mortgage payments, schooling, and access to online services, as well as household demographics. Results are reported for the nation, the 50 states and the District of Columbia, and the 15 largest metropolitan areas. A separate effort, the Small Business Pulse Survey, measures the experiences of small business owners. (see related story in this issue page 13).

The Household survey was originally funded only through the end of July, and the Small Business survey through the end of June. But on July 31, the Office of Management and Budget authorized a second wave of both surveys. The small business survey resumes in mid-August and will run through mid-October. The household survey will resume in mid-August, and an end date was not announced. Both surveys will adjust their questions so that key data will be comparable to the first wave, with new questions to reflect changing conditions. The second wave had a lot of support, because both surveys are extensively used by state and local officials and executives in many industries, from health care to hospitality, transportation, and food.

A “pulse survey” is a recurring set of questions, usually taken online, that helps leaders understand how their communities are doing. Marketers and human resources executives use these kinds of surveys to keep tabs on the satisfaction, productivity, and attitudes of customers and employees. Privately produced pulse surveys work well when the group being measured is small and doesn’t change quickly. They become junk when anyone can respond. The reason is a core principle of survey research called the “sampling frame.”

A sampling frame is a list of people who form the population from which a sample is taken. A survey can’t reflect the characteristics of an overall population unless its sample is representative. In other words, if 21 percent of the overall population is aged 18 to 24, a survey’s results will only approach reliability for that group if 21 out of 100 survey respondents are aged 18 to 24. If that hypothetical survey gets only 15 young adults to respond, their answers can be “weighted” so that 21 percent of the published results are made from their responses. But weighting also adds “sampling error” to the survey, making the results less reliable.

Internet surveys that do not ask for a respondent’s demographic characteristics are are of no value at all to researchers and data scientists because they can’t be made representative. In fact, it’s easy to create a “bot”—a simple set of computer commands—that will automatically answer an internet survey hundreds of times. Internet surveys that collect demographics and weight their results are far better, but they have another weak point. In order to remain representative for subgroups, the number of respondents must increase dramatically. So if you’re seeking reliable numbers that describe food insecurity among adults aged 55 to 64, or the prevalence of anxiety and depression in Alabama, and you need to ask different people those questions every week, you need to get a lot of people to respond.

The US government can field a reliable weekly pulse survey and report the results for all 50 states and 15 metro areas because it owns the ultimate sampling frame. The Master Address File (MAF), jointly produced and maintained by the Census Bureau and the US Postal Service, comes very close to listing every address in the country. The MAF has been continually refined over the last 20 years, and the Census Bureau’s Velkoff estimates that it also captures about 80 percent of Americans’ e-mail addresses and phone numbers. About 90 percent of Americans use the internet at least occasionally, according to the Pew Research Center. Sending out mass e-mails costs next to nothing and processing the same set of questions once a week isn’t prohibitive. Put it all together, and a reliable weekly survey that is available to anyone at no charge becomes a possibility—but only if you’re the US government, which is required by law to protect the confidentiality of data on individuals. The government’s reach, combined with the confidentiality law, make its sampling frame much better than the enormous lists curated by tech companies like Google and Facebook. Also, there’s no way a tech company is going to give away something that valuable for free.

The federal statistical system is spread across 125 different agencies and employs tens of thousands of people. The Pulse surveys are teaching this elephant how to dance.

The Census Bureau has learned a great deal since the Pulse Survey launched, according to Velkoff. She also helped develop the American Community Survey (ACS), which streamlined the decennial census and now produces reliable neighborhood-level demographic profiles that are refreshed once a year. The MAF is the sampling frame for the ACS, too.

As the Bureau’s liaison to international statistical agencies, Velkoff brings innovative ideas from other countries. Tinkering with Census Bureau products must be done carefully, she says, because the data are so valuable to so many people. But the gains are potentially even more valuable. For example, the Bureau has begun sending text message reminders to respondents who ask for them, because people are far more likely to notice text messages than they are to check their e-mail. “We’re learning how to make our messages shorter,” Velkoff says.

WHAT’S REALLY HAPPENING DURING THE PANDEMIC?

The Household Pulse Survey turns the anecdotal evidence reported by health care workers and journalists into data that can be used to allocate resources.

It confirms that low-income working families are among those hit hardest by the virus. In mid-July, 60 percent of households at the bottom of the income scale ($25,000 or less) had lost income since mid-March, compared with just 32 percent of those at the top ($200,000 or more). Other demographics match the pattern. The youngest workers (aged 18-24) were significantly more likely than average to have lost income, along with Blacks, Hispanics, those who had not completed high school, and those who were divorced, separated, or never married. And 58 percent of households containing children said they had lost income in mid-July, compared with 45 percent of households with no children. School closings have had an enormous economic impact.

The survey also confirms a sudden, rapid increase in food insecurity. An average of 10 percent of all households in the Survey say they “sometimes or often” do not have enough to eat. While the Pulse Survey’s results are not strictly comparable to other surveys, several pre-pandemic measurements registered food insecurity at much lower levels. The survey conducted during the third week in June showed 13.9 million children living in a household which “sometimes or often” did not have enough to eat, according to Lauren Bauer of the Brookings Institution. Earlier surveys estimated that number at 2.5 million children in 2018, and 5.1 million at the peak of the Great Recession in 2008.

Americans need hard facts on what the pandemic is doing to their country. The Census Bureau’s innovative response foreshadows a permanent change in the way federal agencies collect data.

Americans living in lower-income households remain most likely to “sometimes” or “often” lack enough food to eat, but food insecurity since the COVID virus has risen fastest among middle-income households.

One of the most revealing sections of the survey was written in conjunction with mental health experts at the National Center for Health Statistics. Respondents were asked how often they were “feeling nervous, anxious, or on edge,” “having little pleasure or interest in doing things,” and “feeling down, depressed, or hopeless.” The questions were worded to match those used by doctors to screen patients for depressive or anxiety disorders. A recent “meta-analysis” of several large studies estimates that that 20 to 25 percent of women will suffer from depression at some point during their lives, along with 7 to 12 percent of men. In the mid-July Pulse survey, 55 percent of American women and 48 percent of men said they felt down, depressed, or hopeless for at least several of the last seven days.

It might not be surprising to learn that the pandemic has a lot of people feeling down. But the strength of the Pulse Survey is in the details. Experts at the Center for Disease Control who analyzed the survey found that overall, about 40 percent of Americans were showing symptoms of anxiety or depressive disorder in mid-July. The prevalence of those symptoms was highest in young adults, and it rose from 47 percent of those aged 18 to 29 in mid-April to 55 percent in mid-July. Anxiety and depression were much lower in adults aged 70 to 79, and those levels did not rise over the four-month period. And they were lowest among the group at greatest risk of death. Among adults aged 80 and older, the share who were anxious and depressed actually declined, from 21 percent in mid-April to 19 percent in mid-July.

It makes sense when you think about it. Among Americans aged 65 and older, 84 percent collect Social Security benefits and 100 percent are covered by Medicare. But households headed by young adults have no guaranteed source of income, most of them have lost income, and 22 percent of Pulse survey respondents aged 18 to 24 also have no health insurance. Until they get sick, young people have more to worry about. Half of American households have lost employment income since mid-March, according to a weekly survey taken in mid-July. The share of households losing income also increased four percentage points since the first week the survey was taken, at the end of April. But that was actually good news. At the end of April, an additional 38 percent of householders said they expected to lose income in the next month. That catastrophe was averted by the federal Paycheck Protection Program, which reimbursed employers who did not lay off their workers during the emergency. Funds from the PPP arrived just as week 1 of the survey was being collected. Americans need hard facts on what the pandemic is doing to their country. The Census Bureau’s innovative response foreshadows a permanent change in the way federal agencies collect data.

The Household Pulse Survey’s data tool (https://www.census. gov/householdpulsedata) is an easy way to browse this rich data source. It shows exactly how many Americans are struggling. It reveals who is most likely to need immediate help. And it may also show the way to a new era in federal data collection and distribution.