Friday Notes, July 15, 2022
Dear Friends —
A vitamin is a substance that we need in tiny amounts for to stay healthy and strong, to do what we want or need to do. Our bodies don’t make vitamins on our own so we have to get them in food or pills. Sometimes we get vitamins in meals; sometimes we make a point of eating specific foods because of their vitamin content (oranges, dark green leafy vegetables, fish oil); and sometimes we take vitamin supplements in one form or another. Some vitamins are water-soluble, washing out in our pee; others take up residence in fat, available when called upon.
I’ve started to play around with the idea that data and evidence from research and evaluations are like vitamins for decision makers — information they consume, within their daily diet or as a special add-on, that makes the work they’re doing more likely to achieve its goals. The vitamin analogy makes us think about whether the data and analyses are in the metaphorical egg they have for breakfast, in kale, in bitter-tasting capsules, or in GummyVites.
Stay with me here.
One way that data and analysis can lead to better government spending and more effective programs is if it’s part and parcel of everyday activities, one of the resources that’s available for getting work done. For example, say a district education official gets a live feed of data from the 600 schools in her jurisdiction, with up-to-date information about teacher and student attendance. Day-to-day, she might be able to deploy temporary teachers where they are needed; over time, she would be able to see patterns of absences and try different solutions.
Another way is more intentional — like eating kale to get enough vitamin A, K, B6 and C. Some decision makers understand the value of high quality information to achieve the good outcomes they’re hoping to foster, and are willing to put in the work and potentially take some heat. Most people don’t like kale as much as candy, but people who are health-conscious and knowledgeable about the benefits of superfoods happily eat kale and may even forego the candy. Correspondingly, mission-oriented policymakers who are persuaded about the benefits of evidence may champion its use. That district education official, for instance, might ask her team to do a special study of teacher absenteeism, and then use it as an input into next year’s plans for recruitment, training, and supervision.
A third way to use data, research, and evaluations is more like vitamin supplements, where every once in a while people with specialized skills and a bit of distance bring to a decision makers a package of evidence around program effectiveness from other places — maybe even a systematic review. Or they design and conduct a study to try to answer a complicated question that’s troubling people charged with designing or running a government program. This also takes intentionality on the part of the decision maker, but can be an efficient way to get a dose of evidence at the right time.
Then there are ways to create and promote ideas that are both evidence-based and politically attractive. Like GummyVites: kids like candy, and policymakers like ideas that will win them votes or advance their careers. You sneak some vitamins into gummy bears so kids get a nutritional boost — and you can embed empirically sound design into sexy and popular programs so there’s a greater chance they will do some good. We see that in conditional cash transfer programs, for example, and — in the best cases — with universal health coverage and education access programs that benefit everyone but are designed, using solid evidence, to be particularly responsive to people who are most disadvantaged.
I’m not sure the vitamin metaphor is a good one, but I do know that it’s important to recognize and pursue the many routes through which data and evidence can enter into consequential decision making. A pluralistic vision of the “data and evidence community” is part of what it’s going to take to get to better public policy.
In case you’re not sufficiently freaked out by artificial intelligence and technology more generally, take a look at “How Normal Am I?”
And as we enter the high season for summer vacations . . .
Have a good weekend,
-Ruth