The headlines blared “How Much Weight Did We Gain During Lockdowns? 2 Pounds a Month, Study Hints” (New York Times) and “Americans have piled on the pounds during the COVID-19 pandemic.”
The reference is to a study by the American Psychological Association asking about weight gain or loss during the pandemic. They found 61% of adults experience undesired weight changes:
To break that down, when the New York Times says we all gained two pounds a month, it’s referring to only the 42% of Americans who had undesired weight gain, not the average American. It’s not including those people who lost weight or stayed the same or were fine with their weight gain. And the pounds listed here are the average, heavily weighted, if you’ll pardon the phrase, toward those with the largest fluctuations in weight. The median weight gain among those who felt they gained too much is actually 15 pounds. So the “we gained two pounds a month” headline is bogus on many, many levels.
Since this study factors in how we felt about our weight loss, how much weight did we gain in 2020?
Not much:
In fact, it’s down from 2018.
A scientifically semi-literate press or sensationalist headline writers aren’t fully to blame here. In fact, it’s unfortunately common even in our data-centered fundraising world. How many times have you seen this graph?
Wow; revenue sure look like they went up… except that this graph isn’t starting at 0. Here’s a less tricky graph that shows off your .2% growth:
And here’s one that highlights an important reference point: the budget:
It’s important to ask “compared to what” lest people try to fool you (or worse, themselves) with their metrics. Here’s one I saw live: a new online team came it and the director bragged about the increase in open rates.
Clearly, much better subject lines under the new regime, no? More people are getting our messages.
Well, for clarity, let’s look at this on a month-by-month basis.
So, something happened in July that spiked open rates. Maybe it’s the new team, but we must ask why. One of the common culprits, when you are looking at percentages, is a change in N, the denominator. Let’s look at the same graph, but instead of percentages, we are going to look at the number of people who opened the email.
Huh. Our big spike disappeared.
In looking into this, July is when we started suppressing people who had not opened an email in the past six months. This is actually a very strong practice, preventing people who don’t want to get email from you, have moved on to another address, or were junk data to begin with off your files. As a result, your likelihood of being called spam goes down significantly.
So it wasn’t that twice as many people were opening emails; it was that half as many people (the good half) were getting the emails. Still a victory but for Team Data Hygiene, not Team Awesome Subject Line Writer.
So be sure to compare your data to relevant outside points and ask some basic questions about from whence it came.
(And if you are looking for COVID-19 weight gain, Krispy Kreme is offering free donuts to anyone who gets vaccinated. It’s the right thing to do, plus, mmmmmmmm… donuts…)
