Fitbit claims its users cost their employers a lot less in health care — here’s why that study is flawed

People who used Fitbits cost their employer nearly 25 percent less in health care costs, according to a new study from Fitbit. At first blush this appears to be a tantalizing result for companies looking to save money, but a closer look shows that gains aren’t as dramatic as they appear.

Here’s the claim: Fitbit says that employees at a large company cost almost $1,300 less to employers when they used a Fitbit for two years. These numbers look great, but there are flaws in how the study was conducted. The biggest problem is that the results don’t include the people who dropped out. An employee can opt out at any time, according to Amy McDonough, vice president of Fitbit Group Health. “We did not look at the number that opted out,” she adds. This isn’t just a hypothetical, either. More people started the program than completed it, though Fitbit declined to provide details on how many. This means that the people who completed the program cost $1,300 less, but we don’t know if the people who started and left saw any savings.

Everyone’s health care costs declined, not just people with Fitbits

Dropouts skew results. “If you’re just tracking the people who stuck around and not that many people stuck around, your outcomes might look better,” says Sherry Pagoto, a professor of medicine at the University of Massachusetts Medical School. Ignoring dropout means we can’t tell if there are differences between the people who stick with the study and the people who don’t. We don’t know for sure, but it could be that someone in good shape likes being able to monitor their activity, while a couch potato would get tired of exercise reminders and drop out. If you ignore the people who drop out, the final results are not only more positive, they make it seem like the regimen works for everyone.

The study began with a first year where no one had a Fitbit; this data was used as a baseline for the “before and after.” Afterward, people who opted to be studied received a partially subsidized Fitbit. Fitbit then compared this group’s insurance claims against a control for the next two years. This is what’s called an opt-in study — of the roughly 20,000 employees at the company, Fitbit recruited about 900 volunteers.

In general, opt-in studies like this one create the possibility that those who join are already healthy and that’s why their medical claims are going down. Fitbit worked to prevent this by creating a control group that roughly matches the age, gender and previous health information of the participants. This is the only ethical way to run a study in the real world. Still, it’s best to have a kind of study called a randomized control trial, where half the people who want to participate don’t get a Fitbit at all. Randomized controlled trials are considered the gold standard because they help ensure that the results are from the Fitbit, rather than differences between groups of participants.

All this matters because of how popular Fitbit is with companies that have “corporate wellness” programs. Companies provide health insurance to their employees. If their employees get sick often, they need to pay more. So, around 80 percent of companies do things like subsidize gym memberships to encourage everyone to be fit. Businesses including Target, IBM, and Bank of America have all bought subsidized Fitbits for employees, so it’s important to figure out if this is working.

Abandonment of fitness trackers is a big issue

With that in mind, let’s take a closer look at the numbers. The Fitbit group saw a 24.7 percent reduction in their medical claims, and they did cost the company less. But those in the control group also saw a 9.3 percent reduction. Everyone’s health care costs declined.

Finally, people had to use their Fitbits pretty consistently to see a gain. About 350 participants used the gadget for at least 274 non-consecutive days over two years. These were the ones who saw a big reduction in medical costs. The difference was much smaller for people who logged less than those 274 days. And there was little significant change for those who logged fewer than 182 days.

It’s common sense that you can’t benefit from Fitbit if you stop using Fitbit. But this is a key problem that raises questions about the long-term effectiveness of activity trackers as a corporate wellness strategy. Abandonment rates are high across the board. One research firm reported that over a third of people ditch their “smart wearables” within six months. Another study published just today — which used the Fitbit Zip specifically — concluded that it doesn’t really help health, probably in part because 90 percent of people abandoned it after a year. If people won’t continue using the Fitbit, the initial numbers might not matter. Having medical costs go down at the beginning and then rise again when employees abandon the gadget is not the ideal outcome for any company.

Fitbit has been getting a lot of heat recently. A recent JAMA study found that fitness trackers in general can backfire when it comes to weight loss by providing a false sense of security. In response to today’s study on how the Fitbit doesn’t work, a spokesperson said that, “We are confident in the positive results our millions of users have seen from using Fitbit products.” This is all well and good, but we need to take a closer look at where these positive results come from, and how long they last.