Practical Data Science

Share this post
Post hoc, ergo propter hoc
harterrt.substack.com

Post hoc, ergo propter hoc

Ryan Harter
Jun 17, 2020
Share this post
Post hoc, ergo propter hoc
harterrt.substack.com

Economists have a handy phrase to describe a fairly common fallacy: "Post hoc ergo propter hoc" meaning "After, therefore because".

Wikipedia has an example of how this might look in the wild:

A tenant moves into an apartment and the building's furnace develops a fault. The manager blames the tenant's arrival for the malfunction. One event merely followed the other, in the absence of causality. (link).

In tech this might manifest as part of a feature launch. Say we launch a feature on a Monday and on Tuesday our retention goes way up. We're tempted to think that our feature launch caused the increase in retention.

This might be a fine assumption if our metrics are generally constant but more often than not this is just another case of correlation not meaning causation. Just because these events happened together (albeit with a lag) doesn't mean that they were causally linked.

With how volatile the world has been this year, this is even more relevant. Our metrics are moving up and down for reasons totally unrelated to our individual actions.

Usually, I'd deal with this type of risk by running a controlled experiment. We randomly split some users into two equal groups and give them different product experiences. If the users behave differently, then we know that the difference is caused by the difference in the product experience and not some arbitrary outside force.

Share this post
Post hoc, ergo propter hoc
harterrt.substack.com
Comments

Create your profile

0 subscriptions will be displayed on your profile (edit)

Skip for now

Only paid subscribers can comment on this post

Already a paid subscriber? Sign in

Check your email

For your security, we need to re-authenticate you.

Click the link we sent to , or click here to sign in.

TopNew

No posts

Ready for more?

© 2022 Ryan Harter
Privacy ∙ Terms ∙ Collection notice
Publish on Substack Get the app
Substack is the home for great writing