A year or so ago, I read this article about how Wells Fargo ended up in such a mess. If you don't remember, Wells Fargo was opening accounts in their clients' name without their consent and ended up paying a few hundred million dollars in fines.

Long story short, a big part of the problem was that WF set a few metrics to guide the company, set strong incentives to optimize those metrics, and blindly let the machine get to work. The company did a great job of optimizing the metrics but lost sight of the strategy the metrics were meant to represent.

This tendency to confuse metrics for a strategy is called Surrogation (I keep forgetting this word, which is half of why I'm posting this here). When I'm talking to other data scientists I usually hear this put like, "When a measure becomes a target, it ceases to be a good measure" (Goodhardt's Law).