The thing is he probably gets more by being the owner of the LA Rams than he loses by leaving St Louis. Sure, St Louis fans would care, especially if they don't have a team that replaces the Rams, but would other fans care? I'd have to say the vast majority wouldn't care at all. Did people say "Oh I can't like the Titans, Bud Adams left Houston for no reason other than to make more money!"
No, I never heard a peep about that. I'm sure Houston fans were plenty salty, but the vast majority? I don't really think so. Ultimately I don't think Stan backs away because he's afraid his legacy among St Louis takes a hit.