If the Rams expect to go forward with Rob Boras as OC (unless the offense really takes off these last few games like the Lions O has under Jim Bob Cooter) and keep the defense intact and have no money left for improving the offense....then yeah. Fisher has to go because nothing will have materially changed.
If, however, the Rams do a Vermeil-type intervention and require Fisher to hire a GW level Offensive guru for the offense with the same level of autonomy and free up resources such that the offense can improve (yeah, that will probably mean Chris Long is gone or kept on a dramatically reduced/restructured deal), then I'd keep him for another season, his last season of his contract.