You are partially correct. Going smaller in engine displacement doesn't always help in fuel economy. Look at the very vehicle in this thread. Going up the 3.6 from the 3.0 without penalty.
But for many many customers today, FE is more important than 0 to 60. If you can provide a smaller engine that provides decent drivability and maximum FE that gets it done for most people...................not generally the people that hang out in Muscle Car threads...........................but most OTHER people.
Look at the Turbo 4 in the Regal. A large "mid-size" car. It gets decent FE while providing superior performance.
Ford is putting their Turbo 2.0L in the Exploder, Edge and even the "full-size" Taurus.
When CAFE hits, you will likely see more and more and more of that, like it or not. The government knows better.....................................no they don't, but they think they do.
It's a balance thing that goes well beyond the engine displacement. It's number of gears in the trans, final drive ratio, and probably most important, the powertrain calibration.