In some cases I'd agree. Yesterday as I was looking over Connelly's SP+ ratings, in that post I noticed one area UGA rates higher than Alabama is in special teams play. Can we get a decent picture through three games? When one of the three is a game where they didn't punt nor kick a field goal? Yeah, that stat is skewed a bit. It's like opening with Army and seeing fans complaining about time of possession.
It's the stats that shine a positive light you can't use to make an observation. All stats that are negative? All true and significant and must be taken into account.