If Alabama played ND/Pac12 in a home-and-home, that would mean one less home game. That game would usually draw 90,000+ ticket buyers at $45 each (plus concessions, plus merchandise, plus economic impact). The TV contract is already in place with the conference/ND and doesn't create an additional revenue source for the visiting team. The schools negotiate how much to pay the visiting team and some of that payment can come from the TV contract.
There are two kinds of FBS teams. Those that are willing to take a beating for the revenue to build and maintain their sports programs and those that are trying to position themselves for a possible bowl berth and need wins to do that. If they're trying to get a bowl berth, they won't be playing a lot of Power5 or Group of 5 teams. Some, but not a lot.