That could be viewed as a disincentive to level up, similar to when people wanted to keep their pokemon at CP 2999 or below, because CP 3000+ mons lost motivation faster.
They could offer some sort of incentive for defeating defenders with an attacker of lower CP (similar to the prestiging concept), such as a nominal amount of stardust or something. For example, if a defending Rhydon has a remaining (motivation-adjusted) CP of 600, and you defeat it with, say, a Poliwag or Bellsprout with a lower CP, you get 50 stardust. They could even scale it up so you get 100 stardust if your attacker has half the CP of the defender. Or, instead of stardust, they could make it an XP bonus, which would benefit lower level players more. This would bring back the idea of prestiging, which gave a use to those mid-level mons that no longer have much purpose.