Well, if you have not a maxed out bag space you can consider it, but if you have it maxed out I don't think that having 2-3 items of each is such a loss on space...
If you are in a point that you have like 10 upgrades, then ok, delete them, but I have also all my evolutions and I have 2 metal coat, 2 upgrade, 1 king's rock and 2 sun stone, and I am not planning at all to delete them, as you stated they are too rare to consider it, unless they are ruinning all the rest of the game.
Look at this way, if evolutions items takes out like 20 of your bag space, and you have 100 pokeballs, delete 20 balls, you will get 20 pokeballs in less than 10 stops, while you would need thousands to get that 20 evolution items.
The most common items are pokeballs and regular potions, if you have a good amount of them I would personally delete some of those before evolution items.
