used to clean all mine don't anymore.. If I am loading a box of "special" deer hunting ammo I will clean the primer pocket and clean the burr off the flash hole on the inside.
It's only necessary if there is a physical need.
Those that do probably do it for piece of mind rather than any perceived increase in accuracy or reliability.