It’s usually said about a charge under a magnetic feild. The magnetic force goes perpendicular to the direction of motion of the charge(F=q*v×B). Work is done only if the force is applied along the direction of motion. So on a moving charge, magnetic force does no work.

Not sure how that plays on magnets though. Magnets are magnetic because electrons go in circles producing the feild, and it might be because electric feild comes in and do the work but it’s not clear for me either