Is it bad for you to hanlde guns and ammunition and not wash your hands right after?

i read text on ammo boxes saying that the lead in it can cause birth defects and other problems. but it seems like alot of people dont pay attention to things like that. so is it bad to handle a gun and not wash.