That is how social norms work: Negative feedback.You think people who don't care about anyone else is going to respond well to be told they are being rude? I don't want to get into an argument with a selfish person that's just going to made me mad. I have no authority to tell anyone to do anything and a selfish person isn't going to say, "yeah you're right, sorry about that I put my headphones on".
And assuming the worst outcome beforehand is not a healthy way to live.