I hate to say it but I have done this many times. Passing as long as there is enough room to do so, it never bothered me on how many people are on the trail as long as there was room and no one was coming the other way with a good view of what was up there. Sure, I'll do it.
Most of the time I stay back and study how each is riding and then when the timing is right I will go by, of course faster then they are going to make the pass but not WOT by any means. I then pull in front and continue on my way. The rest of the group usually follows and that's all there is to it.
1) No one should be riding side by side other then to pass, so if you were doing that you were just as much in the wrong as the guy passing.
2) Stay to the right and if you notice someone behind you, they aren't part of your group, simply move over and let them by, or motion them by. They got to that position by riding faster then you so let them continue at that.
3) Confronting them is wrong unless it was something that was totally stupid, then a simple talk that remains calm would have been the best thing. That way each of you remain cool and what was done is done. Move on, life isn't all about 10 minutes ago.