It may a little, but if there are enough total players it won't matter, and since it's optional any player could just not join those games.

You could make the same argument for any game setting.