The Tribunal is currently one of the most controversial disciplinary systems in video games. Everyday the Tribunal sub-forum on the League of Legends website is filled with page after page of complaints about unjust bans leading many people to wonder what the conviction rate of cases that reach the Tribunal actually is. Are these players actually justified in their complaints or did they deserve what they got?
For those who don't play League of Legends, the Tribunal is a form of community-based policing implemented into the game via a report system. You can report players in each game by clicking a report button and choosing one of several offensive categories such as verbal abuse, negative attitude, intentional feeding, etc. Once you get a certain number of reports, they are compiled into a Tribunal "case." Players level 20 or above may then go onto the League of Legends website and vote in up to 20 Tribunal cases per day. They simply vote to pardon the player in question or punish him or her.
In order to determine the punishment rate as well as several other interesting bits of information, I conducted an experiment over the past couple of months in which I simply voted punish over and over (see images 2-6 in the slide show above for just a few examples). A potential flaw which has been pointed out in several comments on this article is that the act of repeatedly punishing people in the experiment itself may have skewed the data. This is not a concern as Riot employees have said that people who spam punish in the Tribunal eventually have their votes no longer weigh in toward the end result of the case. If Riot is to be believed, then my punishment votes did not affect the outcome of any of these cases. Furthermore, most of the cases were decided by a majority so one vote would not have made a difference.
According to the principles of statistics and random sampling, a random sample of 1,067 is required to draw accurate conclusions about a population as big as the League of Legends community (which had over 32 million players as of October and has likely grown since then) within a 95% confidence level with a confidence interval of +/- 3%. For this experiment, I voted in 1,087 total cases. (For more on sampling size please click here).
In 1,087 cases voted, 953 resulted in a verdict of punishment. This means when a case reaches the Tribunal, you have an 87.7% chance of being punished. To put that in perspective, juries in federal criminal trials in the United States between 1989 and 2002 were found to return a guilty verdict 84% of the time (source). In other words, your chances are better of being found not guilty of a federal crime than making it out of the Tribunal unscathed.
Of the 953 punishments, 433 were time bans, 56 were permabans, and 464 were warnings. Adding time bans and permabans together, there were 489 total bans. This means you are more likely to actually be banned by the Tribunal (51.3%) than you are to get off with just a warning (48.7%). 5.2% of players who reach the Tribunal are permabanned.
The total number of "toxic days prevented" by time bans was 2,860. Since there were 433 time bans, the average number of days a player is banned was 6.6. Bans are given out in three, seven, and fourteen day intervals with your sentence increasing after each punishment. A few other interesting facts to note is that I accrued a "Justice Rating" of 22,492, obtained a ranking of 516, and at one point achieved a streak of 52 cases correct. That's 52 cases in a row that resulted in punishes.
The information found in this experiment shows the punishment rate of the Tribunal is too high. To see that players on a video game are punished more often than accused criminals in a federal court of law is ridiculous.
The problem is that many cases are wrongfully voted to be punished. For example, in image seven of the slide show above you can see a Graves player was reported by a troll who was continuously calling him a noob. The report was clearly invalid and yet the majority of the Tribunal voted to punish Graves anyway. The result was a time ban. Image eight of the slide show shows a Miss Fortune being berated for taking someone's aura. She asks for them to "please stop calling her names" and as a result, they report her for "negative attitude." The majority of the Tribunal somehow agreed and voted to punish her. The result was a time ban. Or look at image nine of the slide show which depicts a Graves player being reported for "being a loudmouth" despite several other players in the game disagreeing and saying they liked him just fine. The majority of Tribunal did not like him and voted to punish. The result was a time ban.
Another conclusion that can be drawn from this data is that the punishments doled out by the Tribunal are too harsh and do not work. The fact that the ban rate is higher than the warning rate should serve as a red flag that the system is not working as intended. If the Tribunal worked, one would expect the warning rate to be much higher than the ban rate because players who get warned by the Tribunal would be correcting their behavior and not re-offend (resulting in a ban). Instead, players who get warned obviously do not amend their behavior. They end up re-offending and getting banned. What's more, it can be deduced that even players who get banned do not learn their lesson and are banned multiple times, thus driving up the ban rate so high it even surpasses the initial warning rate. In short, the fact the ban rate is higher than the warning rate shows that players are not deterred or rehabilitated by Tribunal punishments at all.
Astonishingly, when cases do get pardoned, many of them are wrong. Check out image ten in the slide show (warning: profane language) where a player calls his teammates noobs and drops multiple F-bombs. The Tribunal decided to pardon this guy by a "strong majority" vote! Or image eleven (warning: profane language) which shows very similar behavior, a player calling his teammates noobs, retards, and dropping F-bombs. Once again, the Tribunal pardoned him by a "strong majority." Perhaps the most eye opening example is found in image twelve where a player displays flagrant racism telling a teammate to "gtfo mexican" and instantly instructing other players in the game to report him because of his race. The Tribunal voted to pardon this behavior by an "overwhelming majority." This is a clear sign that the Tribunal is not working.
The question then becomes, why are these players being pardoned if the Tribunal is already too harsh to begin with? One possible answer is that the majority of Tribunal voters are the same trolls who act this way in the game. Riot Games has said that calling teammates noobs should be punishable in every instance but yet the Tribunal has pardoned this type of behavior time and time again. On the other hand, they punish normal in-game behavior such as taking auras.
There are five main reasons behind punishment. They are: 1) deterrence, 2) incapacitation, 3) rehabilitation, 3) retribution, and 5) restitution. It has been established using the data above that the Tribunal does not deter nor does it rehabilitate offenders. It does not incapacitate offenders (remove them from the population) because players can simply make alternate accounts, called "Smurf Accounts," and play on those while they wait for the ban on their main account to expire. Restitution also does not fit with the Tribunal. Restorative justice focuses on the victim and helping make them feel whole after a crime is committed against them. The Tribunal has absolutely nothing to do with the "victim" in League of Legends. Once you report a player, you are never told whether or not they got banned for "privacy reasons."
The only theory of punishment that even remotely fits here would be retribution - that the player is being punished because that is what they deserve. But as shown in several examples above, many of the cases are wrongfully voted to be punished. Even if a player does deserve the punishment he or she gets, they never "feel" the punishment because they can easily create a Smurf Account. In fact, their behavior is likely to get even worse because they no longer have to worry about their account getting banned.
This actually hurts the game overall because toxic players playing on low level accounts might discourage legitimate new players they get matched up against. Imagine playing League of Legends for the first time and facing an opponent (or having a teammate) that is racist, vulgar, and not fun to play with. This might make you no longer want to play the game at all. This player would not have been there if not for the Tribunal. That doesn't sound like retribution to me.
In conclusion, the Tribunal is a flawed system that should be completely reworked if not removed altogether. It does not work and currently makes the game worse, not better.