Leetify Rating Upgrade

So, what changed?

1. The system now awards rating when the bomb is defused with Ts still alive, when the bomb explodes, or when a round ends due to time expiring (not just when a duel occurs)
2. Leetify Rating is now awarded for saving weapons and equipment
3. The kill reward system has been tweaked to favor kill order less heavily when kills happen in quick succession
4. The damage reward is split more evenly when dealing a lot of damage to a low-health player

If you were just here for the TL;DR, there it is. But for those with more questions, let's dive into each of these in detail!

1. End of Round Adjustments

The old kill participation model views the round as "complete" when the final kill takes place, but this isn't true in three cases: bomb defusal (where at least one T is alive), bomb explosion (with at least one CT alive), or time expiring (with at least one T saving).

Bomb Defusal (T Save)

The bomb being defused with Ts still alive is the most obvious case where this was an omission in the previous model. If it is a 1v5 for the remaining CT, their odds of winning the round are extremely low. However, if they get a ninja defuse without getting a kill, they obviously deserve a ton of credit, as they have just singlehandedly turned the round from a loss to a win. This should correspond to a huge reward in Leetify Rating. The old model did not award any rating for this action, as a kill was not involved.

The updated model awards credit based on the degree to which each CT participated in the retake. A relevance factor is assigned to each CT who was alive when the bomb was planted. This relevance factor is computed from the percentage of the retake that they were alive for. If a CT dies just a few seconds into the retake, they are considered less relevant than a CT who lived for the entire retake. If two CTs are left to close out a 2v1 and one of them dies right before the other CT defuses, then the CT that died late in the retake will be given a high relevance factor. The remaining win probability at the time of the defuse is distributed between the CTs based on their relevance factor for the retake.

The relevance factor is just the percentage of the retake that the CT was alive for. If they live for 3 seconds in a retake that takes 30 seconds, they will be assigned 10% relevance. If they live for 15 of the 30 seconds, they will be assigned 50% relevance. The win probability remaining when the retake is complete will then be distributed evenly between each CT according to their assigned relevance. Any CTs that died before the retake will have 0% relevance and consequentially get no credit.

If the final kill is recorded with time left to defuse the bomb, the CT team has acquired 100% win probability. In this case, there is no remaining win probability to distribute after the bomb is defused. Therefore, this section only applies to cases where the CTs defuse the bomb while there are still Ts alive. All Ts that are alive when the bomb is defused are penalized equally. Any Ts that died during the retake have been penalized according to the change in win probability that their death resulted in, so they are not double-penalized for the defuse as well.  

The specific CT who defused the bomb is not taken into account. Assuming a teammate is not AFK, defusing is a team effort. If two CTs are alive and one defuses while the other protects them, they both deserve credit for the round win.

At the start of the next round, the Ts that survived will get a reward for saving their gear. This is explained in the next major section, 2. Save Rewards.

Bomb Explosion (CT Save)

In the case where the bomb goes off and CTs are still alive, this means there was some win probability left to distribute in the form of Leetify Rating. The remaining win probability is rewarded to all of the Ts that were alive when the bomb was planted. These Ts are assigned a relevance factor based on when they died during the retake. This relevance factor is calculated exactly the same way as the CT retake relevance factor, as explained in the section above. Then, when the bomb explodes, the remaining win probability is split up amongst the Ts based on their relevance factor. Any CTs that were alive at the end of the round (or that died to the bomb explosion) are penalized equally.

At the start of the next round, the CTs that survived will get a reward for saving their gear. This is explained in the next major section, 2. Save Rewards.

Time Expired (T Save)

The Ts saving is the simplest case. The remaining rating is distributed evenly between each of the CTs that are alive and subtracted evenly from each of the Ts that are alive.

Similar to the bomb explosion case, any Ts that save will get rewarded at the beginning of the next round for the gear that they held onto. Let's dive into how we distribute these rewards.

2. Save Rewards

Saving is a fundamental part of good decision making in CS:GO. However, since Leetify Rating operates on changes in win probability within the round, the previous version of the rating did not do much to reward players for saving.

This upgrade now does four extra calculations when a player saves:

1. Compute the initial win probability of the team that saved
2. Compute this win probability as if they had not saved any gear
3. Reward the players who saved based on how much the saved gear increased their win probability
4. Subtract this rewarded amount evenly from each of the opponents that were alive at the end of the previous round

Saving players cannot gain more rating than the opponents were rewarded to conclude the previous round. If the win probability increase at the start of the round exceeds the rating gained by the opponents to conclude the previous round, the total save reward is capped at the previous end of round reward.

Note that Leetify Rating is a zero-sum metric, so anything gained by one team must be lost by the other team. In the case of a save, this means that in order to give the saving team a reward for carrying their gear forward, the other team must lose an equal amount of rating. Since the players that were alive at the end of the round were awarded rating corresponding to the remaining win probability, these are the players that are penalized at the start of the following round.

The amount rewarded to each saving player is proportional to the value of the gear they saved. A player who saved an AWP will gain more rating than a player who saved an AK in the same round.  

The saved equipment bonus is awarded at the start of the next round, so saving on the final round of a game will not lead to any bonus in Leetify Rating.

3. Kill Participation Rewards

Our old system overvalued the order of kills. Previously, the system relied on duels, which are a complete series of traded kills. Each player was awarded rating based on the outcome of the duels they participated in. This was a great solution to make sure that entry fraggers did not get too harshly punished for the drop in win probability that occurs when they die first in a push but subsequently get traded by their teammates. However, this did not solve another problem related to kill order. When two kills happen around the same time as each other (but not necessarily in the same location), this can lead to large changes in the win probability between these two events. If the kills happen away from each other, it is rather arbitrary which one happened first. One player should not get punished much more harshly just because their death happened to occur one second before their teammate's death across the map.

To solve this issue, kills are now evaluated in the form of kill chains. A kill is added to the chain when it comes four seconds or fewer after the previous kill. Each kill is weighted based on how close it is to the previous kill in the chain. The weighting is then used to calculate the Leetify Rating gained or lost by that kill. The weighting factor is scaled linearly based on how much time from zero to four seconds has elapsed between the kills. To illustrate this actual calculation, let's look at a few examples.

Example 1: Kill B happens more than 4 seconds after Kill A

When these two kills are 4+ seconds apart, we treat them as if Kill B 100% happened after Kill A. In this case, it is safe to punish the person who died in Kill B completely according to the new win probability generated following Kill A.

Example 2: Kill B happens 0 seconds after Kill A

When these two kills happen simultaneously (~0 seconds apart), we give them 50-50 weighting. Therefore, half of the credit given for Kill A is given assuming that Kill A happened first, and half of the credit given for Kill A is given assuming that Kill B happened first.

This is where this change is really crucial, because before, whichever kill happened a few ticks before the other could be assessed quite differently from the subsequent kill. The win probability may have changed quite a lot from, say, a round going from a 4v5 to a 4v4. The new system ensures that when kills happen in close proximity, they are treated similarly in terms of rating rewards.

Example 3: Kill B Happens 2 seconds after Kill A

In this case, a weighting factor is applied linearly based on the fraction of 0 to 4 seconds that has elapsed between the two kills. This means that Kill B gets a 25% weight, treating 25% of the kill reward as if Kill B happened first. Kill A gets a 75% weight, meaning that 75% of the kill reward is applied as if Kill A happened first.

This accounts for the uncertainty in these nearly simultaneous scenarios. We suspect that these kills could have happened in a different order and not impacted the round outcome, but we want to account for some of that change in win probability with linearly scaling weights. The more time that has elapsed, the more heavily the weights will take the new win probability into account when awarding or subtracting Leetify Rating.

4. Overkill Rewards

We have implemented a new system to more evenly spread the damage credit in the case that someone does a ton of damage to a low-health opponent. This comes in the form of considering up to 50 overkill damage in the computation for the damage dealt portion of the kill reward. As outlined in the "Contribution to Kills" section of the main article, the fraction of damage dealt to an opponent is worth 30% of the credit in any given kill. Consider an opponent is hit for 99. The person who got the kill is only awarded 1% of the damage category, no matter how much damage they dealt. This can correspond to as low as 0.3% of the the full kill reward. Of course, they still get their 35% credit for being the one who actually gets the kill. Nonetheless, we think that these old damage rewards are not appropriate. In this section we will explain why we feel that it is better to spread this credit more evenly between the assisting player(s) and the player that gets the kill.

The idea of overkill in CS:GO seems strange. There is no advantage to dealing more damage than necessary to a player who is low. However, there are cases where it is not clear that a player is low. In these cases the damage reward was punishing the player that got the kill too harshly and giving too much credit to the assisting player(s), even though the assist damage ended up being rather useless. It is a safer play to pull out your pistol or aim for the body when you know an opponent is low. In cases where you do not know the player is low, you will continue to play as normal. Therefore, your rating should be a compromise in this case. Since you didn't have the info that the opponent was low, and you hit a shot that would have been lethal regardless of the opponent's HP, you should retain some of the credit for the damage you dealt.  

Here is a quick case study to show three situations where the overkill reward is applied, and why it is appropriate in each case:

Example 1: Unconfirmed damage

In this case, 99 total utility damage is dealt to the final T without the CTs knowing. BOT Bill hits an AWP shot on the final player. In the old system, BOT Bill would only have gotten one percent of the damage credit for this kill (corresponding to about 0.3% of the total Leetify Rating for the kill), even though they did enough damage to ordinarily kill the opponent outright. In the new system, 99 damage belongs to the people that did the utility damage, and 51 damage belongs to BOT Bill (1 actual damage plus 50 overkill damage). This means that BOT Bill now gets approximately one-third of the damage credit (corresponding to about 10% of the total Leetify Rating for the kill). The fact that the opponent was low was not known, so BOT Bill did not alter their style of play and they would have gotten the kill on a full-health opponent anyway.

Example 2: Confirmed damage (safe play)  

In this case, 99 damage is dealt to the final T and called by a teammate. BOT Bill knows that the remaining player is well within one-hit range with a pistol, so they switch to their USP-S. This makes it easy to get the final kill, and they deal 12 damage with the shot (1 actual damage, 11 overkill). This means that the assisting player who called the damage gets about 10/11ths of the damage reward, or about 27% of the total Leetify Rating for the kill. BOT Bill gets about 3% of the total Leetify Rating from the damage reward. However, BOT Bill will always get 35% of the Leetify Rating for just being the one who actually gets the kill. The damage being communicated provided BOT Bill a huge advantage in the fight, so the assisting player deserves a lot of credit.  

Example 3: Confirmed damage (risky play)

This case is the same as above, but BOT Bill decides to keep the AWP out instead of switching to the pistol. It is much more difficult to hit the AWP shot than to hit a single pistol bullet at any point (you can imagine this same idea with a rifle, in the case of aiming for a headshot vs. aiming for a body shot).  By going for the risky play, you would expect BOT Bill to miss more often in this scenario, get punished, and lose rating in some fraction of these fights as a consequence.

In conclusion, this overkill mechanic rewards everyone when damage is communicated and the safe route to the kill is taken, but does not overly punish a teammate who does not know the opponent is low and goes for a high-damage shot.

That is it for the in depth change list! But you might still have a burning question: how do these changes actually impact the results?

Comparing New and Old Leetify Ratings

The new Leetify Rating has a slightly higher standard deviation than the old rating. This means that the ratings you see will tend to be a little bit farther from zero than before. To show this we have included a comparison of the single-match benchmarks for the new and old rating. Note that these benchmarks are only valid for single games, as you average many games together these no longer hold true. Here is a full breakdown of the old benchmarks compared with the new benchmarks:

In general, your gut feeling for how good a specific Leetify Rating is will still hold true. However, these numbers now capture more dimensions of the game, and someone knows when it is smart to save should see an increase in their ratings moving forward.

Here is a graphic showing the same pro match calculated using the new and old formulas for Leetify Rating. The numbers in the table are the per round average of each Leetify Rating component for each player.

Most players are relatively similar between these two charts. Karrigan benefitted greatly from the End of Round rating calculations, meaning he lives through a lot of round wins and/or was rarely alive when his team lost. This being taken into account, along with the fact that he had the second highest save economy reward, meant that he gained a decent chunk of rating with the new system. The person who benefitted the most from the new system, though, was huNter-. This is partially due to the new kill participation calculation rewarding him more than the old duel reward system (as also appears to be the case with most of the star players in this game).

Am I looking at new or old Leetify Rating?

Any values from before 2022-03-11 will remain unchanged.  This includes match pages, infographics, Year In Review pages, or anywhere else you would see a Leetify Rating. All of these still represent the original calculation. You can see the old benchmarks for these rating numbers in the full Leetify Rating post, or just a few paragraphs above on this page.

Anywhere you see Leetify Rating that has a newer date than 2022-03-11 will be the new calculation for Leetify Rating, as described in this article.  

We will continue to update the original post as more changes are made, but the objective is to only change Leetify Rating when we can address a glaring flaw that has a significant impact on the accuracy of the rating (such as the end of round additions made here). Advanced metrics like this lose a lot of their value if they are constantly changing, and it should be a long time before Leetify Rating needs another upgrade. This said, we do not want to put our pride higher than providing an accurate metric for CS:GO performance. We will certainly update the rating again if there are ways to make it better.

Have ideas for how we can improve Leetify Rating? Or do you want some specific data for a project you are working on? Discord and email are both great ways to reach us, and we would love to hear from you!