overwatch Season 3 Postmortem and Matchmaking Changes

[Starym 3,800]

Scott Mercer took to the forums to talk about stats for S3

 

Scott Mercer took to the forums to talk about Competitive, specifically stats on Season 3 and how the skill ranking distribution ended up. There's also mention of the matchmaking changes that are going to be made in the future, with emphasis on keeping the differentials in win % capped:

Quote

"Well, we’re not going to compromise as much anymore. The matchmaker will no longer create a match above a certain win percentage threshold, and we’re going to turn this new behavior on very soon."

The rest is very much worth a read and gives a very good look at the numbers nehind the scenes and player stats for Season 3.

Scott Mercer (source)

Thanks for all the feedback about Season 4 of Competitive play and the changes we made. I wanted to first talk about how the distribution of skill rating looks across our competitive population. There’s always been a lot of speculation about this due to the lack of complete “official” data, and there are actually some very distinct differences between the perceived population based on third party sites and our complete data.

The data below is based on season 3 data (season 4 is too new), and the maximum (not current) skill rating of the player during the season. The values below are also rounded, so that’s why it doesn’t quite add up to 100%.

Bronze - 6%
Silver - 22%
Gold - 34%
Platinum - 23%
Diamond - 10%
Master - 3%
Grandmaster - <1%

Some interesting notes:
- Median maximum skill rating is not 2500, it’s actually a little above 2300. So if you have a maximum skill rating of 2350, you have a higher SR than 50% of the population!
- If you do a breakdown based upon the more volatile current skill rating, there’s even fewer players above 3000 SR than listed above. Only around 8% of the population was above 3000 SR for season 3 at any one time.

With this information in mind, let’s talk about skill decay. As mentioned in the competitive blog, only having to play 1 game per week didn’t make your current skill rating very accurate or meaningful. Current Skill Rating should reflect your skill rating as an active player. We also show your Season High Skill Rating in your profile, and that won’t change during a season due to inactivity decay. While you have to play 7 games in a week to avoid any decay, we wanted players to have flexibility within the week. Play in one long session on a weekend, or several smaller sessions during the week, as long as you play seven games it’s fine.

We felt like 3000 Skill Rating was a good breakpoint for the decay rules, as it was around 10% of the competitive population (see above). If you do decay, it only affects your current displayed skill rating. This decay does not affect the internal matchmaking rating we use, so we can still place you in fair matches. When you do come back and actively play matches, you’ll also typically gain more SR from a win until your displayed skill rating and internal matchmaking rating have again reached “equilibrium”.

If you’re curious about when you need to play a game to avoid decay, you can see that by looking on the on the right hand side of the information screen of the competitive play card.

Behind the scenes, we’re going to be making another change to our matchmaking that deserves its own discussion. The simple and primary goal of our matchmaker is creating fair matches. To do that, it evaluates potential matches by synthesizing an expected win %. The matchmaker is normally really good about creating matches with a win % that is close to 50%, but if the participants in the match are either at very high or very low SRs then it gets quite a bit more difficult. There simply aren’t many players at the extremes of SR to find. This is especially true in our lower population regions AND they’re playing at off peak times such as 04:00 in the morning AND the players queue in a large group. Normally in these situations, we wait to try and find a good match, but eventually compromises have to be made so the players in question don’t wait forever in queue.

Well, we’re not going to compromise as much anymore. The matchmaker will no longer create a match above a certain win percentage threshold, and we’re going to turn this new behavior on very soon. When we do, you could possibly wait a very long time or even not find a match at all in extreme cases. To find fair matches quickly, try to play during prime hours for your region. It also easier to match you if you’re solo or in a smaller group. In a future patch, when we think we can’t find a match based on the current matchmaking population then we’ll warn you that the wait might be a very long time. We didn’t want to overreact and create a strict rule such as “You can’t queue as a group above X Skill Rating.” There are some locations and times that can handle high skill groups and still find fair matches for them. We’ll start with this new threshold not being very aggressive, but we can adjust dynamically if needed. When we do add the UI, we’ll also make the check more aggressive.

Moving forward, we are currently looking at our win/loss streak bonuses and SR volatility, tuning and improving our expected win % calculations, and other elements of the matchmaking system. Competitive Play and Matchmaking are two systems that we’re constantly working to refine over time, and as always your feedback helps us tremendously. Keep it coming, and good luck in season 4!