heroes Comments on Performance-Based Matchmaking by Lead Systems Designer Travis McGeathy

[Stan 5,240]

Redditor Elitesparkle gathered insightful comments by Lead Systems Designer Travis McGeathy about the upcoming performance-based matchmaking system.

Heroes of the Storm will be soon updated with a new MMR system that takes individual gameplay into account. The current system is limited to wining / losing a game. The new system is more accurate, because it uses various parameters to calculate your MMR. Lead Systems Designer Travis McGeathy answered some interesting questions about the new system over on reddit.

Screenshots from the What's Next Panel

The 2018 update is scheduled for mid-December.

Blizzard (Source)

Any changes also planned for uncertainty?

Uncertainty doesn't exist in the same form in the new system. The core concept of something that limits how much MMR can adjust on a per-game basis is there, but the factors that go into it are different.

Looks like new system promotes maining a few heroes rather than flexibility, true or not, and how big is the impact if true?

The most important thing is still winning the game. That hasn't changed. Sometimes, that means sticking to heroes you know really well. Other times, that means having flexibility in your hero roster and playing the heroes that best fit the situation.

Might be a stupid question, but if it's only about winning, why do we have stats-based MMR? why the win isn't the only factor? Isn't the concept of stats-based MMR less accurate than the winrate itself to calculate your odd of winning?

It's about speed more than accuracy. The current system, based entirely on wins vs loses, works, but because your skill is diluted in a team of 5, unless you are dramatically better (or worse) than the rest of the team, it can take a large number of games before the difference in skill translates to more wins (or loses) and moves your MMR to where it should be. By factoring in individual skill to the equation, it can arrive at the proper MMR much faster which means the matchmaker can make better matches overall.

Is draft/banning taken into account? Choosing the right hero in draft may be more important than performing better than peers on a certain hero.

Drafting matters in the same way as the above statement. Drafting is a skill and how well you draft affects your win rate.

All in all, do you have some additional insight into what parameters are used? How zoning, peeling, bodyblocking, skillshot blocking is taken into account? How will Force Wall-Tassadar be able to compete with Archon-Tassadar?

We have great insight into what parameters are used...but aren't going into that level of detail externally.

Also do you plan to add more parameters in the mix?

We'll add more parameters as it makes sense.

There's some concern about how reliable is data on low played heroes — maybe like Chen, Rexxar, TLV, anything on that? (Though I can't imagine any answer here but "we think our data is reliable enough" :D)

We have more than enough data on all existing heroes. For new heroes or major reworks, the performance-based adjustments will basically be disabled until enough data exists.

Are we right to assume that new MMR calculation would be applied to all game modes? For QM, it looks like a step in the direction of per-hero MMR, do you think we may end up there, or at least do you think it may be viable to maintain?

It will apply to all game modes that have MMR.

Can you win a game yet still lose MMR points? IE: a player goes full tilt, intentionally tries to throw. But the other 4 players are just dominating and still win the game.

No. You'll only gain points on a win and lose them on a loss. If you get full carried for a game, you won't lose points, but you also aren't going to gain very many either.

I think people's main concerns are along the lines of "who determines which stats are important" and "why these stats and not these other ones". I assume you guys are running some kind of adaptive machine learning, but most people are not going to understand that so maybe some kind of brief summary into how that works can help to alleviate the main concerns. Just a suggestion.

Quite right. For the "who determines which stats are important", its the players. We've chosen the stats to monitor, but which ones are important for that situation is something we are measuring, not determining ourselves.

About revealing MMR and the general clarity. Assuming "no" on "if you're going to reveal MMR" (:D) I'd like to ask whether we would be able to see when the adjustment takes place on the post-match screen, like we see PRA now.

We won't have visible MMR for this update, but I do still want to do it in the future. For now, you'll mainly see the effects through rank points. We're planning to swap out personal rank adjustment for a more direct performance adjustment which will give better insight while filling a similar role of keeping rank close to MMR. I'm not sure right now whether that will be for this season or the next yet.

Spot on and part of why visible MMR hasn't been a big priority for us but, there are some advantages to it:
- Currently, rank is the only visible indicator of skill. This forces us to link rank to MMR so they don't diverge too much because players get upset when they see people of different rank in their games even if they are actually the same skill. That's not ideal and leads to oddities with matchmaking when rank and MMR do diverge.
- Even though we don't show MMR, players have created other sources to approximate their MMR. The information isn't very accurate, but being the only source of knowledge, it gets cited frequently when players feel there are issues with matchmaking. There's two issues here: first, it gives a false impression that matchmaking is poor for some players and second, it makes it harder to understand when there are actually issues with matchmaking since the real issues can get buried. Effectively, for players that really care about MMR, we already have most of the downsides associated with visible MMR without having the upsides of it being accurate information.

But what order of magnitude are you all thinking? 10 points? 20, 50? Just trying to get a rough understanding of the overall impact it might have on gaining.

That's a tuning point we haven't settled on yet.

The system, unless specifically programmed to do otherwise, will think a death at the last moment is extremely meaningful since it is normally a 60s death timer.

Time is factored into the measurements. If you're dead for the last few seconds of a match, there wouldn't be any measurable impact relative to everything else that happened that game.

Regarding the new performance based MMR changes, does this mean if my team has captured the winning altar on Towers of Doom and for the fun of it I walk into the enemy death zone and die, my MMR will be adversely affected from this little bit of fun? Or perhaps there are 5 enemies killing my core and there are 4 allies dead. I have no chance to defend the core but rather than try and risk receiving an extra death I won't lose as much MMR if I stay in base? Curious about this.

Everything is by time, including deaths. If you die in the last second of a match, its not going to mean much. Even a full death in the last minutes would really only be a minor modifier overall once diluted against your overall performance in the game. The biggest factor for MMR adjustment is still whether you won or lost. Your overall performance is secondary to that and any particular moment is just a small part of that overall performance.

Does this mean that winning a game quickly and playing well will reward the same personal adjustment as having good stats in a long game? Some people are concerned that players will want to artificially extend game lengths to try and pad their stats before winning. Especially in games where the opponent is greatly outmatched.

Yeah, definitely. Everything factors game time into it for that exact reason.

[XeaKon 23]

My conclusions:

-Win ratio is the most important

-Game lenght does not affect performance

-It might promote the usage of the same hero(he tried not to give a clear answear so this means the answear wouldn't please the players)

-We still don't know how MMR is calculated.

[Valhalen 1,484]

11 hours ago, Stan said:

The most important thing is still winning the game. That hasn't changed. Sometimes, that means sticking to heroes you know really well. Other times, that means having flexibility in your hero roster and playing the heroes that best fit the situation.

That will force people to actually learn the Heroes in Try Mode or VS AI before compromising entire team compositions because they don't know how things work. For instance, we all know that "Nova main" who doesn't even know that the red eye icon means you've been detected.

[SleepySheepy 41]

I was hoping new system would go away from winning being only major parameter, because it's team statistic, not something that is entirely upon you.

I'll only mention one other system from I game I used to play, WN8 for World of Tanks.
This measure is community driven, nothing official. Despite being system, well, that calls out for  stat-whoring people (don't know other word), it's really hard to do so that you aren't actively winning. Main component to WN8 was doing damage and assisting others in doing damage, only way to break system was to be able to somehow do a lot of damage on game that will be lost, really hard to do. 
Winrate is part of it, but it only made a small portion.

I'm not sure why are they persistent on winning being main driver of what happens with your MMR.
 

Can you win a game yet still lose MMR points? IE: a player goes full tilt, intentionally tries to throw. But the other 4 players are just dominating and still win the game.

No. You'll only gain points on a win and lose them on a loss. If you get full carried for a game, you won't lose points, but you also aren't going to gain very many either."

They stated it themselves, it's still not accurate in those extremities.

Edited November 14, 2017 by SleepySheepy

[positiv2 952]

2 hours ago, SleepySheepy said:

I was hoping new system would go away from winning being only major parameter, because it's team statistic, not something that is entirely upon you.

I'll only mention one other system from I game I used to play, WN8 for World of Tanks.
This measure is community driven, nothing official. Despite being system, well, that calls out for  stat-whoring people (don't know other word), it's really hard to do so that you aren't actively winning. Main component to WN8 was doing damage and assisting others in doing damage, only way to break system was to be able to somehow do a lot of damage on game that will be lost, really hard to do. 
Winrate is part of it, but it only made a small portion.

I'm not sure why are they persistent on winning being main driver of what happens with your MMR.

The main difference is that World of Tanks has 15v15 matches, where teamwork is not as crucial as in HotS (that's not to say that it is not important in WoT), and since there is no draft phase, you don't have to be able to adapt to enemy team's composition. You can simply roll with a Tier X SPG and get a high rating. If people were to do that in HotS, the team compositions would be pretty bad, so, as they said, those are reflected in the win rate. 

[Undeath 67]

4 hours ago, SleepySheepy said:

I was hoping new system would go away from winning being only major parameter...
 

Can you win a game yet still lose MMR points? IE: a player goes full tilt, intentionally tries to throw. But the other 4 players are just dominating and still win the game.

No. You'll only gain points on a win and lose them on a loss. If you get full carried for a game, you won't lose points, but you also aren't going to gain very many either."

They stated it themselves, it's still not accurate in those extremities.

And they did. Now it's not the only major parameter, but one of many. But it's still a priority, which is very logical, isn't it? Or you were hoping that even when losing a match you could still EARN mmr? I'm sorry but I couldn't help reading that between your lines.

And just skipped the rest of your post about WoT. It's completely irrelevant simply because comparing games of the same genre is a risk of running into wrong argumentation, let alone mentioning games of completely different nature.

[gamerk2 20]

I understand winning and losing being the primary parameters.  If they weren't, EVERYONE would just prioritize Nova and try and go 15-0 every match, losses be damned.

Speaking as someone who hovers around a 50% win rate (keeping in mind I play my lowest ranked hero to fill quests), I have a lot of games with people who, frankly, have no idea about even basic strategy.  I also go on a ton of long (10 games+) loosing streaks.  And I almost never carry.  HotS has been VERY frustrating at times as a result.  As far as I'm concerned, MMR matchmaking can't get much worse then it currently is.

[Esmer 13]

15 hours ago, gamerk2 said:

I understand winning and losing being the primary parameters.  If they weren't, EVERYONE would just prioritize Nova and try and go 15-0 every match, losses be damned.

Speaking as someone who hovers around a 50% win rate (keeping in mind I play my lowest ranked hero to fill quests), I have a lot of games with people who, frankly, have no idea about even basic strategy.  I also go on a ton of long (10 games+) loosing streaks.  And I almost never carry.  HotS has been VERY frustrating at times as a result.  As far as I'm concerned, MMR matchmaking can't get much worse then it currently is.

Yep, matchmaking in HotS is terrible, that's why I quit. Even though I pretty much always had the best stats (xp, damage, ka/d ratio, heal) in lost matches, this system is not making me coming back.

Enough is enough.

Edited November 15, 2017 by Esmer

[Godeyes 25]

Quote

 I assume you guys are running some kind of adaptive machine learning, but most people are not going to understand that so maybe some kind of brief summary into how that works can help to alleviate the main concerns. Just a suggestion.

Why people are so afraid to ask the damn question ? That question is: when will you reveal what is taken into consideration when calculating one's MMR ?

If Blizzard want HotS to be a competetive game then how players are supposed to be competetive when they don;t know how system works ?? This irritates me so much :/ If the system is as good as they describe it then there should be no problem with telling people how does it work exactly. If they don't want to reveal this info then there is sth wrong with it imo.

[Rammbone 0]

I hope HotS dies out as a result of the poor matchmaking that they've had since the start. Came over to see the pretty models left after hitting master q'ing as Aba/Murky. This game is the pretty version of a League of Legends shortbus.

[SleepySheepy 41]

On 14. 11. 2017. at 5:21 PM, Jonar said:

And they did. Now it's not the only major parameter, but one of many. But it's still a priority, which is very logical, isn't it? Or you were hoping that even when losing a match you could still EARN mmr? I'm sorry but I couldn't help reading that between your lines.

And just skipped the rest of your post about WoT. It's completely irrelevant simply because comparing games of the same genre is a risk of running into wrong argumentation, let alone mentioning games of completely different nature.

Yeah, you "read" it right. Seems silly to me that MMR will be positive or negative just depending on win/loss.

Is that wrong thinking? I would say it's not, but of course, I could be wrong.