|
An Overview of Valve's Roster Lock Policy
A Crude (and Brief) Look at the Frequency of Player Transfers:
Both critics and supporters of the roster lock policy have asserted that Valve implemented the roster locks in order to increase the stability of the scene by reducing player transfers. However, looking at the raw number of transfers made prior to the policy during a seven-month stretch (November to June, 2014-2015) and comparing that to the same period of time the roster lock policy has been in place, the number of player transfers has increased fairly substantially, nearly by 50% (364 for 2014-2015 and 536 for 2015-2016.)
:: Methodology: I referenced the Liquipedia Dota transfer page and compiled all of the Team->None, None->Team, and Team->Team transfers. There is most certainly noise that needs to be eliminated (full teams being picked up counts as five transfers) and I may go back and fix this at a later date.
Fig.1 Number of Player Transfers Per Month, Pre- and Post- the implementation of roster locks
It is clear that, if anything, the stability of the scene has declined during this period of time. Players seem to be joining and leaving teams at a faster rate during this time frame than in the year previous, with the two obvious peaks being the periods of time when the rosters unlocked. The reasons for this could be much more complicated. Teams may be adjusting to the system, potentially dropping and swapping players overzealously. Another possible reason could be that the game is experiencing revitalization, since the Majors may have brought an increase in sponsor interest and organizations looking to pick up a team.
Again, it is important to note that this set of data covers only the same months from 2014-2015 that the roster lock has covered in 2015-2016. The period of time before and after the International was excluded since we have not yet breached that territory yet with roster locks in place.
When you look at the graph for players going from Team->None (kicks and departures) it shows what appears to be an amplification of the past trend.
Fig.2 Number of Player Transfers (Team-None) Per Month, Pre and Post implementation of roster locks
It is entirely possible that this is just noise, or purely coincidence, but none-the-less, it is interesting to use it to speculate how Valve may have anticipated player transfer frequency changing as a result of introducing the Majors. If Valve repeals or drastically adjusts the roster lock policy after this year, it would likely indicate that they anticipated player transfers being somewhere below the blue line after they implemented the policy. If the policy stays in place, it is possible they viewed the policy as a tool to stop the scene from descending into a constant state of roster shuffles, and that player transfers would inevitably rise. In this case, the increase in player transfers may be marginal in comparison to what Valve anticipated, making the program a success.
Impacts of the roster lock: Case Studies
1) Three-two-two and w-three-three
Four days before the Manila Major roster lock, on March 22nd of all dates, Team Secret released their mid-player, w33ha, and their offlane-player Misery. Arteezy and Universe, previously of Evil Geniuses, would fill their positions. The ensuing shuffle brought attention to a potential issue with Valve’s new roster lock system.
(Source: Twitter, @w33ha)
Could teams potentially wait to replace players until immediately before the roster lock in order to prevent them from joining the competition?
Knowing that your team would have to replace you on short notice could be a powerful competitive edge. If you consider the Team Secret and Evil Geniuses situation, the teams in question were considered two of the top teams in the world, and rivals to some extent. Theoretically, players could effectively sandbag their former team by not agreeing to a trade until the last minute, when it would be near impossible to find a suitable replacement on such short notice. While this did not happen, for a brief period of time people believed that Sumail and Fear would also depart Evil Geniuses, leaving PPD as the only remaining member, essentially killing the squad completely. Valve may have been forced to address this issue more seriously if this had been the case. Instead, the scene seemed to stabilize remarkably fast to meet the roster lock deadline, with Digital Chaos absorbing w33ha and Misery and the previous Digital Chaos members all finding homes on new rosters. What this incident showed is that highly valued players will have teams broken and reformed around them, even on very short notice.
Take-away: Misery and w33 being kicked also shows that Valve seems to be hesitant to address something that has yet to become a glaring issue. It is likely that community outrage and Valve’s willingness to dole out punishments after particularly egregious acts (matchfixing, for example) would keep teams and players from blatantly abusing the roster lock system. Players also seem hesitant to fully burn bridges. For example, Arteezy was able to return to both Secret and Evil Geniuses even after high profile departures. The system seems to be partially held in place by good will, something that until they state otherwise, Valve can be said to be endorsing with their lack of explicit input on the matter.
2) The Net debacle
(Source: https://pixabay.com/en/net-material-texture-pattern-macro-984115/)
When Fnatic’s carry player Mushi had health issues and needed to take a brief haitus, a substitute player, 343, was brought onto the roster as a replacement. The team held together well, performing decently despite the absence of Mushi, and their solid past results led them to be invited to the Manila Major.
Mushi returned after his health issues had been resolved, but 343’s strong performance led the team to replace their previous support player, Net, with 343. This was permitted, since 343 had been previously registered as a substitute player by the time of the roster locks and Net was not technically released form the team, rather he stepped down and took the substitute role. Becoming a substitute allowed him to play freely with other teams competing in the open qualifiers, should he desire to do so. A key point to note is that Net agreed to the switch. However, Net later released a statement of his own, indicating clearly that he was frustrated with the way his removal from the team unfolded.
Net’s replacement drew up comparisons to a similar event, when a member of a previous iteration of Fnatic’s Dota squad, Era, was having mental health issues. Era still wanted to compete at the International and eventually claimed that he had been cleared by a doctor and was ready to attend the International. Fnatic claimed that the team was still concerned about his health and had already been practicing with a replacement player. Valve took time to deliberate but ultimately stopped Fnatic from replacing Era, stating that it was the players, not the organization that had earned the invitation to the International.
Take-away: Valve may have incorporated the rule regarding substitutes into the roster lock policy in order to avoid the type of issues that were encountered with Era. Again, rather than commenting directly, Valve has chosen to have a more hands-off approach, seemingly only intervening when the situation deteriorated to a point where the player and team were not making progress towards a mutually amiable solution.
Interestingly, Valve made their position regarding invites and siding with players very clear when addressing the Era issue, but have not stepped in on behalf of Net. This situation may be different given the fact that Net effectively agreed to step down, something that seems to be undisputed given that the has not expressed the intent to take further action.
3) Zai’s Return
After finishing up high school, former Evil Geniuses and Team Secret player Zai announced that he would be playing with the Romanian team Kaipi. Though still registered as a substitute for Evil Geniuses, he was allowed to do so given he went through open qualifiers with his new squad when attempting to compete at a Major or the International.
Take-away: The reason Zai’s time with Kaipi is notable, though only if he proceeds to win the International open qualifiers with his team, is that a player can only be registered to one team when competing in events related to Majors or the International. Should Kaipi win the open qualifiers Zai is essentially in uncharted territory, as apparently Valve hasn’t clarified what would happen in this situation as of yet. It is unclear whether he would be bound to Evil Geniuses as a substitute player, or if winning the open qualifiers would override his substitute status.
4) The revolving door of Team Secret and Evil Geniuses Roster Swaps
(Source: https://pixabay.com/en/door-revolving-door-input-113355/)
Today it was announced that Universe had left Team Secret and returned to Evil Geniuses. He will be swapping places with Bulba, who will be filling the offlane role for Team Secret. Current speculation is that Zai may return to Evil Geniuses main roster and Fear may swap back to the carry position in order to accommodate him, leaving Aui_2000 as the odd man out.
Take-away: These changes to the team’s rosters mean that both Evil Geniuses and Team Secret will have to compete through the open qualifiers. Even when Newbee slumped astoundingly prior to returning to defend their title, they were still directly invited to the following International. It is unclear whether Valve will prioritize the penalty of breaking the roster lock or having Evil Geniuses, the current champions of the International return to defend their throne via a direct invite.
Valve's Approach and the Future of the Roster Lock Policy
The previously listed events indicate strongly that Valve has a very laissez-faire attitude towards events that transpire in the professional Dota 2 scene. This is common knowledge, as barring a few bans for match-fixing and the occasional brief blog post, incidents are typically allowed to run their course without Valve intervention. Ultimately the most reliable indicator of where Valve stands on many of these issues will be their adjustments to the policy rather than any sort of spontaneous adjustments. Until Valve takes action we can only assume that they are observing what has transpired thus-far in regards to the roster lock policy and will incorporate their observations into any adjustments they may make.
Should the w33ha and Misery situation have devolved further, with Evil Geniuses completely disbanding, we would have potentially received a glimpse into Valve’s decision making process. If the either Net situation or the Zai situation comes to a head, we may yet see some definitive action on Valve’s part. Finally, whether or not they invite Evil Geniuses directly to the International this year will serve a strong indicator of how much weight they put in enforcing the roster lock policy, and may show whether or not the policy will be revised following the event.
If I had to place money on Valve’s intentions when implementing the roster lock policy I would bet that Valve anticipated the scene destabilizing if Majors, but no roster locks, were implemented. Many people have treated the policy as something that has decreased the number of transfers, but in actuality the number of transfers has strongly increased during this time frame despite the policy's implementation, and the lack of action on Valve's part may indicate that this is precisely what they expected to happen. Looking forward, it will be compelling to see how Valve addresses the hurdles their policy has encountered, as that should further clarify their intentions for the professional Dota 2 scene, something they have been fairly opaque about in the past.
Max Marion-Spencer Twitter: @bayakadota Wordpress: bayakadota.wordpress.com
|
Few things: 1) Where did you get the data on the roster changes? 2) The data needs to be normalized in some way (player changes per total number of pro-players or similar), and plotted in this normalized fashion.
|
On June 10 2016 07:34 emperorchampion wrote: Few things: 1) Where did you get the data on the roster changes? 2) The data needs to be normalized in some way (player changes per total number of pro-players or similar), and plotted in this normalized fashion.
Ah, my fault, I totally blanked on adding that. I went through and compiled the transfers from the Liquipedia Dota transfer list for each time period. I recorded each Team -> None, None -> Team, and Team -> Team transfer. There are certainly issues with the methodology, but it was meant more as a general point of reference vs. a steadfast statistic. I may go back and eliminate sources of error I know may be skewing it (for example, when a five man team changes sponsor and it is listed as a transfer)
Forgive me, but I'm not entirely sure what you mean. I get the general idea, but if you wouldn't mind explaining a bit further. Part of my motivation for putting this together was to begin the slow process of shoring up my skill with statistics since it's been forever and I'm completely out of shape. Thanks for the feedback.
|
The issue is big regarding your data collection. By just looking at TL transfers, you're limited to whatever people were recording at those times. I don't think it's a stretch to say that the number of transfers would go up if the overall number of Dota teams goes up (which it is over time). If you want a hard piece of data, collect data on just the TI3 teams from 2013(the roster that ended TI) to 2014now (how many transfers did each team have), TI4 teams from 2014end-2015now, and TI5 teams from 2015end-2016now.
Has Valve's roster lock policy created fewer player transfers? That'd be something that you'd have to follow the above to collect hard data on.
Has Valve's roster lock policy created "stability in the scene"? You'd have to define what stability actually is, and measure whether it has happened or not. I think overwhelmingly player and team owners would say no, the scene is not stable. Looking at Zai's case, Universe, Net...it seems like there's no stability. But there are stable teams that do exist (OG, Liquid... Alliance is stable now, but they were shuffling players for a while). What's the right balance? I think it's not a bad system, but people need more transparency and for the rules to be implemented logically. 2 weeks for a 4 month lock was ridiculous, but somehow, it was approved.
|
On June 10 2016 08:16 bayaka wrote:Show nested quote +On June 10 2016 07:34 emperorchampion wrote: Few things: 1) Where did you get the data on the roster changes? 2) The data needs to be normalized in some way (player changes per total number of pro-players or similar), and plotted in this normalized fashion. Ah, my fault, I totally blanked on adding that. I went through and compiled the transfers from the Liquipedia Dota transfer list for each time period. I recorded each Team -> None, None -> Team, and Team -> Team transfer. There are certainly issues with the methodology, but it was meant more as a general point of reference vs. a steadfast statistic. I may go back and eliminate sources of error I know may be skewing it (for example, when a five man team changes sponsor and it is listed as a transfer) Forgive me, but I'm not entirely sure what you mean. I get the general idea, but if you wouldn't mind explaining a bit further. Part of my motivation for putting this together was to begin the slow process of shoring up my skill with statistics since it's been forever and I'm completely out of shape. Thanks for the feedback.
No problemo, it seemed like you were taking things seriously so I wanted to point out a few things I noticed.
You are just looking at raw # of transfers, and saying there is an increase in transfers. But what if we have 100% more players versus last year, but only 50% more transfers. Now you actually would see a decrease in transfers per player, and your conclusion is completely different than before. Here we are normalizing with respect to number of players (each point = #of transfers in the month / total # of players active in the month). I think this is important because we would expect to see an increase in the number of transfers with an increase in players.
In terms of statistics, since it's only two years we can't make any general conclusions yet, maybe 5 or 10 years down the road (depending on how things go...).
|
|
Imo it was somewhat silly to begin with to expect the roster lock system to bring more roster stability.
Some people made fun of the supposed instability in the rosters of top dota teams before the locks, but in reality they generally changed 1 to 2 players 1 to 2 times a year. Especially if it's 1 change a year, it seems very natural to me. At some point you realize whether you still have more capacity to improve, you may need an influx of new ideas in the team, players may lose motivation, relationships may deteriorate etc. 2 changes a year (typically after TI and around new year) is a bit more quick, but basically none of the top teams changed more than that. Lower tier teams often changed more. You are not doing well, you are not getting any prize money, you maybe don't even have a salary, there is little to tie you into a team.
The roster lock system doesn't change any of this. This year there were 3 transfer periods. You are still allowed to do the same amount of changes per year without punishment. There are basically no teams that have stayed the same from pre TI5 era. Some teams formed after TI5 and have stayed the same since (on course for the 1 change per year). Some formed after TI5 and then made another change post Shanghai (most of the Chinese teams). Secret and EG are the exception due to this latest shuffle with 3 changes, but this was imo necessary due to how disastrous the post Shanghai change turned out to be. Otherwise they wouldn't have been any different from what the standard in the dota scene is. And to add, it has been the standard for successful teams. By no means is there evidence that the teams who stay together the longest are the ones who end up succeeding the best imo.
For lower tier teams the situation also isn't particularly different. There's only a certain amount of spots for the regional qualifiers, and if you are not getting invited there you can make as many changes as you want. Not to mention as we have seen in past open qualifiers even those don't seem quite as threatening as they seemed, the teams that have been expected to get out of open quals have typically been the ones who have come through.
|
It's created a Binge & Purge situation, more than it's necessarily changed the roster swapping of the top tier teams. It would take a chunk of time to clean up the data to see how much it's actually effected things. Data Normalization can be fairly difficult, on its own. Especially for a volunteer situation, which is the data set from Liquipedia.
Though I think the Binge & Purge situation is worse for the scene. It produces more chain reactions among the top teams when someone does change. Which I believe is probably worse, over all, for the teams.
There's also the situation where there's no benefit to a team to "lock in" early during the Transfer period. So, by next year, without changes, someone is going to drop a roster change 24h before the deadline.
|
On June 10 2016 10:38 emperorchampion wrote:Show nested quote +On June 10 2016 08:16 bayaka wrote:On June 10 2016 07:34 emperorchampion wrote: Few things: 1) Where did you get the data on the roster changes? 2) The data needs to be normalized in some way (player changes per total number of pro-players or similar), and plotted in this normalized fashion. Ah, my fault, I totally blanked on adding that. I went through and compiled the transfers from the Liquipedia Dota transfer list for each time period. I recorded each Team -> None, None -> Team, and Team -> Team transfer. There are certainly issues with the methodology, but it was meant more as a general point of reference vs. a steadfast statistic. I may go back and eliminate sources of error I know may be skewing it (for example, when a five man team changes sponsor and it is listed as a transfer) Forgive me, but I'm not entirely sure what you mean. I get the general idea, but if you wouldn't mind explaining a bit further. Part of my motivation for putting this together was to begin the slow process of shoring up my skill with statistics since it's been forever and I'm completely out of shape. Thanks for the feedback. No problemo, it seemed like you were taking things seriously so I wanted to point out a few things I noticed. You are just looking at raw # of transfers, and saying there is an increase in transfers. But what if we have 100% more players versus last year, but only 50% more transfers. Now you actually would see a decrease in transfers per player, and your conclusion is completely different than before. Here we are normalizing with respect to number of players (each point = #of transfers in the month / total # of players active in the month). I think this is important because we would expect to see an increase in the number of transfers with an increase in players. In terms of statistics, since it's only two years we can't make any general conclusions yet, maybe 5 or 10 years down the road (depending on how things go...).
Okay, that definitely helps, thanks. It's good to know how it could be "ideal." And that makes sense, the data pool is limited for sure, given of course that it's only two years and that things change so frequently in the Dota world.
On June 10 2016 10:32 schmitty9800 wrote: The issue is big regarding your data collection. By just looking at TL transfers, you're limited to whatever people were recording at those times. I don't think it's a stretch to say that the number of transfers would go up if the overall number of Dota teams goes up (which it is over time). If you want a hard piece of data, collect data on just the TI3 teams from 2013(the roster that ended TI) to 2014now (how many transfers did each team have), TI4 teams from 2014end-2015now, and TI5 teams from 2015end-2016now.
Has Valve's roster lock policy created fewer player transfers? That'd be something that you'd have to follow the above to collect hard data on.
Has Valve's roster lock policy created "stability in the scene"? You'd have to define what stability actually is, and measure whether it has happened or not. I think overwhelmingly player and team owners would say no, the scene is not stable. Looking at Zai's case, Universe, Net...it seems like there's no stability. But there are stable teams that do exist (OG, Liquid... Alliance is stable now, but they were shuffling players for a while). What's the right balance? I think it's not a bad system, but people need more transparency and for the rules to be implemented logically. 2 weeks for a 4 month lock was ridiculous, but somehow, it was approved.
I agree that there is a potential source of error incorporated by using Liquipedia, but there has to be a cut off as far as the quality of the teams you would look at. Liquipedia goes pretty deep into lower-tier teams, and it seems to be the best resource available.
|
On June 10 2016 16:30 spudde123 wrote: Imo it was somewhat silly to begin with to expect the roster lock system to bring more roster stability.
Some people made fun of the supposed instability in the rosters of top dota teams before the locks, but in reality they generally changed 1 to 2 players 1 to 2 times a year. Especially if it's 1 change a year, it seems very natural to me. At some point you realize whether you still have more capacity to improve, you may need an influx of new ideas in the team, players may lose motivation, relationships may deteriorate etc. 2 changes a year (typically after TI and around new year) is a bit more quick, but basically none of the top teams changed more than that. Lower tier teams often changed more. You are not doing well, you are not getting any prize money, you maybe don't even have a salary, there is little to tie you into a team.
The roster lock system doesn't change any of this. This year there were 3 transfer periods. You are still allowed to do the same amount of changes per year without punishment. There are basically no teams that have stayed the same from pre TI5 era. Some teams formed after TI5 and have stayed the same since (on course for the 1 change per year). Some formed after TI5 and then made another change post Shanghai (most of the Chinese teams). Secret and EG are the exception due to this latest shuffle with 3 changes, but this was imo necessary due to how disastrous the post Shanghai change turned out to be. Otherwise they wouldn't have been any different from what the standard in the dota scene is. And to add, it has been the standard for successful teams. By no means is there evidence that the teams who stay together the longest are the ones who end up succeeding the best imo.
For lower tier teams the situation also isn't particularly different. There's only a certain amount of spots for the regional qualifiers, and if you are not getting invited there you can make as many changes as you want. Not to mention as we have seen in past open qualifiers even those don't seem quite as threatening as they seemed, the teams that have been expected to get out of open quals have typically been the ones who have come through.
I definitely agree, but I have to wonder if Valve implemented them because they thought introducing the Majors would cause the whole scene to break apart. That's why I think hopefully when/if they adjust it, we can see how they have felt about it, whether or not they really thought it would lead to less roster swaps overall or just less roster swaps relative to how many there could have been if they didn't put the system in place.
On June 10 2016 20:27 Taf the Ghost wrote: It's created a Binge & Purge situation, more than it's necessarily changed the roster swapping of the top tier teams. It would take a chunk of time to clean up the data to see how much it's actually effected things. Data Normalization can be fairly difficult, on its own. Especially for a volunteer situation, which is the data set from Liquipedia.
Though I think the Binge & Purge situation is worse for the scene. It produces more chain reactions among the top teams when someone does change. Which I believe is probably worse, over all, for the teams.
There's also the situation where there's no benefit to a team to "lock in" early during the Transfer period. So, by next year, without changes, someone is going to drop a roster change 24h before the deadline.
It's obviously hard to say whether it would have happened anyway due to Majors, or if the roster locks just introduced some artificial hysteria regarding roster adjustments. I could see teams viewing the period of time where the rosters unlock with a sense of urgency like a "now or never" type of thing and then booting players for relatively minor infractions or slumps. Intuitively, I think that if they keep the system in place for another year that the number of transfers may level off, and that it's potentially a combination of hysteria and a natural increase due to the implementation of the Majors system that is causing it to spike so much.
|
One possible factor is the fact that the roster lock list being made public allowed for better documentation of the roster changes on liquipedia. Especially for the CN and SEA teams, there are plenty of tier 2/3 teams that made roster changes that LP wouldn't have been able to document without the list published by Valve. Those changes would have happened anyways (both before and after the creation of the transfer period) but they got added to LP this time around only because of the transfer list.
|
On June 11 2016 01:47 tehh4ck3r wrote: One possible factor is the fact that the roster lock list being made public allowed for better documentation of the roster changes on liquipedia. Especially for the CN and SEA teams, there are plenty of tier 2/3 teams that made roster changes that LP wouldn't have been able to document without the list published by Valve. Those changes would have happened anyways (both before and after the creation of the transfer period) but they got added to LP this time around only because of the transfer list.
That's a really strong point actually. Focus on transfers would have definitely been more NA+EU-centric prior to the full list of rosters being made available.
|
I think the introduction of the majors is the key. before there was only The International that had a de facto lock once per year. now there's three explicit ones. From the outside, it seems like players are largely to blame for instability. Weak, unenforceable contracts means that players who are in demand can do whatever they please. It's not as if team owners are swapping players like trading cards like they do in professional sports.
Really it's the organizations and players on unsponsored or small sponsor teams that suffer while the rtz's of the world benefit from their leverage and connections. Valve have shown over the years that they do not want organizations to have more control and do not want to consolidate control themselves. I think they would rather have a player-centric decentralized but slightly chaotic scene rather than one dominated by historically scummy organizations or Riot style control.
Maybe as organizations mature and gain more credibility (i.e. real sports franchises picking up teams) they would be willing to give organizations more tools to promote stability. As for players, the incentives of someone like rtz vs an unknown do not align and so calls for a player's union fizzle out.
|
One problem with the multiple Roster Locks is something that comes up in economics. You can produce Scarcity without anything being actually scarce. You see this in Bank Runs and runs on grocery stores when bad weather is coming in. Your supply chain can be fine, but if some exterior event encourages a large enough portion of people to start buying when they wouldn't, the supply suddenly drops. Then, the next set of people panic and overbuy, and you get your a pretty big shortage when nothing was actually in short supply before the panic.
Thinking on the Roster Lock a bit more, I think the "Invite to Open" effect both isn't enough of a penalty and actually causes more problems for the Tier 2/3 teams. For the Manila Major, we saw 2 Tier 3 teams come through the NA Open and do well. Those teams really needed that experience and it's good for the scene. But say that 2 TI-quality teams are suddenly in the Open Qualifiers of the same Region? You've now blocked off Tier 2/3 teams from getting experience.
We also needs to talk about the two different time-frames of the Roster Lock. There's the "Everyone has Qualified for the Major" time period and the post-Lock but pre-Qualifier period. The first needs to be the hard-lock. Valve doesn't want to deal with replacing teams headed to a Major. But in the 2nd time period, from Lock Date to Invite Date, I think a roster change should just limit a team down 1 phase, rather than 2, which will discourage changing but should mute some of the binge/purge effect. It'll also help alleviate the time when someone actually does blow up a roster 24h before the deadline. Because we know that's coming.
|
|
|
|