A problem with hive is that it only averages your hive score amoung the communities you play on. (So far I know)
So if you play on 1 server only like forever and get 2000, its not the same as someone who has 2000 playing on many servers.
Not that I have a good idea yet to fix that.
Correct, but there is enough random mixing where the differences are not too great. There is a difference between EU and NA, but we can still change continents without it being a big problem. What is left of the AUS community probably is the most isolated, but their skill values are not unreasonably different.
It is about the same level of problem as the oil leak in my old truck. It is a problem it is annoying, but it does not make it drive different. I could fix it if I rebuilt the entire engine, but it is not a big enough problem to warrant that much effort. My truck is entirely functional, even if it stains driveways.
If you want, we could switch to adam learning rates. https://arxiv.org/pdf/1412.6980.pdf
(this is essentially adagrad + momentum, so if you win a bunch of games in a row, your update keeps going up.)
Only problem with adam is that the update blends the current value with the previous value, so your skill could go down even if you won the game, just because you lost the previous game, which I think people would hate.
There might be some way we could take inspiration from this but prevent that particular behavior, I'd have to think about it a bit.
I think I like that particular behavior. Maybe it will teach people to think that the number shouldn't be that important to them personally.
Nah, we can't change psychology. Bring on the nerd rage.
Since hive 2 was introduced my score fluctuates from 200 all the way up to 1600, and it all depends on which team I get shuffled to.
I can fact check this. This page shows your hive skill value over time since January 2017. It does not fluctuate as much as you say. Before January 2017, you didn't play all that much. You started with a skill value of 1014 when Hive 2 began recording.
Thank you for posting that...
Jan 31st - 989
Feb 4th - 293
Apr 8th - 1776
Seems like it has fluctuated exactly how much I said... As low as the 200's, and as high as the 1700's... Nothing whatsoever has changed regarding my skill level or play style in order to make it drop then go back up by such a huge margin.
@mofo1, no amount of explanation can make up for the deficit of your understanding capability. for months, explanations have been thoroughly tossed your way. you seem to ignore every bit of relevant info, or simply cannot comprehend. if the latter is the case, perhaps its time you keep your misinformed complaintas to yourself instead of perpetuating your false notions time and time again here on the forums.
Your so called "explanations" have not been valid reasons why those two things cannot or should not be done.
I never said they could not be done, or should not be done. I only ever tried to explain why it is not at all easy to implement, and some of the tradeoffs that come with them. I have been wanting separate marine and alien skill values for a long time now. Team preferences would be an amazing feature too.
Since hive 2 was introduced my score fluctuates from 200 all the way up to 1600, and it all depends on which team I get shuffled to. When I get placed on Marine, my hive drops like a rock.. When I get to play Alien, it skyrockets. If I've been playing Alien a lot and my hive skill is around 1500 and I get shuffled to Marine, then the teams are unbalanced because it's expecting me to perform at a 1500 hive skill level, when I can only perform at a 200-500 hive skill level at best.
You said it fluctuated from 200 to 1600 based on what team shuffle places you on. Are you telling me that you have been playing almost exclusively aliens since February 4th? Following your reasoning, that is what your skill over time graph shows. Even if you attempt to claim that you have mostly played alien since February 4th, it still wouldn't matter. That is because you don't understand how the hive skill system works.
Nothing whatsoever has changed regarding my skill level or play style in order to make it drop then go back up by such a huge margin.
This statement clearly shows that you do not understand. I will spend my time once again trying to explain. I will use Moultano's blog post to explain how hive works. I will try to explain it myself in my own language, relating it to your own skill value over time.
Imagine a mountain covered by fog. You are lost somewhere on that mountain, and you are trying to get to the top. When you look around at your surroundings, you can only see a few feet around you. You look around, guess which direction is going most steeply uphill, and set off in that direction. While you are walking, the landscape is too confusing to figure out whether you are still going uphill or not, so you walk for a fixed number of steps in that direction, and then stop to look around again. Eventually, by repeating this process of going up the steepest slope you can find, you hope to get to the top.
This analogy is essentially what computers do when they use an optimization algorithm called “stochastic gradient descent.” While it’s one of the simplest optimization algorithms out there, it is what powers the deep learning revolution that has achieved breakthrough performance on nearly every artificial intelligence task. (To make the analogy more accurate, imagine that the mountain is in a million dimensional space, and you can only see the ground directly under your feet.☺)
Hive uses the same optimization algorithm to improve its (extremely simple) model. And because of that, we can use research originally applied to deep learning to improve how it learns. The new implementation of hive uses Adaptive Subgradient Methods for Online Learning and Stochastic Optimization a.k.a. AdaGrad. Despite the intimidating name, it has an extremely simple implementation and intuition. In our foggy mountain example above, adagrad modifies the distance and direction that you travel so that you go further in directions you haven’t explored before. In hive terms, that means new players will get much larger updates to their skill values than players that hive already knows a lot about. This should dramatically reduce the number of games it takes for hive to converge, and should help to mitigate smurfing.
When hive2 was implemented every player's skill value had the potential to have dramatic updates. This is because of the addition of adagrad learning. The following graph shows the relationship between the learning rate variable called an adagrad sum used by hive to the change of skill value. The greater the absolue change of the skill value, the greater the absolute change in the adagrad sum. The adagrad sum never goes down; it only increases. The rate the adagrad sum increases is determined by the absolute size of the hive skill value. The hive skill value updates based on how unexpected the win or loss was. An unexpected win or loss will make the skill value have a large change. The adagrad sum influences the size of the hive skill value update. The higher the adagrad sum, the smaller the hive skill value update will be to a point. The update size will never be so small that your skill value can't won't change over time. If you have a serious of unexpected wins or losses, your hive skill will move accordingly, but in small increments. This is as opposed to when the adagrad sum is 0 and the hive skill updates can be quite large. This is a machine learning process that helps hive find your true skill value faster. You can think of it as hive becoming more confident about your skill value over time. This is why hive2 makes smurfing not as easy as it was before, which is what the OP was describing.
Your hive skill had such a massive drop, even though your real skill level had not changed, because hive 2 was still learning about you. At that time you did not have many games recorded by hive2. Your adagrad sum was low, you had a very unexpected loss on about February 4th, and hive adjusted your skill accordingly. Hive skill values are not considered significant until after an average of 90 games are recorded. That is an average, which means it could be more or less than 90. At that point in time you had very few games recorded. I know you didn't have many games recorded because I have the data.
It is incredibly unimportant that your hive skill dropped to 200. It happened only because hive was still learning about you. The size of the skill update had nothing to do with your playing aliens or marines more often. It had nothing to do with a change in your skill level or play style, because they did not change. It takes time for hive to learn about a player. The time Hive 2 takes is multiple times faster than hive 1, but it still takes time.
Look in the spoiler at my skill value over 313 rounds from when hive 2 began recording until about December 15th. After my first game I dropped from ~1800 skill to ~400. I even stayed there for about 40 games, despite my true skill value being closer to 2300. Those 40 games had expected outcomes, so my hive skill didn't update very much. I remember I was losing almost every game, even though hive had underrated me. It was quite a long losing streak. The graph ends at 313 games and about 2200 hive skill. My hive skill value is currently at 2400 months later.
If you would like to see more skill graphs over time, here is a small selection of rookies and veteran skill values from when hive 2 began recording until about December 15th. This is a small selection and may not represent all players. It is still interesting to look at. https://imgur.com/a/R2iHz
Look in the spoiler at my skill value over 313 rounds from when hive 2 began recording until about December 15th. After my first game I dropped from ~1800 skill to ~400. I even stayed there for about 40 games, despite my true skill value being closer to 2300. Those 40 games had expected outcomes, so my hive skill didn't update very much. I remember I was losing almost every game, even though hive had underrated me. It was quite a long losing streak. The graph ends at 313 games and about 2200 hive skill. My hive skill value is currently at 2400 months later.
Good for you, your skill is steadily increasing. Interesting that you have a relatively straight line twice. Why is that? Stacks? Even at other stages of more or less constant skill the variance is more visible.
Look in the spoiler at my skill value over 313 rounds from when hive 2 began recording until about December 15th. After my first game I dropped from ~1800 skill to ~400. I even stayed there for about 40 games, despite my true skill value being closer to 2300. Those 40 games had expected outcomes, so my hive skill didn't update very much. I remember I was losing almost every game, even though hive had underrated me. It was quite a long losing streak. The graph ends at 313 games and about 2200 hive skill. My hive skill value is currently at 2400 months later.
Good for you, your skill is steadily increasing. Interesting that you have a relatively straight line twice. Why is that? Stacks? Even at other stages of more or less constant skill the variance is more visible.
Just a long stretch of very expected outcomes. The second time my line flatness I remember it being a mix of wins and losses. I remember the games not seemingly any different than any other time. Flatlines like that are actually fairly common, especially in the highest and lowest skill values.
I am not really steadily increasing. My hive skill was about 1800 when hive 2 began recording. That was around the 10th percentile of skill values. My hive 2 skill value has leveled off at about 2200 a few months ago.
Here is a graph of players hive 2 skill values after 90 games compared to their hive 1 skill values. It is interesting to look at.
You will notice that there was a lot of players who look like they moved from exactly (1000, 1000). These are all players whose hive 1 skill value was s till at their starting skill or not far from it. One of the problems with hive 1 was that some players started from 0 skill value, while others started at 1000. Hive 2 fixed that.
The exactly 1000 hive 1 skill players were great in number. One thing hive 2 fixed was the starting value. Every hive skill was decreased by formula involving playtime. The more playtime you had, the less your skill decreased. Most non-rookie players had their skill values increase with hive 2. My ~1800 skill value in hive 1 was around the10th perecentile, and my ~2200 skill value in hive 2 is also around the 10th percentile.
Even though I am speaking of hive 1 and hive 2 as separate things, they really are not. It is the same system with the same underlying processes. Hive 2 is more like Hive+. It is the same hive system with a tune up.
Look in the spoiler at my skill value over 313 rounds from when hive 2 began recording until about December 15th. After my first game I dropped from ~1800 skill to ~400. I even stayed there for about 40 games, despite my true skill value being closer to 2300. Those 40 games had expected outcomes, so my hive skill didn't update very much. I remember I was losing almost every game, even though hive had underrated me. It was quite a long losing streak. The graph ends at 313 games and about 2200 hive skill. My hive skill value is currently at 2400 months later.
Good for you, your skill is steadily increasing. Interesting that you have a relatively straight line twice. Why is that? Stacks? Even at other stages of more or less constant skill the variance is more visible.
Just a long stretch of very expected outcomes. The second time my line flatness I remember it being a mix of wins and losses. I remember the games not seemingly any different than any other time. Flatlines like that are actually fairly common, especially in the highest and lowest skill values.
Yeah, that's just a different way of calling a stack isn't it? Mathematically, if both teams have similar hive shouldn't either outcome be relatively unexpected? I can't tell the variance just from the graph, but you very obviously gain/lose hive at a higher rate at other times.
Stacks are a possibility, but I remember that the games were unusually even during the second flat line. I remember those games were on TGNS over about 2-3 evenings. Most games I play are on TGNS, but not all. That The first flat line is more unusual in that I lost almost 40 rounds in a row, leaving a near flat line, despite being underrated.
The mathematical function in this link shows the probability of a team winning or losing based on an average hive skill difference.
As I said, those flatlines happen quite frequently and not because of stacks. Stacks can produce that behavior too, but that does not mean it is the sole cause. Look at the other skill over time graphs. It happened a few times with other players, or all the time with some players.
I don't get it. How does hive decide that it is an expected outcome and thus minimal to no adjustment is required if teams are equal? Or is it because teams are equal that hive decides that whichever result is basically a coinflip anyway and doesn't require adjustment?
It has to do with the difference in average skills between teams. It is a little more complicated than that though. If you desire a more in depth explanation you should ask @moultano or @McGlaspie.
Yeah, that's just a different way of calling a stack isn't it? Mathematically, if both teams have similar hive shouldn't either outcome be relatively unexpected? I can't tell the variance just from the graph, but you very obviously gain/lose hive at a higher rate at other times.
What do you mean by "relatively unexpected"? If both teams have the same average skill, then the probability that either team will win should be 50%. Do you mean, "if over a series of rounds wherein both teams have similar hive that a player x wins most of those rounds is relatively unexpected", then yes. However, if it's unexpected then that player's hive skill will consistently rise.
I meant in any single round. Yes, it is 50%, I know that. It was in response to Nordic's assertion that he had a long stretch of very expected outcomes. A very expected outcome to me signifies a stack (per hive score). Maybe 2000 v 500 or something like that where one team has a significant edge over the other and is maybe 90% expected. 50% is relatively unexpected as both teams have an equal chance. A very unexpected outcome would be the 500 beating the 2000.
So for Nordic's hive to flat line indicates to me that he somehow managed to always be in the first scenario (a stack), regardless of whether he wins or loses.
I meant in any single round. Yes, it is 50%, I know that. It was in response to Nordic's assertion that he had a long stretch of very expected outcomes. A very expected outcome to me signifies a stack (per hive score). Maybe 2000 v 500 or something like that where one team has a significant edge over the other and is maybe 90% expected. 50% is relatively unexpected as both teams have an equal chance. A very unexpected outcome would be the 500 beating the 2000.
So for Nordic's hive to flat line indicates to me that he somehow managed to always be in the first scenario (a stack), regardless of whether he wins or loses.
Alright, well I agree that they look like stacks, whether he's winning on the stacked team or losing on the non-stacked team.
So for Nordic's hive to flat line indicates to me that he somehow managed to always be in the first scenario (a stack), regardless of whether he wins or loses.
I remember those games. They did not play like stacks at all. Those flatlines happen quite a lot on a lot of different players. I can't imagine those players finding 40 games in a row, or more, without a shuffle happening.
Almost everyone in the highest and lowest hive skill values have an almost entirely flat line for all games. I think Joshy had one of the most wildly moving lines out of the top of the leader board, but it is still quite flat. Why are almost every game these people play expected regardless of if they win or lose? Is it because they stack every game? Not one of them has an individual win rate over 60%. Losing doesn't seem to ever be unexpected. Do you think this many players can keep up an intentional stack this long with shuffle votes happening multiple times before most games start?
So for Nordic's hive to flat line indicates to me that he somehow managed to always be in the first scenario (a stack), regardless of whether he wins or loses.
I remember those games. They did not play like stacks at all. Those flatlines happen quite a lot on a lot of different players. I can't imagine those players finding 40 games in a row, or more, without a shuffle happening.
You don't need to be on the side of the stack for a game to be a stack; you could be on the anti-stack side. Hence, if you win a stacked game, your elo will barely move, if at all, and if you lose against a stacked team, the same applies.
Almost everyone in the highest and lowest hive skill values have an almost entirely flat line for all games. I think Joshy had one of the most wildly moving lines out of the top of the leader board, but it is still quite flat. Why are almost every game these people play expected regardless of if they win or lose? Is it because they stack every game? Not one of them has an individual win rate over 60%. Losing doesn't seem to ever be unexpected. Do you think this many players can keep up an intentional stack this long with shuffle votes happening multiple times before most games start?
The lowest hive skills are likely mostly comprised of people who haven't yet improved enough to where they can make much of an impact to the game, and so they just throw games over and over until they go past that threshold.
It's probably the middle where you see most movement, where people have learned enough to the point that they make an impact, and add wrinkles to their game so that they make general, steady changes in elo.
The highest skill people are usually people who have spent 213154367346344543k hours in the game and have for the most part plateaued, so their elo isn't moving much.
I think what aeglos is referring to is not a flatline in the sense you're taking it. You could have an elo of 1000 and five games later, an elo of 1000, but there is more than one way it could happen. Here's just two:
Scenario 1:
game 1: 1000
game 2: 1000
game 3: 1000
game 4: 1000
game 5: 1000
Scenario 2:
game 1: 1000
game 2: 950
game 3: 1000
game 4: 1050
game 5: 1000
Scenario 1 - correct me if I'm wrong - would be the result of stacks. You could be on the winning or losing side, so long as when on the winning side you're on the stacked team, and when you're on the losing side you're against the stacked team.
Scenario 2: These matches are likely quite stacked, but where you're losing lots of elo you're on the stacked team, and when you're winning lots you're on the anti-stacked team.
However, upon reviewing that graph you posted above, it looks like the straight line is just you taking a break from the game and coming back to play a game where your elo didn't change (i.e. it was a stacked game).
So another explanation for the relatively 'straight' lines (I'm really thinking of the very first one) is that you played a few relatively stacked rounds, but spaced far more apart. I bet if you played them all in one day that wouldn't look that straight at all.
can you explain to me why balance based on varying factors of below isn't implemented?
K:D, broken down into lifeform play, it's obvious if someone spent 40minutes as a lerk and got 50kills vs someone who went lerk for 40 minutes and got 1 kill who is the more skilled player, the skill transfer isn't that great for alien-marine i know that, but we're not judging everything off one data-set are we now? so why not account for each individual players skill breakdown when implementing it?
for marines it should include
time spent building structures, and kills / death
for aliens it should include
time spent as lifeform and kills / death for those specific life forms
the problem is you're trying to balance tons of bad metrics. most of the reasons stacking happens is because good players want to play on the team of there own choosing, nothing to do with a rookie who has 1500hive1 skill who joins a stacked team, they add nothing to the team other then to add to some bullshit filter that you apply when taking your statistics.
think of it from my point of view, you have a massive inbalance on a server 18 people playing 6 people on the server are really good, but not always does the hive reflect that, sometimes it does but it's not very accurate, so you break down the stats into individual player, for marines as above, time building and k:d, for aliens time spent and k:d (with offset for gorging, heal points and structures built etc etc.) algorithm needs to split the server up into junk and not junk when accounting for that balance, you may have 4 really good players who are consistently good and can ruin the whole game, so you don't stack the guy who builds and has high K;D on marines with the best gorge on the server, you split them onto respective teams, otherwise you're going to get a really good lerk and a really good pistol hero who just shoots to increase his accuracy each game on aliens, which can fuck with balance because he's not a very good lifeform.
Please tell me why this cannot be done and why i am completely wrong in my approach?
@Archie im no stat wiz, but the parameters you've outlined are not sufficient for proper, in-depth skill calculation. what about structure damage? what about "suppressing fire"/"meat shielding"? what about communication? and this is quite important, there are games where there's a mediocre com without a mike, and a good field com makes the actual difference.
Yes, calculating by more parameters seems like a good idea (let's not talk about performance issues), but then you have to account for EVERY parameter, otherwise your new system will be less accurate than the current, "plain" (oh dear, how i wish i could understand that algo a-z) algorithm...
And I don't think it's possible. NS2 highly depends on player cooperation and communication, and I just don't see any program (short of a GAI) that could pull this off =/
can you explain to me why balance based on varying factors of below isn't implemented?
K:D, broken down into lifeform play, it's obvious if someone spent 40minutes as a lerk and got 50kills vs someone who went lerk for 40 minutes and got 1 kill who is the more skilled player, the skill transfer isn't that great for alien-marine i know that, but we're not judging everything off one data-set are we now? so why not account for each individual players skill breakdown when implementing it?
for marines it should include
time spent building structures, and kills / death
for aliens it should include
time spent as lifeform and kills / death for those specific life forms
the problem is you're trying to balance tons of bad metrics. most of the reasons stacking happens is because good players want to play on the team of there own choosing, nothing to do with a rookie who has 1500hive1 skill who joins a stacked team, they add nothing to the team other then to add to some bullshit filter that you apply when taking your statistics.
think of it from my point of view, you have a massive inbalance on a server 18 people playing 6 people on the server are really good, but not always does the hive reflect that, sometimes it does but it's not very accurate, so you break down the stats into individual player, for marines as above, time building and k:d, for aliens time spent and k:d (with offset for gorging, heal points and structures built etc etc.) algorithm needs to split the server up into junk and not junk when accounting for that balance, you may have 4 really good players who are consistently good and can ruin the whole game, so you don't stack the guy who builds and has high K;D on marines with the best gorge on the server, you split them onto respective teams, otherwise you're going to get a really good lerk and a really good pistol hero who just shoots to increase his accuracy each game on aliens, which can fuck with balance because he's not a very good lifeform.
Please tell me why this cannot be done and why i am completely wrong in my approach?
The most reliable metric for evaluating someone's skill is to what extent they contribute to winning rounds. The better you are, the more often you'll be able to win against high-skill opponents. You could get two lerks, one of which goes around picking off marines (useful) but then not realising he might be needed back at the hive for a SG push. You lose the hive and you lose the game. Then you get another lerk who realises that the spikes are incredibly useful for stultifying SG pushes, and so gets back in time. They might end up with the same number of points, KAD, etc but one of them stopped the marines from winning, and the other didn't. It's not normally as simplistic as this but you get the point.
I definitely think, though, that things like umbra should get you more points. You could be the most important factor in a team fight with your umbra, and come out of it with no points, whereas some noob, spazzy Onoses can left-click in the right general direction and come out with 40.
I see what you're saying though. You could get a balance in terms of elo, but a complete imbalance in terms of team composition. For example, i na 6v6 shuffle you could see:
Team 1:
1:Best commander
2:Second best commander.
3:Best gorge
4:Second best gorge
5:Best skulk
6:Second best skulk
Team 2:
1:Best lerk
2:Second best lerk
3:Best fade
4:Second best fade
5:Best LMG
6:Best SG
I think this is generally not that likely though, and if you balanced around stats like K/D it would be more like the first hive system which was shite.
"time spent as lifeform and kills / death for those specific life forms"
Maybe some sort of algorithm that adapts this to time spent as lifeform correlated to win ratio. This would be terribly, terribly complex though. You'd have to take into account when they're lerking, for example. Early lerk is more aggressive, with more bites. Middle lerk is more RT biting with spikes. Late lerk needs to support with umbra etc. And that's just one lifeform.
moultanoCreator of ns_shiva.Join Date: 2002-12-14Member: 10806Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts
One thing to keep in mind, the size of the updates for hive2 are larger at the beginning, but smaller once converged when compared with hive1, so if you are seeing lots of updates in the range +/- 20, that player has probably just converged.
Short summary of answers to other questions:
- We don't use other metrics because metrics become incentives and distort the game. It's easy to accidentally reward spawn camping after the game is over (and many other bad behaviors that are difficult to anticipate in advance.)
- The basics of the algorithm are really simple. Predict the game outcome as a probability that you win. Subtract this probability from the actual game outcome, 1 if you win, 0 if you lose. Multiply by the "learning rate" which is a decreasing function of how many games give has seen you in, and add to your score. If there's an even chance that you win or lose, the reward will be symmetric. You'll gain as much on a win as you lose on a loss. If the prediction is more skewed than the rewards are asymmetric. The difference between this and classic ELO are the details of how leaving and joining teams are accounted for, and the learning rate.
So for Nordic's hive to flat line indicates to me that he somehow managed to always be in the first scenario (a stack), regardless of whether he wins or loses.
I remember those games. They did not play like stacks at all. Those flatlines happen quite a lot on a lot of different players. I can't imagine those players finding 40 games in a row, or more, without a shuffle happening.
Almost everyone in the highest and lowest hive skill values have an almost entirely flat line for all games. I think Joshy had one of the most wildly moving lines out of the top of the leader board, but it is still quite flat. Why are almost every game these people play expected regardless of if they win or lose? Is it because they stack every game? Not one of them has an individual win rate over 60%. Losing doesn't seem to ever be unexpected. Do you think this many players can keep up an intentional stack this long with shuffle votes happening multiple times before most games start?
It doesn't have to be an actual stack. Just a stack by hive. Maybe it was on a captain's night and players who knew you just picked you high regardless. You claim that it happens to a lot of players, but thats missing the point entirely. If it wasn't a stack, then why is this flatline happening? Why is the game, regardless of whether you win or lose however comfortably or badly, regarding it as a foregone conclusion? Especially if its a shuffle as you said?
I didn't find Joshy, but I looked at Wob's (no.1 player apparently). The variance is visible. The flat lines are dates he didnt play.
One thing to keep in mind, the size of the updates for hive2 are larger at the beginning, but smaller once converged when compared with hive1, so if you are seeing lots of updates in the range +/- 20, that player has probably just converged.
Short summary of answers to other questions:
- We don't use other metrics because metrics become incentives and distort the game. It's easy to accidentally reward spawn camping after the game is over (and many other bad behaviors that are difficult to anticipate in advance.)
- The basics of the algorithm are really simple. Predict the game outcome as a probability that you win. Subtract this probability from the actual game outcome, 1 if you win, 0 if you lose. Multiply by the "learning rate" which is a decreasing function of how many games give has seen you in, and add to your score. If there's an even chance that you win or lose, the reward will be symmetric. You'll gain as much on a win as you lose on a loss. If the prediction is more skewed than the rewards are asymmetric. The difference between this and classic ELO are the details of how leaving and joining teams are accounted for, and the learning rate.
That's even more puzzling since Nordic's first flat line is games 2-40. I admit game 1 saw a decrease of over a 1000 points, but what about the subsequent games?
Watching Schu stream just goes to show how hive is not a "skill" based score. His aim way better then mine... his hive score, not way better (4200ish v 4400ish). i guess this shows that you can contribute a lot to a team aside from killing.
All I can guess is that he never initiates cheep wins (CC rush) and generally doesn't play to the "Pub meta". By that I mean when you watch him play he seems to expect his team to be helpful. When there are 2 important objectives he will direct his team to the easier one and take on the more challenging one himself. Now this would be ok if the easier one wasn't critical. (Watched him take Nano while pipe was being arced from neck. 2 onos walked past the past the neck power to die attacking arcs while a third onos killed c12 power)
He also parasites even after armor 1... it's pub, a good skulk can solo 2-3 rines. I guess it's practice for when the rines actually shoot back. I'd hate to see him play to win in pubs instead of practice...
Watching Schu stream just goes to show how hive is not a "skill" based score. His aim way better then mine... his hive score, not way better (4200ish v 4400ish). i guess this shows that you can contribute a lot to a team aside from killing.
All I can guess is that he never initiates cheep wins (CC rush) and generally doesn't play to the "Pub meta". By that I mean when you watch him play he seems to expect his team to be helpful. When there are 2 important objectives he will direct his team to the easier one and take on the more challenging one himself. Now this would be ok if the easier one wasn't critical. (Watched him take Nano while pipe was being arced from neck. 2 onos walked past the past the neck power to die attacking arcs while a third onos killed c12 power)
He also parasites even after armor 1... it's pub, a good skulk can solo 2-3 rines. I guess it's practice for when the rines actually shoot back. I'd hate to see him play to win in pubs instead of practice...
I think you're drawing the wrong lesson here. Yes there are problems with using hive as a measurement of skill, but nonetheless it provides a *rough* measurement of your *relative* skill - i.e. your skill relative to that of your typical opponents. 4k for a player who regularly plays lots of comp =/= 4k for a player who primarily plays in pubs.
(Also, along the lines of what you said, comp players often are not trying their best to win in pubs; instead they are trying to practice something, or they are just messing around waiting for a gather to start, or...)
this is something which has been discussed several times here. your hive skill number is relative to the sum of the skills of players who frequent the same servers you frequent.
not very skilled players can easily inflate their hive scores by winning often on servers which tend to have lower skilled participants.
you only have to be the best on the server you're on at a given time to farm skill. this is why a pubber at 3k is not necessarily as "skilled" as a competitive player at 3k.
The average division 1 competitive players hive skill is 3500. That is well within the top 1% of hive skill values.
I can't tell what your point is, good sir.
As a rough measure of reletive skill, division 1 competitive players have an extremely high reletive skill value.
That's an apt summary of what you said above, but I guess my question was how it fits into the discussion. If the point was supposed to relate to the MadMax/Hobbes/2cough exchange, I didn't see the connection.
The average division 1 competitive players hive skill is 3500. That is well within the top 1% of hive skill values.
I can't tell what your point is, good sir.
As a rough measure of reletive skill, division 1 competitive players have an extremely high reletive skill value.
I agree with Hobbes. What you're saying is, "hive is quite useful in that if you're very good it will say you're very good", but this isn't dealing with MadMax's confusion over the real reason why people like Schu have lower hive scores than people who are clearly not as good as him. I'm pretty sure there are not 18 people who are better at the game than Herakles for example. To be fair though, there are some pub players better than some div 1 players, here and there, but not as many as the hive scores would suggest.
Comments
Correct, but there is enough random mixing where the differences are not too great. There is a difference between EU and NA, but we can still change continents without it being a big problem. What is left of the AUS community probably is the most isolated, but their skill values are not unreasonably different.
It is about the same level of problem as the oil leak in my old truck. It is a problem it is annoying, but it does not make it drive different. I could fix it if I rebuilt the entire engine, but it is not a big enough problem to warrant that much effort. My truck is entirely functional, even if it stains driveways.
I think I like that particular behavior. Maybe it will teach people to think that the number shouldn't be that important to them personally.
Nah, we can't change psychology. Bring on the nerd rage.
Thank you for posting that...
Jan 31st - 989
Feb 4th - 293
Apr 8th - 1776
Seems like it has fluctuated exactly how much I said... As low as the 200's, and as high as the 1700's... Nothing whatsoever has changed regarding my skill level or play style in order to make it drop then go back up by such a huge margin.
Your so called "explanations" have not been valid reasons why those two things cannot or should not be done.
The numbers you said were reasonably close to your high and low, that is correct. What you have incorrect is why it fluctuated like that.
You said it fluctuated from 200 to 1600 based on what team shuffle places you on. Are you telling me that you have been playing almost exclusively aliens since February 4th? Following your reasoning, that is what your skill over time graph shows. Even if you attempt to claim that you have mostly played alien since February 4th, it still wouldn't matter. That is because you don't understand how the hive skill system works.
This statement clearly shows that you do not understand. I will spend my time once again trying to explain. I will use Moultano's blog post to explain how hive works. I will try to explain it myself in my own language, relating it to your own skill value over time.
When hive2 was implemented every player's skill value had the potential to have dramatic updates. This is because of the addition of adagrad learning. The following graph shows the relationship between the learning rate variable called an adagrad sum used by hive to the change of skill value. The greater the absolue change of the skill value, the greater the absolute change in the adagrad sum. The adagrad sum never goes down; it only increases. The rate the adagrad sum increases is determined by the absolute size of the hive skill value. The hive skill value updates based on how unexpected the win or loss was. An unexpected win or loss will make the skill value have a large change. The adagrad sum influences the size of the hive skill value update. The higher the adagrad sum, the smaller the hive skill value update will be to a point. The update size will never be so small that your skill value can't won't change over time. If you have a serious of unexpected wins or losses, your hive skill will move accordingly, but in small increments. This is as opposed to when the adagrad sum is 0 and the hive skill updates can be quite large. This is a machine learning process that helps hive find your true skill value faster. You can think of it as hive becoming more confident about your skill value over time. This is why hive2 makes smurfing not as easy as it was before, which is what the OP was describing.
Your hive skill had such a massive drop, even though your real skill level had not changed, because hive 2 was still learning about you. At that time you did not have many games recorded by hive2. Your adagrad sum was low, you had a very unexpected loss on about February 4th, and hive adjusted your skill accordingly. Hive skill values are not considered significant until after an average of 90 games are recorded. That is an average, which means it could be more or less than 90. At that point in time you had very few games recorded. I know you didn't have many games recorded because I have the data.
It is incredibly unimportant that your hive skill dropped to 200. It happened only because hive was still learning about you. The size of the skill update had nothing to do with your playing aliens or marines more often. It had nothing to do with a change in your skill level or play style, because they did not change. It takes time for hive to learn about a player. The time Hive 2 takes is multiple times faster than hive 1, but it still takes time.
Look in the spoiler at my skill value over 313 rounds from when hive 2 began recording until about December 15th. After my first game I dropped from ~1800 skill to ~400. I even stayed there for about 40 games, despite my true skill value being closer to 2300. Those 40 games had expected outcomes, so my hive skill didn't update very much. I remember I was losing almost every game, even though hive had underrated me. It was quite a long losing streak. The graph ends at 313 games and about 2200 hive skill. My hive skill value is currently at 2400 months later.
If you would like to see more skill graphs over time, here is a small selection of rookies and veteran skill values from when hive 2 began recording until about December 15th. This is a small selection and may not represent all players. It is still interesting to look at.
https://imgur.com/a/R2iHz
I accept this challenge
Good for you, your skill is steadily increasing. Interesting that you have a relatively straight line twice. Why is that? Stacks? Even at other stages of more or less constant skill the variance is more visible.
Just a long stretch of very expected outcomes. The second time my line flatness I remember it being a mix of wins and losses. I remember the games not seemingly any different than any other time. Flatlines like that are actually fairly common, especially in the highest and lowest skill values.
I am not really steadily increasing. My hive skill was about 1800 when hive 2 began recording. That was around the 10th percentile of skill values. My hive 2 skill value has leveled off at about 2200 a few months ago.
Here is a graph of players hive 2 skill values after 90 games compared to their hive 1 skill values. It is interesting to look at.
You will notice that there was a lot of players who look like they moved from exactly (1000, 1000). These are all players whose hive 1 skill value was s till at their starting skill or not far from it. One of the problems with hive 1 was that some players started from 0 skill value, while others started at 1000. Hive 2 fixed that.
The exactly 1000 hive 1 skill players were great in number. One thing hive 2 fixed was the starting value. Every hive skill was decreased by formula involving playtime. The more playtime you had, the less your skill decreased. Most non-rookie players had their skill values increase with hive 2. My ~1800 skill value in hive 1 was around the10th perecentile, and my ~2200 skill value in hive 2 is also around the 10th percentile.
Even though I am speaking of hive 1 and hive 2 as separate things, they really are not. It is the same system with the same underlying processes. Hive 2 is more like Hive+. It is the same hive system with a tune up.
Yeah, that's just a different way of calling a stack isn't it? Mathematically, if both teams have similar hive shouldn't either outcome be relatively unexpected? I can't tell the variance just from the graph, but you very obviously gain/lose hive at a higher rate at other times.
The mathematical function in this link shows the probability of a team winning or losing based on an average hive skill difference.
As I said, those flatlines happen quite frequently and not because of stacks. Stacks can produce that behavior too, but that does not mean it is the sole cause. Look at the other skill over time graphs. It happened a few times with other players, or all the time with some players.
What do you mean by "relatively unexpected"? If both teams have the same average skill, then the probability that either team will win should be 50%. Do you mean, "if over a series of rounds wherein both teams have similar hive that a player x wins most of those rounds is relatively unexpected", then yes. However, if it's unexpected then that player's hive skill will consistently rise.
I don't know if I missed what you meant.
So for Nordic's hive to flat line indicates to me that he somehow managed to always be in the first scenario (a stack), regardless of whether he wins or loses.
Alright, well I agree that they look like stacks, whether he's winning on the stacked team or losing on the non-stacked team.
I'm not sure how it would flatline otherwise.
Almost everyone in the highest and lowest hive skill values have an almost entirely flat line for all games. I think Joshy had one of the most wildly moving lines out of the top of the leader board, but it is still quite flat. Why are almost every game these people play expected regardless of if they win or lose? Is it because they stack every game? Not one of them has an individual win rate over 60%. Losing doesn't seem to ever be unexpected. Do you think this many players can keep up an intentional stack this long with shuffle votes happening multiple times before most games start?
You don't need to be on the side of the stack for a game to be a stack; you could be on the anti-stack side. Hence, if you win a stacked game, your elo will barely move, if at all, and if you lose against a stacked team, the same applies.
The lowest hive skills are likely mostly comprised of people who haven't yet improved enough to where they can make much of an impact to the game, and so they just throw games over and over until they go past that threshold.
It's probably the middle where you see most movement, where people have learned enough to the point that they make an impact, and add wrinkles to their game so that they make general, steady changes in elo.
The highest skill people are usually people who have spent 213154367346344543k hours in the game and have for the most part plateaued, so their elo isn't moving much.
I think what aeglos is referring to is not a flatline in the sense you're taking it. You could have an elo of 1000 and five games later, an elo of 1000, but there is more than one way it could happen. Here's just two:
Scenario 1:
game 1: 1000
game 2: 1000
game 3: 1000
game 4: 1000
game 5: 1000
Scenario 2:
game 1: 1000
game 2: 950
game 3: 1000
game 4: 1050
game 5: 1000
Scenario 1 - correct me if I'm wrong - would be the result of stacks. You could be on the winning or losing side, so long as when on the winning side you're on the stacked team, and when you're on the losing side you're against the stacked team.
Scenario 2: These matches are likely quite stacked, but where you're losing lots of elo you're on the stacked team, and when you're winning lots you're on the anti-stacked team.
However, upon reviewing that graph you posted above, it looks like the straight line is just you taking a break from the game and coming back to play a game where your elo didn't change (i.e. it was a stacked game).
So another explanation for the relatively 'straight' lines (I'm really thinking of the very first one) is that you played a few relatively stacked rounds, but spaced far more apart. I bet if you played them all in one day that wouldn't look that straight at all.
K:D, broken down into lifeform play, it's obvious if someone spent 40minutes as a lerk and got 50kills vs someone who went lerk for 40 minutes and got 1 kill who is the more skilled player, the skill transfer isn't that great for alien-marine i know that, but we're not judging everything off one data-set are we now? so why not account for each individual players skill breakdown when implementing it?
for marines it should include
time spent building structures, and kills / death
for aliens it should include
time spent as lifeform and kills / death for those specific life forms
the problem is you're trying to balance tons of bad metrics. most of the reasons stacking happens is because good players want to play on the team of there own choosing, nothing to do with a rookie who has 1500hive1 skill who joins a stacked team, they add nothing to the team other then to add to some bullshit filter that you apply when taking your statistics.
think of it from my point of view, you have a massive inbalance on a server 18 people playing 6 people on the server are really good, but not always does the hive reflect that, sometimes it does but it's not very accurate, so you break down the stats into individual player, for marines as above, time building and k:d, for aliens time spent and k:d (with offset for gorging, heal points and structures built etc etc.) algorithm needs to split the server up into junk and not junk when accounting for that balance, you may have 4 really good players who are consistently good and can ruin the whole game, so you don't stack the guy who builds and has high K;D on marines with the best gorge on the server, you split them onto respective teams, otherwise you're going to get a really good lerk and a really good pistol hero who just shoots to increase his accuracy each game on aliens, which can fuck with balance because he's not a very good lifeform.
Please tell me why this cannot be done and why i am completely wrong in my approach?
Yes, calculating by more parameters seems like a good idea (let's not talk about performance issues), but then you have to account for EVERY parameter, otherwise your new system will be less accurate than the current, "plain" (oh dear, how i wish i could understand that algo a-z) algorithm...
And I don't think it's possible. NS2 highly depends on player cooperation and communication, and I just don't see any program (short of a GAI) that could pull this off =/
The most reliable metric for evaluating someone's skill is to what extent they contribute to winning rounds. The better you are, the more often you'll be able to win against high-skill opponents. You could get two lerks, one of which goes around picking off marines (useful) but then not realising he might be needed back at the hive for a SG push. You lose the hive and you lose the game. Then you get another lerk who realises that the spikes are incredibly useful for stultifying SG pushes, and so gets back in time. They might end up with the same number of points, KAD, etc but one of them stopped the marines from winning, and the other didn't. It's not normally as simplistic as this but you get the point.
I definitely think, though, that things like umbra should get you more points. You could be the most important factor in a team fight with your umbra, and come out of it with no points, whereas some noob, spazzy Onoses can left-click in the right general direction and come out with 40.
I see what you're saying though. You could get a balance in terms of elo, but a complete imbalance in terms of team composition. For example, i na 6v6 shuffle you could see:
Team 1:
1:Best commander
2:Second best commander.
3:Best gorge
4:Second best gorge
5:Best skulk
6:Second best skulk
Team 2:
1:Best lerk
2:Second best lerk
3:Best fade
4:Second best fade
5:Best LMG
6:Best SG
I think this is generally not that likely though, and if you balanced around stats like K/D it would be more like the first hive system which was shite.
"time spent as lifeform and kills / death for those specific life forms"
Maybe some sort of algorithm that adapts this to time spent as lifeform correlated to win ratio. This would be terribly, terribly complex though. You'd have to take into account when they're lerking, for example. Early lerk is more aggressive, with more bites. Middle lerk is more RT biting with spikes. Late lerk needs to support with umbra etc. And that's just one lifeform.
Short summary of answers to other questions:
- We don't use other metrics because metrics become incentives and distort the game. It's easy to accidentally reward spawn camping after the game is over (and many other bad behaviors that are difficult to anticipate in advance.)
- The basics of the algorithm are really simple. Predict the game outcome as a probability that you win. Subtract this probability from the actual game outcome, 1 if you win, 0 if you lose. Multiply by the "learning rate" which is a decreasing function of how many games give has seen you in, and add to your score. If there's an even chance that you win or lose, the reward will be symmetric. You'll gain as much on a win as you lose on a loss. If the prediction is more skewed than the rewards are asymmetric. The difference between this and classic ELO are the details of how leaving and joining teams are accounted for, and the learning rate.
It doesn't have to be an actual stack. Just a stack by hive. Maybe it was on a captain's night and players who knew you just picked you high regardless. You claim that it happens to a lot of players, but thats missing the point entirely. If it wasn't a stack, then why is this flatline happening? Why is the game, regardless of whether you win or lose however comfortably or badly, regarding it as a foregone conclusion? Especially if its a shuffle as you said?
I didn't find Joshy, but I looked at Wob's (no.1 player apparently). The variance is visible. The flat lines are dates he didnt play.
That's even more puzzling since Nordic's first flat line is games 2-40. I admit game 1 saw a decrease of over a 1000 points, but what about the subsequent games?
All I can guess is that he never initiates cheep wins (CC rush) and generally doesn't play to the "Pub meta". By that I mean when you watch him play he seems to expect his team to be helpful. When there are 2 important objectives he will direct his team to the easier one and take on the more challenging one himself. Now this would be ok if the easier one wasn't critical. (Watched him take Nano while pipe was being arced from neck. 2 onos walked past the past the neck power to die attacking arcs while a third onos killed c12 power)
He also parasites even after armor 1... it's pub, a good skulk can solo 2-3 rines. I guess it's practice for when the rines actually shoot back. I'd hate to see him play to win in pubs instead of practice...
I think you're drawing the wrong lesson here. Yes there are problems with using hive as a measurement of skill, but nonetheless it provides a *rough* measurement of your *relative* skill - i.e. your skill relative to that of your typical opponents. 4k for a player who regularly plays lots of comp =/= 4k for a player who primarily plays in pubs.
(Also, along the lines of what you said, comp players often are not trying their best to win in pubs; instead they are trying to practice something, or they are just messing around waiting for a gather to start, or...)
Plus Schu kinda sucks
not very skilled players can easily inflate their hive scores by winning often on servers which tend to have lower skilled participants.
you only have to be the best on the server you're on at a given time to farm skill. this is why a pubber at 3k is not necessarily as "skilled" as a competitive player at 3k.
so basically what hobbes said.
I can't tell what your point is, good sir.
As a rough measure of reletive skill, division 1 competitive players have an extremely high reletive skill value.
That's an apt summary of what you said above, but I guess my question was how it fits into the discussion. If the point was supposed to relate to the MadMax/Hobbes/2cough exchange, I didn't see the connection.
I agree with Hobbes. What you're saying is, "hive is quite useful in that if you're very good it will say you're very good", but this isn't dealing with MadMax's confusion over the real reason why people like Schu have lower hive scores than people who are clearly not as good as him. I'm pretty sure there are not 18 people who are better at the game than Herakles for example. To be fair though, there are some pub players better than some div 1 players, here and there, but not as many as the hive scores would suggest.