Hiveskill convergence rate | Hive 2 better?
barnie
Join Date: 2016-07-26 Member: 220695Members
Join Date: 2016-07-26 Member: 220695Members
After playing some rather one-sided rounds on a server checked the hiveskill on the player that was carrying.
A player with 3.9 KDR but only 1200hivescore @ 66 hours playtime logged on hive.
That was the explanation for the unbalanced rounds, a player like that should be rated 2500+.
So why did hive rate this player so low?
Maybe because he was a comp player mostly playing other comp players?
Or is hive just too slow in coming up with a proper rating?
But 66 hours of logged playtime should be plenty to come up with a rating.
Hive seems to rate skill relative to the players/teams the one is playing against. [citation needed]
So if you mostly play in a certain skill group of players i.e. Comp/NSL then your skill rating is in relation to that group of players and not the playerbase as whole because it is only based on rounds won/lost.
Conclusion: We need a skill rating system that rates player skill relative to the whole playerbase and not only the player you happen to be playing against.
Based on accuracy, KDR, points/sec and wins/losses.
@Nordic i require you to go full explanation mode
A player with 3.9 KDR but only 1200hivescore @ 66 hours playtime logged on hive.
That was the explanation for the unbalanced rounds, a player like that should be rated 2500+.
So why did hive rate this player so low?
Maybe because he was a comp player mostly playing other comp players?
Or is hive just too slow in coming up with a proper rating?
But 66 hours of logged playtime should be plenty to come up with a rating.
Hive seems to rate skill relative to the players/teams the one is playing against. [citation needed]
So if you mostly play in a certain skill group of players i.e. Comp/NSL then your skill rating is in relation to that group of players and not the playerbase as whole because it is only based on rounds won/lost.
Conclusion: We need a skill rating system that rates player skill relative to the whole playerbase and not only the player you happen to be playing against.
Based on accuracy, KDR, points/sec and wins/losses.
@Nordic i require you to go full explanation mode
Comments
Seriously with kdr that high you can wipe out entire packs of aliens/marines and easily win 9/10 games simply by rushing and cutting off the other teams res..
Because that guy being still extremely underrated after ~300 logged rounds is ridiculous!
What's the status of Hive 2, I read there was a test and it got removed it again.
Well maybe he just got himself an aimbot recently.
There are times of day where every server I join has an average hive skill of less than 1000. There are no higher skilled servers I can join. In these servers I have the potential to go 50/5 KDR if I tried. If I played there often I could potentially get >3.5 KDR most games. Does this mean I am a 3000 hive skill player? No. It is reletive. I can go play in a gather full of 3000 hive skill players and <1 KDR.
A players ability to get kills and win engagement is the most important skill in the game. A high KDR is generally a good indicator of a skilled player, but please do not think that is always true. Things are not that black and white.
@Foxy is a moderator and told me not to go full explanation mode. Because I am so kind I will do as you request but I fear I may receive a PM.
Yes, Hive 1 is that slow to accertain a players skill. There are also cases where hive severely undervalues a player for some reason regardless of how much they have played. You may have found one of these players. They do exist.If he went from 0 to 1200 hive skill in 66 hours he is moving up quickly. I have also seen players who are magnitudes more skilled than myself yet have significantly less hive skill. It is hard to know why.
If you don't mind, send me that players hive profile or steam id. I wouldn't mind looking at it, but this is only for my own curiosity.
One of the most anticipated features of Hive 2 is Adagrad. If you are interested here is a scientific paper about it. I am no where near qualified enough to explain it in detail. I can describe how it will effect hive. Adagrad will introduce a variable learning rate into hive such that it learns faster in the beginning and learns slower over time. In the spoiler you will see the results of a simulation comparing Hive 1 + Adagrad in yellow, Hive 1 in red, and the players underlying skill in blue. The X axis is number of rounds played, and the Y axis is hive skill. For reference, the median round length in NS2 is about 15 minutes.
If this picture is still not working you can use this link to see where I got it from.
I do agree that it would be better if hive was less relative to who you play against. This is not exactly a solvable problem. It is not that hive does this poorly, but it is how all skill rating systems work. They learn your skill by comparing you to who you play against. Using accuracy, KDR, points/sec and wins/losses won't change this.
I will also comment that using Win/loss alone does work. You can dig through this post/thread if you would like to see my evidence showing this.
No. Hive 2 is mostly a backend upgrade to the system. It will add in a few new features to the hive algorithm such as Adagrad which I briefly explained earlier in this post. Adagrad will make it so hive finds a players true skill much much quicker. This may be true but don't expect Hive 2 to solve all the problems, make every game balanced, and solve world hunger. Hive 2 will only make it better but there is only so much it can do.
Hive 2 is currently deployed as a mod via the UWE extension system. It is collecting data alongside Hive 1 but is unused in any other way. The hive website only shows information from hive 1 and will not work with hive 2. We may not have a front end website for some time.
Then maybe we need separate hive skill for casual/pub and comp/gather servers?
Because a 3.9 kdr player is going to dominate and destroy balance on the vast majority of pub games they're in... Especially if they have a lower hive score than they should due to playing gathers.
There will not be a separation between Marine/Alien/Commander when it comes to skill values on initial go-live. Moultano and I have discussed the possibility of adding a "Skill-Offset" based on team and/or role; however, these changes will require careful consideration for their impact to the algorithms accuracy. There are a number of shortcomings of Hive-1 that have been addressed with Hive-2 (pertaining to calculations and design). Moultano is in the process of designing a new shuffle algorithm which should help and alleviate the issue people have experienced in the past.
With the new (and entirely reengineered) Hive2 architecture, we will be able to quickly and easily make augmentations and adjustments as needed. For example, there have been 5 updates to Hive2 during the Omega Extension active period (which is still ongoing at the time of this post). These changes have resulted in several bug fixes, performance improvements, and refinements to the underlying system. This type of adaptive and reactive change were simply not possible with the previous Hive1 implementation.
As Nordic explained, Hive2 is superior at finding a player's "true skill" faster and more accurately. As an example, Hive1 would require approximately 200-400 rounds of gameplay in order to "find" a player's skill. Hive2 reduces that, again...approximately to 90 rounds of gameplay. I won't go into innate details of the algorithm and its changes. Moultano has done so already in several posts. [*citations needed]
Long story short, Hive2 will _not_ be the "miracle" fix that, I suspect, some people are assuming it will be. I cannot stress that last statement enough. Rather, it will be another step in improving the overall experience of players for NS2. There is not "silver bullets" or "magic updates" which will suddenly makes things works as desired. That just doesn't happen in the world of software. Instead, these changes are iterative and all heading in the right direction we feel NS2 requires in order to have an opportunity to grow. I should also specifically stipulate, Hive2 will not be the "New NS2Stats". This does not preclude the possibility of Hive2 being a much more rich stats gather source in the future.
I do not post any of this information with the intent of disappointment. Instead, I want NS2 players to have realistic expectations of changes associated with the Hive upgrade. Now, it should be noted that none of this precludes the possibility of further / future improvements. Far from it.
On marine there is literally never ever any time where i contribute to helping the team win..
On alien I couldn't even begin to count how many games I've played a major role in our victory..
My marine score should be 200-500 at the highest.. While my alien score should be closer to 1500-2000..
Separation of marine/alien skill was by far the #1 best thing about hive2.0.. To hear it's not a part of it is saddening.. One skill score for everything is the #1 biggest flaw with the current hive, and now hive 2.0 is going to have the same flaw...
I think we can figure out a good way to learn it, but we'd also have to figure out how to rewrite the balancing algorithms to take per team skills into account, and I'm not sure what the right strategy is. Suppose everyone on the server is better at Marines than Aliens. What criteria do you use to balance the teams in that situation?
As Moultano mentioned, it's not a simple as just recording two different values. There is a LOT of context that would be missing, which is _required_ in order for separated values to function in a useful manner. My sole focus with rewriting Hive (from scratch) is to resolve its two biggest issues: Instability and Inconsistency. As I mentioned before, enhancements can and will be added later.
Back when Matso was working on hive I believe he mentioned he was working on it.
It is good to hear that Hive 2.0 will be a better platform for more sophisticated scoring algorithms.
The user i was talking about here has >300 logged rounds and is still far from where his skill should be.
It is an extreme case of a veteran from before hive coming back starting @ 0 Skill. or a smurf
Maybe players should be put in a skill group a after a few logged rounds by KDR/Accuracy.
For example:
Player has played 12-15 rounds? -> do skillgroup sort. he's got 2.0 > KDR & 30% > accuracy? -> skill is now instantly 2000 -> normal hive proceedings continue from there.
Basically an convergence accelerator for the hive against smurfs.
Balanced team strengths are very important for for enjoyable gameplay these days.
With the shrunk playerbase there is a large disparity between individual players skill and not enough population for every skill level to have their own servers/communities thus the hive balancing has become more important then ever.
Good job for bringing any hopes up for balanced teams and getting the disappointment going..(since you announced that hive2.0 will be there soon)
From my point of view this is the most important issue why hiveskill is so terrible and leads to bad games. And this issue was brought up literally thousands of times.
So, we really have to wait for hive 3.0 which will never come?
I wish I could find the post that said it would be included...
Take a look at this hypothetical balance I created with separate marine and alien skills. You can make a copy and try to balance teams yourself. Is there a way in which none of those players would be forced onto one team or the other more often than not?
Mofo, how would you shuffle a group of 20 players where each player has this kind of skill break down?
But i take faster convergence and better quality over no improvements.
And he said it will be a more versatile platform for new algorithms so separate ratings could be introduced later.
I do think UWE has recognized how important team strength balancing is, why else would they bother to work on it.
Unlikely scenario!
There are quite a few players with exactly the opposite skill distribution, they can shoot quite accurately but they know jacksh*t about alien movement.
If you have for example four players
A (1000m 2000a) (marine skill 1000 alien skill 2000)
B (1000m 2000a)
C (2000m 1000a)
D (2000m 1000a)
There are lots of different ways to shuffle the teams
to achieve equal average skill:
AB marine (average 1000)
CD alens (average 1000)
AC marine (average 1500)
BD alens (average 1500)
CD marine (average 2000)
AB alens (average 2000)
[etc]
If you make shuffle try to maximize average team skill then players will
mostly play on teams where they are at their best (what Mofo is asking for).
The downside would be that you will never be able to play on a team
where your skill is lacking.
Wasn't smallest skill difference between teams with smallest variance the target for the skill solver? [i might be wrong here]
You wouldn't have to make maximum skill on both teams the target.
As i look at it two separate skill values would make for a much larger solution space enabling better quality of shuffle.
(at the expense of more computational resources.)
For example, a 2200 hive skill player is the top 10%. Which ever faction this player is less skilled with would be the team he would play more often than not.
Another example, a 220 skill player is in the bottom 10%. Which ever faction this player is better with would be the team he would be forced to play more often than not.
Mofo claims his marine skill would be about 350 and his alien skill would be about 1750. To use him as an example. If Mofo were to be playing on a low skill level server, he would be marine nearly every single game. If Mofo were to play on a high skill level server, he would be alien nearly every single game. It would not be random like it is now.
The system would always look for the best mathematical outcome which would nearly always have certain players on certain teams. This would result in more balanced games but it would cost player choice. We already have players complaining they are forced to one team too often and it is random. It would not be random with separate team skill values.
Wouldn't additional play time increase their capabilities and eventually lead to a more equal skill level between the two teams for that player?
Forcing players to play on their weaker side most of the times would lead to frustration.
I would just go with a preference setting, giving flexibility to the players.
Vote shuffle went through -> menu pops up: "team preference: [Aliens | dontcare | Marines]"
Note that its preference and not guarantee that you going to end up in that team.
Other then that you could come up with complicated schemes, for example shuffling them to roughly 50/50 playtime in both factions etc.
Yes doing any of the above will likely increase the standard deviation of the shuffle but by how much?
Gamebreakingly much?
Played some really high std-dev games with a prem div comp player + a bunch of noobs vs. a team of average joes.
These rounds endet up surprisingly not terrible.
Its all fun and games until a noob takes the chair.