You seem to assume I haven't bothered to review the thread at all, and that's not the case. But, excuse me for not reading 14 pages of posts that span a time frame of 8 months.
The skills system should be attempting to accomplish an appropriate rating system, nothing else. The only possible way to deviate from that would be suggestions that don't rate players. Attempting to stay with a system that has not and will not ever fully provide reliable data is only putting a bandaid over the real issue.
If you read moultano's original post in the original thread, you can see that the skill system was built to measure the chances of a player winning a match, nothing else. The underlying assumption is that everything a player does is meant to make it more likely that their team will win that match; the ultimate goal is to win and everything else, be it kills, score, teamwork, map presence, aim or communication, should only be a means to an end. By changing the fundamental metric from wins to something else deviates from this assumption. You would no longer be measuring a player's tendency to win, you would be measuring a player's tendency to accrue points in those metrics.
No matter how much expert knowledge and thorough testing went into balancing those alternative metrics, players would always find ways to exploit them. It doesn't even need to be intentional; if the game encourages activity X by awarding a relatively large amount of points from it, experienced and inexperienced players alike will gravitate towards doing that activity, even in situations where it doesn't help their team to win.
Furthermore, points are always situational. The number of possible scenarios is infinite, and the same action in two different scenarios is never of the same value. Killing a fade might be a game-ending feat or a drop in the ocean depending on the bigger picture, yet the same amount of points would be awarded. Even if godlike expert insight could create a system where some of these things were taken into account, they can never be perfectly accurate or even close to accurate, I would argue. Players coming up with novel, unorthodox strategies that all but guarantee a victory would be left for stranded, as the outdated point system does not realise that their actions should be rewarded.
A system based on victories, however, is not situational and could never be outdated. By definition, every action you take that makes it more likely for your team to win will be reflected in your games won, and your skill rating will adjust accordingly. You cannot fool the system. If you find a way to 'exploit' the system and get more relevant victories, you haven't found a hole in the system, you have found a working strategy and should be awarded accordingly. It's then up to the game developers to decide whether that strategy has a place in the game, but within the framework of the current game rules, people who know how to and can use these strategies to win more reliably will have their ratings go up and be considered good, as they should.
Like is the case with many others in this thread, a large amount of your argument stems from personal anecdotes and a gut feeling. If I may play the devil's advocate here and present my own anecdotes, I find that players 'predicting' outcomes based on their 'expert knowledge' of how the game will play out are just as often wrong as they are right. It's just a great, big, satisfactory "I told you so" bias they are experiencing.
You cannot fool the system. If you find a way to 'exploit' the system and get more relevant victories, you haven't found a hole in the system, you have found a working strategy and should be awarded accordingly.
Would you classify stacking as a strategy ? just asking...
I can only think you're not playing NS2 anymore to witness what's happening.
I got a job for you. Give me P as the probability FET has an impact on player retention, and how it has evolved since it was implemented. By extension : measure the progression of stacking from T0 (zero) to actual date. My gut feeling says you would be surprised.
Would you classify stacking as a strategy ? just asking...
It has been explained multiple times why stacking has no effect on the effectiveness of the skill system. If you still keep bringing that up, I have no choice but to assume that you simply do not understand.
Still... Those who stack win. And by doing so increase the skill number they have. Even if it's a little less rewarding than the "unexpected outcome" reward. They win, and their score go up.
So if one team wins because it's stacked (stacked as rookie vs Vets), one must consider : maybe it's a strategy to increase skill score. By doing so, fooling the system in the end. It's dumb but people like growing number. No matter what. No one likes to loose.
Making the system converge with 300 to 500 players, and half of them are stacking... be my guest. Better use a crystal ball.
My critic isn't on the math (or what is math) but on the complete arbitrary assumption that lies behind that tells me one thing. The guy don't understand the many dimension this game has and has reduced to one or two. As i said i would use the Skill system but restricted to the CS and COD kind of game (drone land). Now that i think of it; i guess it can be a copy / past from these games... dunno...
Another example:
2 players have the same score (say 1000) and are of the same skill in other games (K/D scores and stuff).
-Player A plays all week long. 36 games in 7 days. He lost 18 and won 18. He's more likely to stay at the same value he had at day 0: +/- 20 pts.
-Player B plays 3 games on the week end and win. +90 mini. Even when he was completely AFK during one game, he is rewarded.
It is undeniable that player A would be clearly better trained at NS2 than Player B, just by looking at the hours played. Competitively i should add. Still one has a better score than the other.
FET will make teams based on it... and it goes on and on.
That is why I prefer "profiling" players. Having these many scores limited if not enough hours played. Ex :Good aim cannot get 10/10 before 100hrs. Even if there can be abuse. One player can't be fooling every parameter (colors is you remember the "what is skill in the end") all the time. Especially after hundreds of hours played and with many "colors" available.
I think a lot of the complaints are stemming from a negative mindset about FET, a visible skill number, a misunderstanding of how the skill system works, genuine problems inherent with the skill system.
One thing that I feel leads to a misunderstanding of the skill system is our ability to see the number. The skill number is not remotely close to a linear measure. The best example of a similarly hard to understand measure I know of is decibels. You can read on decibels, here.
On top of creating confusing, the number also encourages one to attempt to game the system. It is human nature to want to see a "skill number" go up. The fact that we know what it is encourages people to try and game the system.
We all know the skill system is not perfect. The question seems to be lately, is it better than nothing? To answer this we all rely on our personal experience. Most people only post if they have a negative experience. I being active on these forums, interested in this topic, use my positive experience to try and balance the conversation somewhat.
So why do I have a positive experience? The servers I play on most of the time play with the same group of players with little mixing with outside groups. The servers I play on are not even entirely a community, yet still is the same group even over different servers. On these servers FET is used almost every game and over hundreds of games it feels pretty good. Feels is a subjective word here on purpose. Most of the time when the regulars are on we have good fun balanced games. This could also just be my perception from those games.
I am limited in my experience to playing with these same sets of players on the same sets of servers. The majority of these players skill has come to a convergence for these sets of players. Wooza's server is large and has probably has little mixing with the rest of ns2. I don't play there but I would hypothesis that the skill system works well on that server, because the skills have converged against each other.
The skill system is relative to who you play with. I play with a small number of players, with a large amount of games. Even with as small of a player base as ns2 has, I hypothesis that a lot of players don't mix enough for their skills to converge. This is especially true for competitive players. They don't mix with pubs, and the players don't mix teams. The skill system does not work for competitive gaming because of this.
How do we fix these problems (a negative mindset about FET, a misunderstanding of how the skill system works, genuine problems inherent with the skill system). The negativity is a hard fix. If we fix the problems inherent in the skill system, maybe it will become more neutral but I have no way of knowing. A few have argued for hiding the skill number before but it has never gained traction. I doubt the player base would like having their hard to understand non linear skill number hidden since they had it visible before. The genuine problems are being discussed. Moultano already wants to split up alien and marine skill numbers. He expressed that there are problems with convergence. There are more problems with the skill system that have been discussed previously that are even harder to fix. At this point I would like to see something done, but Moultano is a busy man with a new born baby. I would look to the CDT trello board for more exciting things that are coming.
2.78 win/loss ratio. If you play even games one would expect a win/loss ratio of 1, a bit like flipping a coin. So that player has either won a ton of unfair games, or their skill is actually significantly higher and the skill system has failed to converge on the real high value after 382 hours of recorded gameplay.
A 2.78 W/L ratio is unlikely. There are very few players with those kinds of W/L. Approximately less than 1% of all players have a W/L over 2 . I posted this following histogram previously in my thread giving some basics stats of the hive skill system. Even if you look at the blue line which is players players who have 200+ hours recorded in hive, it is less than 5% who have a W/L over 2. That is how likely a W/L is of 2 or higher, it is much less than 1% for 2.75 W/L and higher.
Another example:
2 players have the same score (say 1000) and are of the same skill in other games (K/D scores and stuff).
-Player A plays all week long. 36 games in 7 days. He lost 18 and won 18. He's more likely to stay at the same value he had at day 0: +/- 20 pts.
-Player B plays 3 games on the week end and win. +90 mini. Even when he was completely AFK during one game, he is rewarded.
It is undeniable that player A would be clearly better trained at NS2 than Player B, just by looking at the hours played. Competitively i should add. Still one has a better score than the other.
Let me give you an analogous example:
-Person A jumps off a moving train, but due to sheer luck, lands on a pile of mattresses and survives unscathed
-Person B is eating cereal, chokes and dies
[sarcasm]Clearly we can't trust any statistics because the statistics I just made up imply that eating cereal is more dangerous than jumping off trains[/sarcasm]
You can conjure your neat little thought experiences all you want, but they have nothing to do with reality. In reality, in the long run, an AFK player will drop down in skill due to being a hindrance to their team, increasing their chance to lose.
At the stack issue: you still do not understand the fundamental way the skill system works. It has been explained many times, several of those explanations aimed directly at you, but you can't grasp it.
We all know the skill system is not perfect. The question seems to be lately, is it better than nothing?
I liked for the time being that the system does pretty much what randomize/shuffle does, yet it has better illusion of fairness and legitimacy and free choice (so people are more likely to vote yes - and it will be better than stacking). To preserve that illusion little longer, it WOULD have been smarter to hide the Skill number, or at least obfuscate it into some military rank or something.
The servers I play on most of the time play with the same group of players with little mixing with outside groups. The servers I play on are not even entirely a community, yet still is the same group even over different servers. On these servers FET is used almost every game and over hundreds of games it feels pretty good. Feels is a subjective word here on purpose. Most of the time when the regulars are on we have good fun balanced games. This could also just be my perception from those games.
If you are a group, it is not unreasonable that you all have comparable (real) skill. If you have comparable skill, then any mapping to teams is balanced (and as such not good measure of the effectiveness of the system).
A 2.78 W/L ratio is unlikely. There are very few players with those kinds of W/L. Approximately less than 1% of all players have a W/L over 2 . I posted this following histogram previously in my thread giving some basics stats of the hive skill system. Even if you look at the blue line which is players players who have 200+ hours recorded in hive, it is less than 5% who have a W/L over 2. That is how likely a W/L is of 2 or higher, it is much less than 1% for 2.75 W/L and higher.
Statistically probability that such players exist are near zero, assuming balanced games (I did calculation of that in the thread you link). Observation of @nemo is correct - they are playing unbalanced games in their favor (or send bad statistics to the hive server). Of course that has little to do with the skill system. It's just stacking and team-switching at its best.
At the stack issue: you still do not understand the fundamental way the skill system works. It has been explained many times, several of those explanations aimed directly at you, but you can't grasp it. Without any insult intended, I must assume that you aren't very smart.
Yet again you play dumb with this wrong example. Get back to the realities of this game. You know the players A will be better than player B just by playing a little more.
And Yet you still pretend FET does the job, while facts aren't on your side at all. We've been through this already but as they say : "repetition is the key to pedagogy".
-Insane stacked team using FET resulting in seeing more stacking behavior. I must not be alone as some people already stated that on their own.
-Nothing is considered except victory while many players think it's just dumb. Especially when you play at high level. Base trade, time, maps, choices, luck, DO HAVE a clearly different weight all things considered. Each one in their own way. Flipping a coin is the same considering the many variables. Even if there was a 200pt to 500pt difference between the 2 pro teams. Anything can happen at the skyscraper level. It's NS.
And when we make a balanced game with a good start (wrong start can happen), NOT using FET. I feel bad about winning. Why ? Because this guys even if they lost the game are also the Vets that i can respect and appreciate for providing such a great challenge. Because they put all their guts in it and they have nothing to be ashamed of when they loose. They can dominate 80% (if not 95%) of the players base while this system says differently as more game are lost.
-This system is described as non linear while "hive" shows a decimal number without units. Not to mention it explains nothing about it. Most people will only take it as a single quantity. Doh!
-This system samples the winnings against the players you usually play with while it is used for any game if FET is voted. Say i move from my usual servers because the ones i used to drop by are empty (sorry for the Australians). The skill number wouldn't be accurate as the number is not representative. Remember, I'm playing with new players only on the week end. The Skill factor was forged against the players on the previous servers (which is dead). It would make me play hundreds of games before FET can be used properly. Are you kidding !!!!
-If 2 player populations start to mix together (Comp and pubbers) they would have to play huge amounts of game in order to get the data converged. It tends to be true as player who actually stay are what's left of Comp and regular pubbers.
So if I'm not very smart; why this system do NOT provide the intended results for everybody, every time, any time, in any situation ? Why the overall score is so badly described (in fact not at all) on the hive website ?
-Why this system look so unforgiving for new player ? They start at 0. More than ever, they probably get the worst feeling when they see the gap between a 2000 and them.
Answer: "you don't understand; it has to converge. play more."
Each new change of behavior (meetingnew ppl) require more games to be played! It's insanely dumb!
Making the data converge will require many more games than what the actual player base can do. It would require playing NS2 like a full time job for many months (even with 300~500 players). Where no NS2 player can do that, wish to do that, and it's assuming the game would have a stable player base. Lately, it still doesn't go up.
Don't you feel there is something dumb about that ?
Especially while they are many more ways to get an accurate evaluation on the many skillS a player has.
It's remarkable that this system is more part of the problem than the solution. That i understand clearly, and nobody have to be smart to get to that conclusion.
The servers I play on most of the time play with the same group of players with little mixing with outside groups. The servers I play on are not even entirely a community, yet still is the same group even over different servers. On these servers FET is used almost every game and over hundreds of games it feels pretty good. Feels is a subjective word here on purpose. Most of the time when the regulars are on we have good fun balanced games. This could also just be my perception from those games.
If you are a group, it is not unreasonable that you all have comparable (real) skill. If you have comparable skill, then any mapping to teams is balanced (and as such not good measure of the effectiveness of the system).
To expand on my experience if it helps discussion, we are not all of similar skill but we are all veterans. I have only looked at a few regulars hive skill just because I was curious but I don't know the average skill for the server. One regular I looked at at ~330 hours recorded in hive but only a ~500 skill number. There are regulars with over 2000 for a skill number. That is just min and max from the few players I have looked at, but there is range. I have ~1800 for a skill number and the server is double down arrow which suggest the average skill on the server is much less than 1800.
Numbers aside the skill does have a bit of range. There is a wide range of accuracy between players. I think the biggest gap in skill comes from a game sense as in what to do in a given situation. We all know not to stay in base but some struggle in deciding when to push and when to hold a phasegate. It is little things like that, that make ns2 really hard to gauge skill.
One of the common commanders is a really great commander who can really rally his team. If he isn't in the chair he has terrible aim. But of course he doesn't always want to be in the chair. Who would?
Just a little aside here -- one thing I don't think I've seen suggested, but would like to see in some form, to desist stacking:
If there's a 400+ difference between average team skills, the winning team loses points and the losing team neither loses or gains points.
You could argue this doesn't help the system, since the stacker player scores might not accurately reflect their skill -- but this would only be true in the instance of consistent stacking. The fact is a stacked game is no game at all. You could ignore the results from a stacked game entirely, however the latter doesn't offer a deterrent for stacking players who -- almost always -- are gaming their hive score by stacking. In that respect, the penalty could be restricted to higher rated or repeat offenders (eh... more variables to store, # of games won with > X skill difference).
Even with a stack, the weaker team always has a chance of winning, no matter how minuscule. This is perfectly in line with the fundamentals of the system. If you say that not a single match ever has been won by the underdog, you are wrong.
Just a little aside here -- one thing I don't think I've seen suggested, but would like to see in some form, to desist stacking:
If there's a 400+ difference between average team skills, the winning team loses points and the losing team neither loses or gains points.
You could argue this doesn't help the system, since the stacker player scores might not accurately reflect their skill -- but this would only be true in the instance of consistent stacking. The fact is a stacked game is no game at all. You could ignore the results from a stacked game entirely, however the latter doesn't offer a deterrent for stacking players who -- almost always -- are gaming their hive score by stacking. In that respect, the penalty could be restricted to higher rated or repeat offenders (eh... more variables to store, # of games won with > X skill difference).
I don't think hive is the main reason people stack. People just want to win, or want to play with their friends, or do it by accident. I would like to pop up a force even teams vote whenever the skill difference is past some threshold though.
@UncleCrunch no matter how stacked the game is, if the hive prediction is right, the expected value of the change to your hive score is 0, so there is no advantage or disadvantage to any team composition. If the hive prediction is wrong, then the expected value moves it in the direction that makes it less wrong.
Just a little aside here -- one thing I don't think I've seen suggested, but would like to see in some form, to desist stacking:
If there's a 400+ difference between average team skills, the winning team loses points and the losing team neither loses or gains points.
Wouldn't that have the exact opposite effect? You're not punishing stacking behavior by giving their hive score a penalty - you're in fact actively encouraging it. Take 3 guys who always stack together with a hive score of 2000. Penalty lowers this down to 1500 now. Now they can legitimately line up against 3 other players on this score, but they'll be stronger, of course.
It also misguides other players. You'll join a game and think: "Oh, the enemy team averages around 1200, our team does too, should be a fair game." - But you get stomped, because enemy team has a bunch of players who are rated way below their actual skill due to this mechanism.
moultanoCreator of ns_shiva.Join Date: 2002-12-14Member: 10806Members, NS1 Playtester, Contributor, Constellation, NS2 Playtester, Squad Five Blue, Reinforced - Shadow, WC 2013 - Gold, NS2 Community Developer, Pistachionauts
@Martigen A more reasonable option is to switch from a logit model to a probit model. This would make the predicted probability of an upset much smaller when there's a big difference in skill, and the skill gained and lost smaller accordingly.
A more complicated factor though is that I think the probability of an upset is asymmetric between the teams. I see a lot of alien upsets via bile / tunnel rushes, but few marine upsets via phase rushes. Aliens tend to always have multiple main bases when winning, and their stuff just takes longer to take down. (That's one of the things I'd like to see fixed. Winning aliens shouldn't be able to relax so much.) Factoring this in is more complicated, and really hard to tell if you've gotten it right. Separate marine and alien skills should mostly take care of it though.
@Martigen A more reasonable option is to switch from a logit model to a probit model. This would make the predicted probability of an upset much smaller when there's a big difference in skill, and the skill gained and lost smaller accordingly.
A more complicated factor though is that I think the probability of an upset is asymmetric between the teams. I see a lot of alien upsets via bile / tunnel rushes, but few marine upsets via phase rushes. Aliens tend to always have multiple main bases when winning, and their stuff just takes longer to take down. (That's one of the things I'd like to see fixed. Winning aliens shouldn't be able to relax so much.) Factoring this in is more complicated, and really hard to tell if you've gotten it right. Separate marine and alien skills should mostly take care of it though.
I think you mis-read the suggestion. It wouldn't alter the score *then*, it's how the points are added or subtracted at the round end going on the player's hive record (and penalising a clear stack).
Even with a stack, the weaker team always has a chance of winning, no matter how minuscule. This is perfectly in line with the fundamentals of the system. If you say that not a single match ever has been won by the underdog, you are wrong.
Oh I agree, and in fact those are some of the best games. I'm talking about stacks where you know the outcome of a match before it even starts, and then it unfolds as the steamroller of arse-rooting that it is. This is easier to see on servers where the average is displayed, not all servers do, and hence suggesting a score diff of at least 400 (maybe 500?). Or...
I would like to pop up a force even teams vote whenever the skill difference is past some threshold though.
See that's a much more sensible idea With one slight adjustment -- just don't let the game start until the difference is < say 350.
Though I also like what you said about probit over logit. And yes, though it's been raised before, having separate marine and alien hive scores would go a long way to helping balanced teams anyway.
Oh I agree, and in fact those are some of the best games. I'm talking about stacks where you know the outcome of a match before it even starts, and then it unfolds as the steamroller of arse-rooting that it is. This is easier to see on servers where the average is displayed, not all servers do, and hence suggesting a score diff of at least 400 (maybe 500?). Or...
No, that is exactly what I mean. "You know the outcome of a match before it even starts" is wrong, since 99% does not equal 100%. There is always a small chance for the weaker team to win, and when that happens, the stackers will get massively penalised for it. They inch their score up with small increments by constantly stacking, but are dropped down a cliff every time the unthinkable, the 1%, happens, resulting in their skill level oscillating around their intrinsic skill. If you win 9 out of 10 times, but gain only 1 point from every victory and lose 10 points from every defeat, you cannot increase your long-run score by stacking, now can you?
This is the entire fundamental idea of the skill system. I and several others have explained it dozens of times so far, but people are still confused. Why is it so hard to understand?
@UncleCrunch no matter how stacked the game is, if the hive prediction is right, the expected value of the change to your hive score is 0, so there is no advantage or disadvantage to any team composition. If the hive prediction is wrong, then the expected value moves it in the direction that makes it less wrong.
In other words you say we have to play wrong games (meaning stacked/imbalanced) in order for the system to make the data converge. And eventually/finally make the right assumption on the outcome of a game.
-Maybe it's boring for some. Playing games that are just worthless. For Winner as for losers, it's just a waste of time.
-Maybe it'll take to many games to get it right. Players don't play the same number of games during a week. Maybe they simply don't want to play with everybody. Maybe it's not possible to try every combination/permutation.
-Maybe NS2 would eventually die before...
To many 'maybe' for me and maybe it eventually is equally felt by the 'usual NS2 rookie' (The Post TV generation). The system has to work now. Not in ten months. The proposal i did (also others did in a different flavor) are quite valid for telling what a player is capable of. I require fewer games to give results. And that's just what we want.
A system is only as good as the data it is based upon. It has to be as much precise as possible. Only store win/loose (against a strength number) is subject to many problems and flaws.
Hmmm. It's nice to loose and earn points.
I think something is broken in the website display of datas. I didn't loose all the games that day, that I'm sure.
I'm also 100% sure that the last NS2 game i played was on the 6Th on Veil. We won. So why a -100 pts gap now ? While there is no record of the last games ?
Maybe :
1/ Some server admin disabled (or forbid) the upload of game results.
2/ Something is broken between some servers and Hive data repository.
3/ Something is broken in Hive itself.
Another set of "maybe"... Surely not helping.
@CDT don't bother fixing Hive. Just 'make' the next build. Some (not me) are impatient apparently .
@UncleCrunch no matter how stacked the game is, if the hive prediction is right, the expected value of the change to your hive score is 0, so there is no advantage or disadvantage to any team composition. If the hive prediction is wrong, then the expected value moves it in the direction that makes it less wrong.
In other words you say we have to play wrong games (meaning stacked/imbalanced) in order for the system to make the data converge
No that's the exact opposite of what I said. Let me work out the math for you. Lets say hive's prediction is Q, and the real probability is P. The update if you win is proportional to (1-Q), and the update if you lose is proportional to -Q. You win with probability P, and lose with probability (1-P). Therefore the expected value is P * (1 - Q) + (1- P) * (-Q). Let's distribute the multiplication. P * (1 - Q) + (1- P) * (-Q) = P - PQ - Q + PQ = P - Q.
No matter what the team composition is, the expected value of the update to your score is proportional to the difference between the real probability and the predicted probability.
@moultano BTW, why does the system assign same bonus/penalty after win/lose to all the team-players and not proportional to their current skill like in traditional Elo?
UncleCrunch, you continue to provide evidence of your complete lack of understanding of both the system and the explanations offered to make it simpler to grasp. Why are you so strongly compelled to criticise something you do not understand?
Who cares if you lose or win points. It isn't your bank account. If the skill numbers were hidden half of the complaints would stop.
The question is not about that... you should know better. I'm way above caring about this score as i think it is utterly not accurate for many players. Good or bad players.
As it has to be explained bit by bit. If there is no recorded activity (at least not shown by Hive website) therefor no new inputs to deal with. How does the score moves from 2054 to 1939 ? And why it shows i lost these 4 games while it was not a legendary bad day ? 1st rule in data collecting : Make sure the data is 100% sure (meaning many thing like CRC, parity and stuff). Maybe it's just a bug on the website. But, maybe...
@moultano BTW, why does the system assign same bonus/penalty after win/lose to all the team-players and not proportional to their current skill like in traditional Elo?
This is suggesting; it's the team performance (outcome of the game) that is "quantified" compared to a Prediction, not individual performances. Personal performances that aren't likely to changes once you have 100hrs played (like accuracy).
No that's the exact opposite of what I said. Let me work out the math for you. Lets say hive's prediction is Q, and the real probability is P. The update if you win is proportional to (1-Q), and the update if you lose is proportional to -Q. You win with probability P, and lose with probability (1-P). Therefore the expected value is P * (1 - Q) + (1- P) * (-Q). Let's distribute the multiplication. P * (1 - Q) + (1- P) * (-Q) = P - PQ - Q + PQ = P - Q.
No matter what the team composition is, the expected value of the update to your score is proportional to the difference between the real probability and the predicted probability.
What are the limits (min max) of P & Q ? I assume that 'real probability' is actually the outcome of the game.
I don't say it doesn't work at all. I say there is no way to tell if a player is good or not at NS2 with this skill number. That is why i say that it would be ok for CS or COD (a drone (or rabbit) game) but not NS2.
The outcome of a NS2 game is unpredictable by definition. Why ? There are events in the game provided by strategic options like tunnel, power nodes etc, that can turn upside down a game. I still vote for more options like that, because commanders start to scan much more when games are lasting (predictability blah blah)... but that's another topic.
It only works in a perfect NS2 world. Unfortunately many things make me think i shouldn't use that adjective about NS2.
Weird : Beside suspicious guys, i did clearly recognize some guys playing NS2 like pro while the Hive says they're new... Probably a second key, isn't it guys . They probably want to see how their skill score evolves. Ultimately their score cannot be used with FET. It wouldn't be wise.
Farming : Maybe some are considering this number to be a "strength" number. Ex : 3 players join as soon as possible the team of their choice. They do have respectable skill score (~1500). The other players join (FET or not). The 3 ruin the game, because in reality they're pros playing in the same team. The question is : What if they were supposed to loose the game ? ...Easy money ?
Slow convergence ? : We know that some players started at 1000 and the last ones started at 0. This suggests the skill number is calculated upon the last known 'position'. Well...
How many games the 2 kinds of rookies (we put aside the vets and +100 hours) have to play together in order to see 'order emerge' (Using FET) ? Especially as krOoze noticed : The whole team gets +X or -X. I believe it would take many more than any other practical system. Right now we know these new players aren't much except fresh meat. But the numbers force FET to consider one is (supposedly) above the other.
Wankers : Anyone seen game in which some jerk ruin the game on purpose ?... I do...
The ultimate goal is to create proper team assignment using FET. The farming and some other things defeat FET in that area as it's mostly the RR players that are assigned. Better make it like this : Vote = Yes, "All RR", FET assign players. After all; it's a vote. Not a semi-vote.
There is still a matter that this skill system/FET do not help with. It's Profiles. You have pushers (pistoleros), and caretaker (Bob the builder). I would say more than 2 kinds but you get the point. Roles are necessary in this game. See replay of last NSL W6 D1 games for more info.
We know we can have the data and process it. One mod show who's "Versed", who's the destructor etc. We have by default K/D, Accuracy, Assist, time building etc, etc... All the things that can be used to properly find who's is good at what. It would help a FET program to properly assign players with better and faster results over time. Who's able to fake it's accuracy ? (aimboter maybe...), Who can fake the time he was building a RT ? etc.
@Martigen A more reasonable option is to switch from a logit model to a probit model. This would make the predicted probability of an upset much smaller when there's a big difference in skill, and the skill gained and lost smaller accordingly.
A more complicated factor though is that I think the probability of an upset is asymmetric between the teams. I see a lot of alien upsets via bile / tunnel rushes, but few marine upsets via phase rushes. Aliens tend to always have multiple main bases when winning, and their stuff just takes longer to take down. (That's one of the things I'd like to see fixed. Winning aliens shouldn't be able to relax so much.) Factoring this in is more complicated, and really hard to tell if you've gotten it right. Separate marine and alien skills should mostly take care of it though.
Power nodes be holding the game back hard
This is way off topic here. You seem to be inserting it into random threads to try to bring attention to it. So again I say:
What is wrong about it? I really want to know.
Why do you think that infestation and powernodes restrict the game too much? Describe what would ns2 look like if it did not have infestation and power nodes? Ns1 isn't an answer.
Make a good argument and the idea will get the attention it deserves. I truly want to hear your thoughts, they just need to be expanded. I like discussing the possibilities.
Comments
You seem to assume I haven't bothered to review the thread at all, and that's not the case. But, excuse me for not reading 14 pages of posts that span a time frame of 8 months.
The skills system should be attempting to accomplish an appropriate rating system, nothing else. The only possible way to deviate from that would be suggestions that don't rate players. Attempting to stay with a system that has not and will not ever fully provide reliable data is only putting a bandaid over the real issue.
No matter how much expert knowledge and thorough testing went into balancing those alternative metrics, players would always find ways to exploit them. It doesn't even need to be intentional; if the game encourages activity X by awarding a relatively large amount of points from it, experienced and inexperienced players alike will gravitate towards doing that activity, even in situations where it doesn't help their team to win.
Furthermore, points are always situational. The number of possible scenarios is infinite, and the same action in two different scenarios is never of the same value. Killing a fade might be a game-ending feat or a drop in the ocean depending on the bigger picture, yet the same amount of points would be awarded. Even if godlike expert insight could create a system where some of these things were taken into account, they can never be perfectly accurate or even close to accurate, I would argue. Players coming up with novel, unorthodox strategies that all but guarantee a victory would be left for stranded, as the outdated point system does not realise that their actions should be rewarded.
A system based on victories, however, is not situational and could never be outdated. By definition, every action you take that makes it more likely for your team to win will be reflected in your games won, and your skill rating will adjust accordingly. You cannot fool the system. If you find a way to 'exploit' the system and get more relevant victories, you haven't found a hole in the system, you have found a working strategy and should be awarded accordingly. It's then up to the game developers to decide whether that strategy has a place in the game, but within the framework of the current game rules, people who know how to and can use these strategies to win more reliably will have their ratings go up and be considered good, as they should.
Like is the case with many others in this thread, a large amount of your argument stems from personal anecdotes and a gut feeling. If I may play the devil's advocate here and present my own anecdotes, I find that players 'predicting' outcomes based on their 'expert knowledge' of how the game will play out are just as often wrong as they are right. It's just a great, big, satisfactory "I told you so" bias they are experiencing.
Would you classify stacking as a strategy ? just asking...
I can only think you're not playing NS2 anymore to witness what's happening.
I got a job for you. Give me P as the probability FET has an impact on player retention, and how it has evolved since it was implemented. By extension : measure the progression of stacking from T0 (zero) to actual date. My gut feeling says you would be surprised.
It has been explained multiple times why stacking has no effect on the effectiveness of the skill system. If you still keep bringing that up, I have no choice but to assume that you simply do not understand.
So if one team wins because it's stacked (stacked as rookie vs Vets), one must consider : maybe it's a strategy to increase skill score. By doing so, fooling the system in the end. It's dumb but people like growing number. No matter what. No one likes to loose.
Making the system converge with 300 to 500 players, and half of them are stacking... be my guest. Better use a crystal ball.
My critic isn't on the math (or what is math) but on the complete arbitrary assumption that lies behind that tells me one thing. The guy don't understand the many dimension this game has and has reduced to one or two. As i said i would use the Skill system but restricted to the CS and COD kind of game (drone land). Now that i think of it; i guess it can be a copy / past from these games... dunno...
Another example:
2 players have the same score (say 1000) and are of the same skill in other games (K/D scores and stuff).
-Player A plays all week long. 36 games in 7 days. He lost 18 and won 18. He's more likely to stay at the same value he had at day 0: +/- 20 pts.
-Player B plays 3 games on the week end and win. +90 mini. Even when he was completely AFK during one game, he is rewarded.
It is undeniable that player A would be clearly better trained at NS2 than Player B, just by looking at the hours played. Competitively i should add. Still one has a better score than the other.
FET will make teams based on it... and it goes on and on.
That is why I prefer "profiling" players. Having these many scores limited if not enough hours played. Ex :Good aim cannot get 10/10 before 100hrs. Even if there can be abuse. One player can't be fooling every parameter (colors is you remember the "what is skill in the end") all the time. Especially after hundreds of hours played and with many "colors" available.
You cannot build a RT and shoot at the same time.
One thing that I feel leads to a misunderstanding of the skill system is our ability to see the number. The skill number is not remotely close to a linear measure. The best example of a similarly hard to understand measure I know of is decibels. You can read on decibels, here.
On top of creating confusing, the number also encourages one to attempt to game the system. It is human nature to want to see a "skill number" go up. The fact that we know what it is encourages people to try and game the system.
We all know the skill system is not perfect. The question seems to be lately, is it better than nothing? To answer this we all rely on our personal experience. Most people only post if they have a negative experience. I being active on these forums, interested in this topic, use my positive experience to try and balance the conversation somewhat.
So why do I have a positive experience? The servers I play on most of the time play with the same group of players with little mixing with outside groups. The servers I play on are not even entirely a community, yet still is the same group even over different servers. On these servers FET is used almost every game and over hundreds of games it feels pretty good. Feels is a subjective word here on purpose. Most of the time when the regulars are on we have good fun balanced games. This could also just be my perception from those games.
I am limited in my experience to playing with these same sets of players on the same sets of servers. The majority of these players skill has come to a convergence for these sets of players. Wooza's server is large and has probably has little mixing with the rest of ns2. I don't play there but I would hypothesis that the skill system works well on that server, because the skills have converged against each other.
The skill system is relative to who you play with. I play with a small number of players, with a large amount of games. Even with as small of a player base as ns2 has, I hypothesis that a lot of players don't mix enough for their skills to converge. This is especially true for competitive players. They don't mix with pubs, and the players don't mix teams. The skill system does not work for competitive gaming because of this.
How do we fix these problems (a negative mindset about FET, a misunderstanding of how the skill system works, genuine problems inherent with the skill system). The negativity is a hard fix. If we fix the problems inherent in the skill system, maybe it will become more neutral but I have no way of knowing. A few have argued for hiding the skill number before but it has never gained traction. I doubt the player base would like having their hard to understand non linear skill number hidden since they had it visible before. The genuine problems are being discussed. Moultano already wants to split up alien and marine skill numbers. He expressed that there are problems with convergence. There are more problems with the skill system that have been discussed previously that are even harder to fix. At this point I would like to see something done, but Moultano is a busy man with a new born baby. I would look to the CDT trello board for more exciting things that are coming.
A 2.78 W/L ratio is unlikely. There are very few players with those kinds of W/L. Approximately less than 1% of all players have a W/L over 2 . I posted this following histogram previously in my thread giving some basics stats of the hive skill system. Even if you look at the blue line which is players players who have 200+ hours recorded in hive, it is less than 5% who have a W/L over 2. That is how likely a W/L is of 2 or higher, it is much less than 1% for 2.75 W/L and higher.
Let me give you an analogous example:
-Person A jumps off a moving train, but due to sheer luck, lands on a pile of mattresses and survives unscathed
-Person B is eating cereal, chokes and dies
[sarcasm]Clearly we can't trust any statistics because the statistics I just made up imply that eating cereal is more dangerous than jumping off trains[/sarcasm]
You can conjure your neat little thought experiences all you want, but they have nothing to do with reality. In reality, in the long run, an AFK player will drop down in skill due to being a hindrance to their team, increasing their chance to lose.
At the stack issue: you still do not understand the fundamental way the skill system works. It has been explained many times, several of those explanations aimed directly at you, but you can't grasp it.
If you are a group, it is not unreasonable that you all have comparable (real) skill. If you have comparable skill, then any mapping to teams is balanced (and as such not good measure of the effectiveness of the system).
Statistically probability that such players exist are near zero, assuming balanced games (I did calculation of that in the thread you link). Observation of @nemo is correct - they are playing unbalanced games in their favor (or send bad statistics to the hive server). Of course that has little to do with the skill system. It's just stacking and team-switching at its best.
Yet again you play dumb with this wrong example. Get back to the realities of this game. You know the players A will be better than player B just by playing a little more.
And Yet you still pretend FET does the job, while facts aren't on your side at all. We've been through this already but as they say : "repetition is the key to pedagogy".
-Insane stacked team using FET resulting in seeing more stacking behavior. I must not be alone as some people already stated that on their own.
-Nothing is considered except victory while many players think it's just dumb. Especially when you play at high level. Base trade, time, maps, choices, luck, DO HAVE a clearly different weight all things considered. Each one in their own way. Flipping a coin is the same considering the many variables. Even if there was a 200pt to 500pt difference between the 2 pro teams. Anything can happen at the skyscraper level. It's NS.
And when we make a balanced game with a good start (wrong start can happen), NOT using FET. I feel bad about winning. Why ? Because this guys even if they lost the game are also the Vets that i can respect and appreciate for providing such a great challenge. Because they put all their guts in it and they have nothing to be ashamed of when they loose. They can dominate 80% (if not 95%) of the players base while this system says differently as more game are lost.
-This system is described as non linear while "hive" shows a decimal number without units. Not to mention it explains nothing about it. Most people will only take it as a single quantity. Doh!
-This system samples the winnings against the players you usually play with while it is used for any game if FET is voted. Say i move from my usual servers because the ones i used to drop by are empty (sorry for the Australians). The skill number wouldn't be accurate as the number is not representative. Remember, I'm playing with new players only on the week end. The Skill factor was forged against the players on the previous servers (which is dead). It would make me play hundreds of games before FET can be used properly. Are you kidding !!!!
-If 2 player populations start to mix together (Comp and pubbers) they would have to play huge amounts of game in order to get the data converged. It tends to be true as player who actually stay are what's left of Comp and regular pubbers.
So if I'm not very smart; why this system do NOT provide the intended results for everybody, every time, any time, in any situation ? Why the overall score is so badly described (in fact not at all) on the hive website ?
-Why this system look so unforgiving for new player ? They start at 0. More than ever, they probably get the worst feeling when they see the gap between a 2000 and them.
Answer: "you don't understand; it has to converge. play more."
Each new change of behavior (meetingnew ppl) require more games to be played! It's insanely dumb!
Making the data converge will require many more games than what the actual player base can do. It would require playing NS2 like a full time job for many months (even with 300~500 players). Where no NS2 player can do that, wish to do that, and it's assuming the game would have a stable player base. Lately, it still doesn't go up.
Don't you feel there is something dumb about that ?
Especially while they are many more ways to get an accurate evaluation on the many skillS a player has.
It's remarkable that this system is more part of the problem than the solution. That i understand clearly, and nobody have to be smart to get to that conclusion.
Numbers aside the skill does have a bit of range. There is a wide range of accuracy between players. I think the biggest gap in skill comes from a game sense as in what to do in a given situation. We all know not to stay in base but some struggle in deciding when to push and when to hold a phasegate. It is little things like that, that make ns2 really hard to gauge skill.
One of the common commanders is a really great commander who can really rally his team. If he isn't in the chair he has terrible aim. But of course he doesn't always want to be in the chair. Who would?
If there's a 400+ difference between average team skills, the winning team loses points and the losing team neither loses or gains points.
You could argue this doesn't help the system, since the stacker player scores might not accurately reflect their skill -- but this would only be true in the instance of consistent stacking. The fact is a stacked game is no game at all. You could ignore the results from a stacked game entirely, however the latter doesn't offer a deterrent for stacking players who -- almost always -- are gaming their hive score by stacking. In that respect, the penalty could be restricted to higher rated or repeat offenders (eh... more variables to store, # of games won with > X skill difference).
More proverbial food for though @moultano!
I don't think hive is the main reason people stack. People just want to win, or want to play with their friends, or do it by accident. I would like to pop up a force even teams vote whenever the skill difference is past some threshold though.
@UncleCrunch no matter how stacked the game is, if the hive prediction is right, the expected value of the change to your hive score is 0, so there is no advantage or disadvantage to any team composition. If the hive prediction is wrong, then the expected value moves it in the direction that makes it less wrong.
Wouldn't that have the exact opposite effect? You're not punishing stacking behavior by giving their hive score a penalty - you're in fact actively encouraging it. Take 3 guys who always stack together with a hive score of 2000. Penalty lowers this down to 1500 now. Now they can legitimately line up against 3 other players on this score, but they'll be stronger, of course.
It also misguides other players. You'll join a game and think: "Oh, the enemy team averages around 1200, our team does too, should be a fair game." - But you get stomped, because enemy team has a bunch of players who are rated way below their actual skill due to this mechanism.
A more complicated factor though is that I think the probability of an upset is asymmetric between the teams. I see a lot of alien upsets via bile / tunnel rushes, but few marine upsets via phase rushes. Aliens tend to always have multiple main bases when winning, and their stuff just takes longer to take down. (That's one of the things I'd like to see fixed. Winning aliens shouldn't be able to relax so much.) Factoring this in is more complicated, and really hard to tell if you've gotten it right. Separate marine and alien skills should mostly take care of it though.
Power nodes be holding the game back hard
Oh I agree, and in fact those are some of the best games. I'm talking about stacks where you know the outcome of a match before it even starts, and then it unfolds as the steamroller of arse-rooting that it is. This is easier to see on servers where the average is displayed, not all servers do, and hence suggesting a score diff of at least 400 (maybe 500?). Or...
See that's a much more sensible idea
Though I also like what you said about probit over logit. And yes, though it's been raised before, having separate marine and alien hive scores would go a long way to helping balanced teams anyway.
No, that is exactly what I mean. "You know the outcome of a match before it even starts" is wrong, since 99% does not equal 100%. There is always a small chance for the weaker team to win, and when that happens, the stackers will get massively penalised for it. They inch their score up with small increments by constantly stacking, but are dropped down a cliff every time the unthinkable, the 1%, happens, resulting in their skill level oscillating around their intrinsic skill. If you win 9 out of 10 times, but gain only 1 point from every victory and lose 10 points from every defeat, you cannot increase your long-run score by stacking, now can you?
This is the entire fundamental idea of the skill system. I and several others have explained it dozens of times so far, but people are still confused. Why is it so hard to understand?
I'm going to put up a poll on this issue to see what most forum-goers think about the current skill system. Here it is: http://forums.unknownworlds.com/discussion/137276/what-do-you-think-of-the-current-skill-calculation-system
Please vote.
People are still writing their theses on things like this, so ... no.
-Maybe it's boring for some. Playing games that are just worthless. For Winner as for losers, it's just a waste of time.
-Maybe it'll take to many games to get it right. Players don't play the same number of games during a week. Maybe they simply don't want to play with everybody. Maybe it's not possible to try every combination/permutation.
-Maybe NS2 would eventually die before...
To many 'maybe' for me and maybe it eventually is equally felt by the 'usual NS2 rookie' (The Post TV generation). The system has to work now. Not in ten months. The proposal i did (also others did in a different flavor) are quite valid for telling what a player is capable of. I require fewer games to give results. And that's just what we want.
A system is only as good as the data it is based upon. It has to be as much precise as possible. Only store win/loose (against a strength number) is subject to many problems and flaws.
Hmmm. It's nice to loose and earn points.
I'm also 100% sure that the last NS2 game i played was on the 6Th on Veil. We won. So why a -100 pts gap now ? While there is no record of the last games ?
Maybe :
1/ Some server admin disabled (or forbid) the upload of game results.
2/ Something is broken between some servers and Hive data repository.
3/ Something is broken in Hive itself.
Another set of "maybe"... Surely not helping.
@CDT don't bother fixing Hive. Just 'make' the next build. Some (not me) are impatient apparently
No matter what the team composition is, the expected value of the update to your score is proportional to the difference between the real probability and the predicted probability.
As it has to be explained bit by bit. If there is no recorded activity (at least not shown by Hive website) therefor no new inputs to deal with. How does the score moves from 2054 to 1939 ? And why it shows i lost these 4 games while it was not a legendary bad day ? 1st rule in data collecting : Make sure the data is 100% sure (meaning many thing like CRC, parity and stuff). Maybe it's just a bug on the website. But, maybe...
This is suggesting; it's the team performance (outcome of the game) that is "quantified" compared to a Prediction, not individual performances. Personal performances that aren't likely to changes once you have 100hrs played (like accuracy).
What are the limits (min max) of P & Q ? I assume that 'real probability' is actually the outcome of the game.
I don't say it doesn't work at all. I say there is no way to tell if a player is good or not at NS2 with this skill number. That is why i say that it would be ok for CS or COD (a drone (or rabbit) game) but not NS2.
The outcome of a NS2 game is unpredictable by definition. Why ? There are events in the game provided by strategic options like tunnel, power nodes etc, that can turn upside down a game. I still vote for more options like that, because commanders start to scan much more when games are lasting (predictability blah blah)... but that's another topic.
It only works in a perfect NS2 world. Unfortunately many things make me think i shouldn't use that adjective about NS2.
Weird : Beside suspicious guys, i did clearly recognize some guys playing NS2 like pro while the Hive says they're new... Probably a second key, isn't it guys
Farming : Maybe some are considering this number to be a "strength" number. Ex : 3 players join as soon as possible the team of their choice. They do have respectable skill score (~1500). The other players join (FET or not). The 3 ruin the game, because in reality they're pros playing in the same team. The question is : What if they were supposed to loose the game ? ...Easy money ?
Slow convergence ? : We know that some players started at 1000 and the last ones started at 0. This suggests the skill number is calculated upon the last known 'position'. Well...
How many games the 2 kinds of rookies (we put aside the vets and +100 hours) have to play together in order to see 'order emerge' (Using FET) ? Especially as krOoze noticed : The whole team gets +X or -X. I believe it would take many more than any other practical system. Right now we know these new players aren't much except fresh meat. But the numbers force FET to consider one is (supposedly) above the other.
Wankers : Anyone seen game in which some jerk ruin the game on purpose ?... I do...
The ultimate goal is to create proper team assignment using FET. The farming and some other things defeat FET in that area as it's mostly the RR players that are assigned. Better make it like this : Vote = Yes, "All RR", FET assign players. After all; it's a vote. Not a semi-vote.
There is still a matter that this skill system/FET do not help with. It's Profiles. You have pushers (pistoleros), and caretaker (Bob the builder). I would say more than 2 kinds but you get the point. Roles are necessary in this game. See replay of last NSL W6 D1 games for more info.
We know we can have the data and process it. One mod show who's "Versed", who's the destructor etc. We have by default K/D, Accuracy, Assist, time building etc, etc... All the things that can be used to properly find who's is good at what. It would help a FET program to properly assign players with better and faster results over time. Who's able to fake it's accuracy ? (aimboter maybe...), Who can fake the time he was building a RT ? etc.
This is way off topic here. You seem to be inserting it into random threads to try to bring attention to it. So again I say:
What is wrong about it? I really want to know.
Why do you think that infestation and powernodes restrict the game too much? Describe what would ns2 look like if it did not have infestation and power nodes? Ns1 isn't an answer.
Make a good argument and the idea will get the attention it deserves. I truly want to hear your thoughts, they just need to be expanded. I like discussing the possibilities.
Maybe you should make a thread about it.