An overhaul of the skill system.
FriendlyNeighbour
North Join Date: 2015-01-25 Member: 201047Members
North Join Date: 2015-01-25 Member: 201047Members
I know there are threads but this is really just a break-away discussion. Instead of coming to a concensus I have a short-term bridge-gap solution (for a lack of a better word) to push.
Everyone know that the "skill" system is broken. In my view this is because stacking is rampant on NS2. Since skill is heavily weighted towards win/lose ratio, this means people who are persistent stackers(which means most NS2 players) will get highly inflated skill points. Time spent is also not a good proxy.
I've often found that K/D ratio is much more accurate than W/L ratio. It's not hard to find people with 1.5 W/L ratio with shit K/D ratio.
It is rare to find people with a K/D ratio above 1.5 with a dismal W/L ratio. I can see three arguments, basically:
1." K/D ratio preference doesn't reward good gorges".
This is true, but this can be compensated with the current score system. As long as K/D is given primacy (up to a certain point, say 1.4 K/D and then getting diminishing returns) the gorge question is less relevant. Also, the amount of people who only play gorge are quite small in my experience(120 hours logged).
2. "K/D ratio preference incentivizes people to care less about teamwork".
This is a variation on the gorge argument but from a systemic point of view rather than what's good for the individual player.
My counter-response is let's try that proposition. Because people I see with high K/D have strong teamworking abilities. Maybe because if you wanna kill lots of people, it helps having a lot of upgrades, which in turn depends on getting most of the resources which in turn depends on team work. The concept of the gimped skulk with almost no upgrades crushing a bunch of lvl 3 weapon jet-packing, shotgun-toting marines is a fanciful concept. Ditto the lvl 1 weapon marine facing high-powered fades and onoses.
3. "The system is fine as it is! Only need a few tweaks!"
No. The system is totally broken. "Skill level" as it is as of right now is often very divorced from actual skill. K/D is a proxy. It's not a perfect solution, which is why I'm branding it as a stop-gap measure. But it is certainly a MUCH better guide to how a good a player actually is(and not just at killing things, I'd like to point out) than stack-inflated W/L ratio primacy.
Above a certain K/D, say, 1.4 or 1.5 I'd say that other factors should be much more important. But what we can't have is the current system which is totally bloated.
Everyone know that the "skill" system is broken. In my view this is because stacking is rampant on NS2. Since skill is heavily weighted towards win/lose ratio, this means people who are persistent stackers(which means most NS2 players) will get highly inflated skill points. Time spent is also not a good proxy.
I've often found that K/D ratio is much more accurate than W/L ratio. It's not hard to find people with 1.5 W/L ratio with shit K/D ratio.
It is rare to find people with a K/D ratio above 1.5 with a dismal W/L ratio. I can see three arguments, basically:
1." K/D ratio preference doesn't reward good gorges".
This is true, but this can be compensated with the current score system. As long as K/D is given primacy (up to a certain point, say 1.4 K/D and then getting diminishing returns) the gorge question is less relevant. Also, the amount of people who only play gorge are quite small in my experience(120 hours logged).
2. "K/D ratio preference incentivizes people to care less about teamwork".
This is a variation on the gorge argument but from a systemic point of view rather than what's good for the individual player.
My counter-response is let's try that proposition. Because people I see with high K/D have strong teamworking abilities. Maybe because if you wanna kill lots of people, it helps having a lot of upgrades, which in turn depends on getting most of the resources which in turn depends on team work. The concept of the gimped skulk with almost no upgrades crushing a bunch of lvl 3 weapon jet-packing, shotgun-toting marines is a fanciful concept. Ditto the lvl 1 weapon marine facing high-powered fades and onoses.
3. "The system is fine as it is! Only need a few tweaks!"
No. The system is totally broken. "Skill level" as it is as of right now is often very divorced from actual skill. K/D is a proxy. It's not a perfect solution, which is why I'm branding it as a stop-gap measure. But it is certainly a MUCH better guide to how a good a player actually is(and not just at killing things, I'd like to point out) than stack-inflated W/L ratio primacy.
Above a certain K/D, say, 1.4 or 1.5 I'd say that other factors should be much more important. But what we can't have is the current system which is totally bloated.
Comments
Skill system could be useful if we had matchmaking but we do not have any.
The current system does not measure skill. It is a misnomer to label it as "skill rating" or whatever. The score each of us has actually reflects the likelihood of being on the winning team through an unprincipled, irregular, and unpredictable allocation system controlled by team-switchers and randomize ready room votes. The likelihood of being on the winning team, for the majority of players, is totally random. I don't know what possessed the designers of the system to delude themselves that the likelihood of being on a winning team can be a surrogate for skill, let alone to actually name a system which doesn't actually measure skill as a "skill" system.
The reason the skill system was created was as a means to balance teams. Sadly the system punishes players who end up on a losing streak. This actually seems to be by design, as a team full of high skilled players with a couple of rookies can have a low average team score, and then could beat a team with a higher average score (because there are no rookies on that side, for example), giving everyone on that winning team a huge point boost. A player who has done well on the losing team ends up losing a ton of points.
Ideally NS2 wouldn't have any skill system with which to balance teams. The strangeness of the game itself is to blame. The developers created a game where it is almost impossible to have a fair match. This is in stark contrast to every other team based game out there. Unlike most other games, in NS2 a match where there is no stomp is the exception to the rule.
I stopped reading here, thanks for another thread.
Okay, I lied. I read your post. You are wrong on immensely many points, and all of the things you mentioned have been discussed in the previous thread.
You guys seem to care too much about this. Do you think the other games do it better? They all do essentially the same. The only difference is that instead of calling it "skill level" they call it "league level" or ranking
I searched for CS:GO ranking system: https://steamcommunity.com/sharedfiles/filedetails/?id=312582297
it is essentially the same, but they give extra points to the MVP player
Do you know the REAL difference between other popular games like League of Legends and NS2?
It is not that their skill system is better. Actually their system has many problems and it is fundamentally similar to ours (yes, they don't look at the kd ratio at all, only your score and win/loses)
The REAL difference is that in NS2 we have the whole community in 1 server. From the top premier division players to the new rookies. Now you explain me how to balance this! The awesome part is that we can have fun playing together most of the time, because this community is more civil than others.
In my experience, most of the time people try hard to balance teams (I don't know in your region, but that is true where I play). But it is not easy, not even when you know all the players. So how do you expect the system to be so perfect?
Like every other game. But maybe you could explain me exactly how are they "punished", besides going into a team with more chances to win.
Your problem is that you look at your "skill level" and you think that this number is your value as a person or gamer. It is not.
Stop looking at the number and play the greatest game of all times
There are several reasons why K/D as a skill indicator is stupid, here are a few:
1) It's super easy to go on a rookie server and just frag-fest the shit out of it till it empties.
2) Even against fair teams, hell even against stacked teams, you can easily get high K/D's by going a higher lifeform. In contrast, it takes considerably more effort to get as high K/D as a marine. Even if you have early leads. (Getting 50/0 as alien is relatively common, but you rarely see a marine with that type of K/D - and if you do, it's cus he's stomping rookies)
In other words, players who only play aliens will probably have their skill level super-inflated compared to players who only go marine or players who play both.
3) With that same line of reasoning, I also predict that people will be less inclined to play the squishy alien characters and go straight for fade - and perhaps even play super conservative with their skulks.
The current system encourages what ever wins the game. That includes sacrificial plays. I think it's a shame to throw that away, and I also think it's unnecessary.
It's not accurately calculating anything that could count towards skill, it's but a phony system designed to be worshiped by public players who don't know any better & go with the flow. Arguably, you could say it IS working as intended to force random equal skill teams & cause a headache.
Pathetic.
Even if it is as perfect as it can realistically be, it still won't be enough to satisfy you guys. It still won't create even games every time, or even most of the time. This is the type of game where even if teams are perfectly even in terms of player skill, there are still countless things that can still turn the game into a complete stomp - especially due to the snowball effect. Hell, to exaggerate to make a point, someone could scratch their arse at the wrong time in the early game, and it can have a lasting impact on the rest of the game.
Not only that, but skill approximaters require a huge wealth of data to ever come close to being accurate - data that we don't have and never will have while the playerbase is this small. So perhaps you want moultano to wave a magic wand and suddenly get tens of thousands of people playing ns2? Perhaps solve all conflict in the middle east too? Eliminate global hunger? Cure cancer?
Lower your goddamn expectations.
You're now educated on actuality
Welcome.
I keep going on every thread with valid arguments and reasoning, never saying the system is perfect, but understanding that the fundamental idea behind it is valid as long as the implementation is done right (which I think it is, and I've made a lot of arguments why I think that).
You (and many others) keep going on every thread basically just saying "the system sucks and everyone believing in it is STOOOPIIIID, but I don't have to explain why because every one can see it".
Inspiring.
Yet AGAIN ITT: People dont know how the skill system works, dont offer plausable fixes to any existant problem, accuse people who know what they're talking about of conspiracy.
In the last thread there were several issues with the skill system pointed out, discussed, and reasonable solutions/ courses of action to take were outlined. None of this productive discussion occurred between the kind of idiots that are making posts like
"This skill system is redundant. You're now educated on actuality
Welcome"
If it comes from anywhere, the death of the community will come about from this sort of incessant nagging over something you dont understand and have no idea how to 'fix' (and ignorance of the people who are actually making progress).
Refer to other thread for >9k posts of nay-sayers getting schooled on the skill system by people that are actually able to contribute an educated opinion.
Zealots must shut down all debate because only their opinion is allowed.
Who made them arbiter of what is allowed to be discussed, and what is a valid argument, and what constitutes a rational argument. Their own giant egos, of course.
And if skill isn't quantifying ability to carry games, then what is it lol. You're not going to get better than W/L without having thousands of concurrent players first
If I have one wish it would be that people would actually research and think about the current system before criticizing it. (like some actually did in previous posts in other threads)
I do not indicate that the system its perfect in any way BUT:
If you read moultano's originally design paper (https://docs.google.com/document/d/1KMVLpPvEwFtoieTsTqeD8xBheFFEuK4PqjiyzGYSWbU/edit?pli=1#heading=h.n7i2r11tuaxi) and look at how it was implemented by the hive system you notice the following details:
Most issues coming up in-game are actually caused by the implementation ( only about 40% was implemented and some mistakes were made)!
If you then research about rating systems in games these days overall you will notice that the one proposed for ns2 is by design ahead the normally used "advanced ELO-rating based" solutions.
One of the major reason it seems to work better in other games is that they relay more on their league systems than on their skill rating.
In the end even the design might be not perfect and some might even bring up the point that statistical math is not suited for "gamer performance" ratings (but a similar debate can be found if you look a bit into sport science in that matter).
So please for the sake of a constructive debate about this matter, think and research about it before you post.
So far I'll give you an example of a design aspects that needs to be improved imho:
Currently the design suggests to only separate commanders and player's skill waging the players performance in each team via the global team w/l rate (btw here a "example implementation issue": it just assumes a 50/50 w/l ratio
But this doesn't cover up that ns2 is a very asymptotic game, so it is very much possible that a player performs pretty different with each team and is far outside of the global balance ratio.
Therefor separating the players rating into two with one rating for each team might work out better. But i know that moultano is also aware of this since some time
Some units are saying it's working fine, some others don't.
But WHO has a simulator able to show the evolution of a given player base ?
The best output would be like a single image showing the difference between the SP number (evolution over time) against the number it should be (the real skillzzzz). A sort of sound wave graph (like the one you get in VLC or Winamp...).
We must assume there is a target number for each player. Whatever one can imagine that lies behind (aim, positioning blah blah). It is the number that the system should give him on average.
Say every player has a horizontal line of pixel on altitude Y (abscissa).
Each time a game is played a new pixel is added on the X axis.
Say Green is the closest and red is far away concerning his SP compared to what it should be. Meaning a player has 1500 and he should be 1500 the pixel is green (0.255.0). Otherwise the more it's going away, the more it goes red.
For simplicity sake we assume players are all playing at the same time. But we can add that.
How to make it happen:
Need the players:
A single table should do it.
table "players"
PlayerID,
Player_SP,
Player_real_SP (the target),
Player_join_instantly
Then we can get it even more complex on the next stage.
localization, (say 3 continents)
Starting_Game, (so we can pour in the population some rookies)
Ending_Game, (rookies got off)
Say we fill the table with some random values with a caped variation. Say somebody who's real SP is 1000 we can give between 500 and 1500. Of course anything can happen : 0 3000, but this one isn't the common dude right ? It is allowed to change the min and max limits later.
Getting closer to the real world
table "server"
Server_id
Server_localization (to fit the continent list)
Getting these lemmings to play
Simple:
We assume that servers are filled with 18 players and so we do 18 players packets for each server. Good thing we have player_ids so we can make a random distribution on servers. Then we activate FET, and we get for all player a server, team, and chance to win.
We must assume that there is a % chance of winning against the odds by luck or an event that is turning upside down the game (ex: tunnel). This would be good to be able to change it at each simulation (not every game) from 0 to 50% chance.
Complex:
Distribution considers some people have already joined a team and only assign those who are not yet in a team (player_join_instantly). What FET does actually.
Distribution considers localization. People are more likely to play on server that are actually closer to them. We can think of a "reunion" for later (the fun part).
Say players play four games in row on the same server. It would need counters and such, but nothing impossible.
A 'Game' table filled and emptied at each turn.
Game on server (0->x)
18 columns (9 marine-9 alien)
W/L ratio.
Lemmings have played a game
Every game produce an outcome. It is determined by the real skill on each team and the % chance of special event (roll the dice). The algorithm add/sub the appropriate value to the skill score and save it to each Player_SP. Of course the real skill don't move.
Display of data
Difference between player_SP & player_real_SP (the target). Say red is 500 pt less or greater than the real skill. So it's red when someone has 1000 instead of 1500 or 2000 instead of 1500. A single table for the gradient will do.
Setting a number of games and population events:
As the system need many games to be played it is encouraged to simulate many. We obtain a lengthy graph. Be sure to make configuration adjustment on your servers.
In the mean time, being closer to reality means to introduce events like a rookie wave. So the Starting_Game, Ending_Game will be used.
What would it help to see ? :
-How is behaving the whole system depending on different elements / events / populations.
-What would be the magnitude of a 'ripple effect'. That would be REALLY interesting.
-Does it converge to the right value accurately and safely (3 continents, Comp/pub etc...).
-A green screen if everything is ok. A not so green screen depending on event
-What events are crippling the numbers...
It can be done with PHP+GD/xxxSQL (surprisingly). Anything that can use a DB actually. I would have started with that if i were to propose a system...
Anyway, a simulation to show the naysayers that such a system works would be pretty simple to build. Create a simple victory condition and steps towards that condition (the game), a few entities (players) with an assigned probability of succeeding in those steps (skill) and then just simulate 10 million 5vs5 games and let the skill values converge according to a similar model as in NS2. Then you can just pick any 10 players, assign them in two teams in any order, and the skill system will predict the outcome of that match, and it will be so.
For example, let's consider the game to be such that for a team to win, they have to succeed in killing all the opposing players. Each player has a randomly determined chance to kill an enemy, ranging from 1% to 99%. Each game round, all players will randomly choose an opposing player and attempt to kill it, and their chance of succeeding depends on their skill. If after a round, all the players on one team are dead, the other team is declared a winner, and if both teams are dead, the game is a draw. Create 1000 player entities with randomly assigned skill values and let the computer simulate 10 million games with randomly assigned teams, and I can assure you that the player entities will have skill ratings directly proportionate to their actual chances of fulfilling game objectives (skill). You could then pick any collection of 10 players, assign them to the teams any way you want, and the skill system would give you a prediction of what the outcome will be. If the skill system says "90% chance of team 1 winning", then I can assure you that if you ran the simulation 100 times with those players, team 1 would win approximately 90 of those games.
You would be correct to argue that NS2 is too complicated for this to work perfectly, but there are a lot of people denying the very fundamental idea of the skill system with "wins don't measure skill because you can stack" without understanding what the skill system is about. Concentrate on the nuances and details that actually need tweaking, but this rampant nonsense about the very fundamental idea behind the skill system being bullpoo is nothing but a knee-jerk reaction, a product of not having researched the topic enough.
You are right that players with K/D of 1.5 or greater and a W/L under 1 are rare. In fact they are only about 1.5% of players.
To have either a W/L or K/D at or above 1.5 isn't very common itself. About 12.3% and 11.4% respectively. How does that effect your argument?
P.S. For me 'force even teams' works fine.
for that it is useful because without it there is no real way to gauge if the match will be competitive without knowing all those involved.
ideally your skill level should be calculated per round based upon the average win/loss ratio, kill/death ratio, hours played and assists per minute of all players then placing you somewhere in that. if yours are higher then your skill level would be higher etc... But it would require a lot of work
In @moultano I trust.
I don't agree that is the ideal. Because, as has been mentioned before, there are certain factors that cannot be quantified on principle. I.e. leadership.
I want to point out something, perhaps a little bit off topic.
Chess computers nowadays are practically unbeatable. There has been quite a few scandals involving chess grandmasters being banned for cheating with Iphones in tournaments.
On lichess.org, a relatively new open-sourced chess website, you can even have your chess games analyzed for you (for free), and it will let you know exactly when and what mistakes you've made in text.
So chess computers could technically just count the mistakes a player makes in a game, and rank him based on that. The technology is surely there? Yet, all the chess websites choose to rank you based on an ELO system, not unlike what moultano and co have made for us in NS2.
but in chess u only have to monitor win loss ratio since it is 2 player. But in NS2 you have to monitor several things... certainly stuff like leadership etc are impossible to really monitor but currently w/l isnt enough because those who play even teams are likely to lose 50% of the games they play and yet completely destroy balance in a typical public game. In that isntance k/d ratio is a better measure. Players with 4.0 and over k/d have the habit of completely dominating a game and often those players have a humble 1500skill due to playing regularly in even matches.
You don't need to go all the way down to the game player by player. Only the outcome. Win/loose. As the system only consider this value. It allows more rendering/calculations per seconds.
The real point is; i would never implement a system without proper testing. In this case a simulator would do it. So the real question is "is the simulator exists". A program that can simulate and render results in many different scenarios (from simple to complex). Frankly if it's not already existing i can only assume that it is unprofessional to say the least. Worse if i am including "common sense".
Aye... wrong example.
The Chess ELO system has suffered many critics during a certain period. Some players did train in a specific opening (the Indian if my memory serves me well). They challenged other players that were less prepared for this and they could go up easily until everybody had the counter and learn it. But in the mean time the "indianers" did stay at the same level.
The top players did had a hard time as they had to analyze something new for them. Memory helps a lot in chess and that's why many champions are trained a the age of 6 (you'll find many studies about that). They didn't loose instantly (surprise effect) but the clock was the problem for them. Analyzing takes time. When uncovering this, the critics started.
To summarize; the critics were: Who is the best player ? The one who has a great memory, or the one that can analyze all situations ? Considering game has a limited time... The fire started as easy as a match can light a barrel of gasoline.
The computer algorithms (not the one on chess2000 (haha)) that runs on computer are able to analyze the strength of a move efficiently since Kasparof vs Deeper blue. Even though DB did had a bug and did a "destabilizing move". Regardless of any opening or way of playing the thing is working quite well since 20 years.
To get the idea
Deeper blue : 11.38 Gflops at the time
Core I7 3770k : 94.41 Gflops...
Soon you'll have it in your pocket.
You guys do realize the current NS2 skill system is very close to what the most popular game in the world uses (League of Legends) for a ranked system. You gain/lose League Points based off win/loss only and the amount is proportional to the skill ranking of other players in the game. The reason LoL's system is more reliable is matchmaking is required and people can't just teamswitch if they don't think their team will win. There is also a much bigger player pool so you don't end up with a player with 5000+ hours in the game playing with/against someone with 10 hours and the system trying to figure that out. Truly trying to balance a game like NS2 with an automated system at this point is an impossible task, and that not even mentioning the fact the game is freaking assymetrical. That's a whole extra wrench thrown into the mix.
All this balancing by K/D talk just made me think about some toxic jungler in LoL raging "God damnit I went 8/0/4 but my team sucks" yet he is still in bronze because he doesn't actually know how to play the game and help the team win. If you have ever posted a screenshot bragging about your pub score on these forums, this is you.