Jiriki's suggestions sounds best, however I'm against any visible statistics. Match making for public shouldnt really be the first priority but some kind of easy pcw (scrim) search that is available for everyone should be. I think it would be fine even if it is just a link to a website where you can leave your time/zone/skill and whether if you have server or not.
<!--quoteo(post=1760311:date=Mar 20 2010, 07:40 PM:name=R_e_n_e_g_a_d_e)--><div class='quotetop'>QUOTE (R_e_n_e_g_a_d_e @ Mar 20 2010, 07:40 PM) <a href="index.php?act=findpost&pid=1760311"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->LOL, Microsoft is never the answer to anything gaming.<!--QuoteEnd--></div><!--QuoteEEnd--> Who cares? This is a mathematical method and its completely irrelevant who made the calculation method.
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->I'm well versed in Bayesian networks and if you think for a minute this won't be abused within an inch of farcicality you're deceiving yourself. Abuse of Bayesian networks is as obvious as people who abuse relevancy measures when they mark an article as irrelevant simply because it did not solve their problem. Thus it's measure of relevancy is severely skewed by an incompetent userbase.<!--QuoteEnd--></div><!--QuoteEEnd--> Skewed by incompetent userbase, wtf? Having a better player on your team increases team's probability of winning (otherwise he wouldn't be a better player) which can show up on statistics. What's the problem here?
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->Statistics are not the answer, lobbies are not the answer, anything that can be easily abused is not the answer. Random assignment is the only unabusable answer.,<!--QuoteEnd--></div><!--QuoteEEnd--> What's wrong with lobbies? Go play some random games in any RTS game where there's no skill balancing and then tell me that mathematics cannot decrease the probability of facing an unequal opponent.
<!--quoteo(post=1760443:date=Mar 21 2010, 10:27 AM:name=Jiriki)--><div class='quotetop'>QUOTE (Jiriki @ Mar 21 2010, 10:27 AM) <a href="index.php?act=findpost&pid=1760443"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Skewed by incompetent userbase, wtf? Having a better player on your team increases team's probability of winning (otherwise he wouldn't be a better player) which can show up on statistics. What's the problem here?<!--QuoteEnd--></div><!--QuoteEEnd-->A naive assumption since there are many alternative ways to increases the probability of winning (which is a debatable measure of skill in the first place). Is it necessary to formulate for you a list for you, or will <a href="http://overlordror.gamerlimit.com/files/2009/09/idle.jpg" target="_blank">a visual one suffice</a>? For anyway you can think of to quantize skill, there is a way it will be severely abused, and although it may work in the general case, it is well-known that only a small amount of griefing/abuse is required to ruin an entire experience. Don't waste devs' time & effort implementing a global skill rating system then engage them in guerrilla warfare trying to hotfix and blacklist abuses. We don't need that for them or NS2.
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->What's wrong with lobbies? Go play some random games in any RTS game where there's no skill balancing and then tell me that mathematics cannot decrease the probability of facing an unequal opponent.<!--QuoteEnd--></div><!--QuoteEEnd--> I've done both, Starcraft and L4D. Players labeling their game with "pros only, or noobs" etc works a hell of a lot better than skill-matching in which I've yet to find a decent match. In fact, self-labeling is the only form of "skill rating" I would suggest; a server/community can mark their server as designated for a certain skill level to which players decide if they are a part of. It's simple, intuitive, and doesn't waste development resources.
Self labeling games is as exploitable as anything else. I believe some sort of "Public rating" could work but only if it is invisible and effects to raise unknown. But really shouldnt be made priority not really that necessery I think servers will get adjusted when casual admins ban anyone decent and certain servers attract more competitive players.
Well one NS1 server I played on used KPD-based team balancing, and apart from throwing me to aliens 10 times in a row it worked quite well even thouh KPD is very exploitable. That was after they had only random enforcement. Even if there're some "exploits" (you go ahead make a list), it doesn't matter if only minority will use it.
My point is that if in NS2 we'll have have to see half the servers using "rules against stacking", I'd say we have a flaw in the game. Stacked teams are actually a big problem in NS1. If people have to develop special plugins to deal with that, its even worse.
About self-labelling, while it may work to some extent, nothing stops a bunch of high-skilled players stacking on a beginner's server and killing everyone's gaming experience. That is something I'd like to address in NS2.
In any case, I don't think they should develop anything like this until final release.
I should probably correct my earlier post: exploitation may not be a problem <i>if and only if</i> skill rating is used for match-making only (in which case there's not much incentive for misrepresent one's self); however as you point out there are enough times when it 'glitches' that most seasoned players will opt for a serverbrowser and a community any day.
Stacks are easily "smashed" (wee dev pun) by a simple random all + no team hopping; if NS2 employs this very simply solution I guarantee skill stacking will be a thing of the past. As for the "abuse" of a self-labeling system, the same point applies to skill-rating abuse: players who want to steamroll nubs will always find a way however you have to trust that there is very little incentive for a skilled player to <i>want</i> to do this. Granted, with NS2 becoming more "accessible" we will see an increase in the incidence of griefing; however I believe at its core, NS attracts skilled players who, more than anything else, want to play with other skilled players. A labeling system gives this power to the player and the community in an open and frank way, without worrying about hidden formulas.
<!--quoteo(post=1760596:date=Mar 22 2010, 04:32 AM:name=TrC)--><div class='quotetop'>QUOTE (TrC @ Mar 22 2010, 04:32 AM) <a href="index.php?act=findpost&pid=1760596"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Self labeling games is as exploitable as anything else. I believe some sort of "Public rating" could work but only if it is invisible and effects to raise unknown. But really shouldnt be made priority not really that necessery I think servers will get adjusted when casual admins ban anyone decent and certain servers attract more competitive players.<!--QuoteEnd--></div><!--QuoteEEnd-->
This is was LoL does to do matchmaking using a modified ELO system. It also takes into account stuff like groups joining as a team and how that typically gives a boost to the overall team's skill rating. Work pretty well.
Basically, a persistent stat would be a good thing for those (perhaps a minority) who want automated balanced games, but keep it hidden so it's harder to exploit (unlike say HoN). And yes, this would require, as R_e_n_e_g_a_d_e points out, locking the teams so no hopping, no players join/leave, etc. otherwise most stat system crap out.
In general a standard server browser system for the generic pubs, and clan scrims should suffice. This whole stat system and autobalance is for the non-clan more skilled players.
Yeah well let me re-iterate my point. I think there definitely should be a server browser in NS2 like TF2 etc. I think servers could use of some global stats tracking as an extra-feature to help out balance teams. I don't think it should be enforced. If there are servers with and without random enforcement, the ones without may well be more popular because people like to pick their team, albeit at the cost of more or less ocassionally stacked teams. Even though it may be a trade-off some servers accept, I dont think it'll be that good for NS2 in the long run, because its kind of a situation that's in NS1. With some statistical help, you could provide some ability to pick a team when you are not outbalancing the situation and provide an option between full random enforcement and freely-chosen teams.
However L4D kind of lobby would be awesome addition. They could use (like we do), votable captains who pick teams which will balance teams rather well. So first 12 (ie.) people will join, vote for captains, and captains will pick teams and they'll be like 2 rounds or whatever is set or agreed on. No need for random teams or whatever.
Actually quantifying skill is an interesting mathematical problem.
<!--quoteo(post=1761048:date=Mar 24 2010, 03:01 PM:name=spellman23)--><div class='quotetop'>QUOTE (spellman23 @ Mar 24 2010, 03:01 PM) <a href="index.php?act=findpost&pid=1761048"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Not really. Win:Loss ratio is a pretty good indicator, even if it takes 20+ games to converge to something reasonable.
Granted, most systems get destroyed once you try to incorporate teamwork elements. One loser bringin' down the whole team.<!--QuoteEnd--></div><!--QuoteEEnd-->
I like how you say "not really" then contradict that in the next sentence.
Win:Loss ratio is a <i>horrible</i> way to judge skill in a teambased game.
I meant Win-Loss using something like a ELO system. Boil it all down to one variable, did what you do allow your team to win?
Then again, there's plenty of theory for trying to quantify skill using other means. K:D, "points", rating by team, etc. Most don't work, at least well.
<!--quoteo(post=1761048:date=Mar 24 2010, 03:01 PM:name=spellman23)--><div class='quotetop'>QUOTE (spellman23 @ Mar 24 2010, 03:01 PM) <a href="index.php?act=findpost&pid=1761048"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec--><!--quoteo(post=1760726:date=Mar 22 2010, 09:21 PM:name=Jiriki)--><div class='quotetop'>QUOTE (Jiriki @ Mar 22 2010, 09:21 PM) <a href="index.php?act=findpost&pid=1760726"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec--> Actually quantifying skill is an interesting mathematical problem.<!--QuoteEnd--></div><!--QuoteEEnd-->
Not really. Win:Loss ratio is a pretty good indicator, even if it takes 20+ games to converge to something reasonable.
Granted, most systems get destroyed once you try to incorporate teamwork elements. One loser bringin' down the whole team. <!--QuoteEnd--></div><!--QuoteEEnd-->
<!--quoteo(post=1761096:date=Mar 24 2010, 09:38 PM:name=spellman23)--><div class='quotetop'>QUOTE (spellman23 @ Mar 24 2010, 09:38 PM) <a href="index.php?act=findpost&pid=1761096"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->I meant Win-Loss using something like a ELO system. Boil it all down to one variable, did what you do allow your team to win?
Then again, there's plenty of theory for trying to quantify skill using other means. K:D, "points", rating by team, etc. Most don't work, at least well.<!--QuoteEnd--></div><!--QuoteEEnd-->
Thous kpd'ish statistics are often used now... For example there is DOTA like game - Heroes of Newerth where thous statistics are used to make teams more even(auto-balance) if the host wants to use that feature...
Win:loss is Terrible. Look at the NCAA brackets (or any other sports schedule) there are always teams that are good but have terrible W:L records because they payed a tough schedule. Vice versa, there are teams with great records that have terrible skill.
Same is true of K:D ratios.
If you want to do something like, I'd recommend assigning every new player a base rating (say 1000). Then for every event (structure distroyed, kill, etc) that furthers the team goals, all players on the event team are awarded points based on the relative value of the event (IE: more for killing a hive than an OC. Adjust for the ratio of points earned. So you earn points slower if your team is slaughtering the competition.) At the end of the match take the ratio of points earned and apply the ratio of the average of the team rating. Deduct the final from the losing teams score and add it to the winner.
That way L33t players are penalized if they don't protect thier mates. and teams can't spawn camp for more rating.
Weighing wins or losses depending on how long they took is pretty pointless. The simple fact of the matter is that if you're better than the average player on the team you increase it's chances of winning, and the other way around. W:L is a pretty good determiner of whether you should move up or not. Actually it's the only one. Over time any discrepancies will even out.
The only 'but' here is that people will have to be able to clear their stats and start over since if we use the system suggested by jiriki, the more matches we have played the harder it will be to move up or down in ranking. Either individual resets or global resets to accomodate for longer term trends.
<!--quoteo(post=1761649:date=Mar 29 2010, 10:11 AM:name=tjosan)--><div class='quotetop'>QUOTE (tjosan @ Mar 29 2010, 10:11 AM) <a href="index.php?act=findpost&pid=1761649"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Weighing wins or losses depending on how long they took is pretty pointless. The simple fact of the matter is that if you're better than the average player on the team you increase it's chances of winning, and the other way around. W:L is a pretty good determiner of whether you should move up or not. Actually it's the only one. Over time any discrepancies will even out.<!--QuoteEnd--></div><!--QuoteEEnd-->
Precisely. However, the time it takes to "even out" is... well... atrocious. And even longer the larger the team sizes.
There have been plenty of alternative systems (such as points for each thing done i.e. kill, buildings built, etc. or even an idea where your teammates give out points or demerits based on your performance) that, in theory, would converge faster, but are much more easily exploited than pure K:D.
Assuming random assignment and match-ups within a reasonable range, pure K:D is the only true measure. Too bad other stupid variables get in the way (I has team mates that I alway play with on my team and thus should get skill rating boosts) and it takes really long to converge properly.
There was a similar debate on the ENSL forums about using win/loss for determining player skill, and I'm convinced that if properly done, it can work. If you have constant teams or ones with little variation, it obviously isn't going to work, because what you end up collecting is the skill of the teams, not the individual. But if teams are getting mixed, the only constant is the player being tracked, so in theory, given enough sampling, the result should be a pretty reliable measurement.
Of course, one could game the system by deliberately joining the team that is more likely to win (You see this all the time already, without global stat tracking); as long as there is player choice on what team to join, I think this is always going be a possible way to exploit the ranking system. Also, if one player continually picks the same role, the ranking system is only really going to capture how well they play that role (This may or may not be a issue at all though).
The problem with that is players who play on servers where their skill is more able to turn the tide have a greater impact than those who play on servers where a single player isn't likely to dictate the outcome.
Again, to any skill rating system I say "No no no." No to hidden formulas. No to exploitation and abuse. No to stat-hoarding. Simply label the server for a certain level and let the community decide openly.
All these simple systems like K/D and basic W:L are from the stone age compared Trueskill or similar sophisticated probability mathematics. Its like discussing whether we should have 3D graphics in NS2 or not.
The point of Trueskill or ELO is not rank it up the more you win, but calculate the how "surprising" the result is. Think about standard ELO, it has result subtracted from the expected value. If you stack teams, this team is more likely going to win anyway, and winning the game won't change your personal ranking at all.
There's one major flaw in this system. Normally people cannot adjust their skill level too much, sure you can play drunk with music on from time to time but that's not a big issue for the system as a whole. Bigger issue is that in NS there're different roles. You can be a terrible fade but a very good lerk etc. What kind of role distribution happens on public NS, is more of an empirical question but its not always random, meaning you can play either bad or good roles more than others. You would actually have to measure that to give a meaningful answer here.
If you want some serious fallbacks in W/L -system, you might wanna check the <a href="http://en.wikipedia.org/wiki/Elo_rating_system" target="_blank">Wiki page of ELO</a>. Some of these apply to Trueskill aswell and some (like deflation) are not much of an issue if the system is aimed at public games.
Renegade, if you are referring to team sizes, this is not an issue really as long as NS is atleast nearly balanced between different team sizes. Besides, why are you sure fanatic against this? If you make this an option for server providers, it would allow server <i>community</i> to <i>decide</i> whether they want to use that data to balance teams or not, that's how you get <i>the best from both worlds</i>. Now in NS1 there was not <i>any of these</i> methods available, instead you got lots of servers with stacked teams (annoying manual rule enforcement to "fix" this issue), or few servers with random teams (guess which ones are more popular), and like 1 server with stats-tracking to balance teams. How is that some kind of global stats-tracking feature going to make make this situation any worse?
Besides Lobby-system, with votable captains would be the ultimate team balancer solution.
I'm still in the "server self labeling, players filter servers by label" camp. Every other method just seems so complex and time consuming. Plus it may place added pressure on the players to not try out new strategies since they may lose points or it may make the game stressful. Every system can be exploited and in such a team based game it is really difficult to accurately gauge the skill of a single person.
I'd rather have server admins choose what type of skill level they want on their servers then have the players join whichever they want. The most obvious case of exploitation is a highly skilled player continually joining newbie geared servers. At that point, I'd leave it up to the players/admin to kick the individual if they're ruining the game. The above exploitation aside, I still think this is the best method and I hope to see something close to this in NS2.
<!--quoteo(post=1761816:date=Mar 30 2010, 12:24 PM:name=Jiriki)--><div class='quotetop'>QUOTE (Jiriki @ Mar 30 2010, 12:24 PM) <a href="index.php?act=findpost&pid=1761816"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Renegade, if you are referring to team sizes, this is not an issue really as long as NS is atleast nearly balanced between different team sizes. Besides, why are you sure fanatic against this?<!--QuoteEnd--></div><!--QuoteEEnd--> No, not team sizes, I mean the difference a skilled player makes differs; think of it as a second derivative of "ELO" - the difference a single skilled player makes on some random pub is going to be much more than his or her difference on a server where vets regularly play. Not to mention as you say the problems of different roles. I'm not so much a "fanatic" against it - if someone wants to implement this it's fine by me as long as I retain the option to use a server browser; however implementing a global rating system is no light work and being global implies that it must likely be done by UWE. The return (a sometime-accurate, sometime-exploitable match-making system only useful for the gaming inexperienced) is nowhere near worthwhile in comparison to the alternatives.
<!--quoteo(post=1762034:date=Apr 1 2010, 11:45 AM:name=R_e_n_e_g_a_d_e)--><div class='quotetop'>QUOTE (R_e_n_e_g_a_d_e @ Apr 1 2010, 11:45 AM) <a href="index.php?act=findpost&pid=1762034"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->No, not team sizes, I mean the difference a skilled player makes differs; think of it as a second derivative of "ELO" - the difference a single skilled player makes on some random pub is going to be much more than his or her difference on a server where vets regularly play.<!--QuoteEnd--></div><!--QuoteEEnd--> This not a valid point. If hes a just a random pub player, hes not going to make much of a difference even on this random pub server. If he however has some skill, its going to affect the expected outcome anyway and him/her turning the tide is expected with a specific probability. If you have a veteran server where teams are balanced, the same player may not affect the outcome, but this is expected because the calculated total probability of team winning is 50% etc. I'm sorry, but it seems you misunderstand the system.
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->Not to mention as you say the problems of different roles. I'm not so much a "fanatic" against it - if someone wants to implement this it's fine by me as long as I retain the option to use a server browser; however implementing a global rating system is no light work and being global implies that it must likely be done by UWE. The return (a sometime-accurate, sometime-exploitable match-making system only useful for the gaming inexperienced) is nowhere near worthwhile in comparison to the alternatives.<!--QuoteEnd--></div><!--QuoteEEnd--> I think a server browser is a must, and what superior "alternatives" are you talking about ?
I wasn't even implying this would be necessary for match making, votable captains are a lot better way assuming people happen to know each other. This however could help both match-making and public servers (where you would join with a server browser) to balance teams as an extra feature.
Sure its up to devs whether using resouces on this is worth it, but in my opinion stacking was and is a serious problem in NS1, and I wouldn't want to see the same problem in NS2.
In case people missed it, I advocate an ELO system (I thought that was obvious).
Pure W:L and K:D are indeed stupid and have no bearing on "skill". I thought we were talking about using Wins and losses in the scope of an ELO system.... sorry about the confusion.
And yeah, the point I was trying to make is that in the long run a ELO system works. However, in a team system it's really hard to combine individual ELO ratings and get a good guess on the team's ability to "win". Especially if one is a new account (low # of games) or 1 really, really bad player bringing the team down (see LoL and HoN). So, yeah, in that sense it's still a fun little (and ugly) math problem.
<!--quoteo(post=1762098:date=Apr 1 2010, 10:14 AM:name=Jiriki)--><div class='quotetop'>QUOTE (Jiriki @ Apr 1 2010, 10:14 AM) <a href="index.php?act=findpost&pid=1762098"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->I'm sorry, but it seems you misunderstand the system.<!--QuoteEnd--></div><!--QuoteEEnd--> I understand it all too well, and its pitfalls: Noobs who frequent Server A and vets who frequent Server B will be rated equally by such a system. If players from each server should happen to cross paths you'll have a disparate situation that contradicts what the stats say. This may be necessary for proper convergence, but what has your system solved if several match-making errors need to occur before this convergence happens? It's no better or worse than the "trial and error" process a new player experiences upon trying different servers anyways; all that's been introduced is a layer of complexity.
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->I think a server browser is a must, and what superior "alternatives" are you talking about ?<!--QuoteEnd--></div><!--QuoteEEnd--> again, self-labeling. Simple implementation, intuitive, works.
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->in my opinion stacking was and is a serious problem in NS1, and I wouldn't want to see the same problem in NS2.<!--QuoteEnd--></div><!--QuoteEEnd--> Agreed, but you've confused the solution: you don't need a "system" to prevent skill-stacking; all that's needed is a simple random-all. If you can explain how this will fail to defeat skill-stacking or can be exploited (unlike your system which can be exploited and abused), I'm all ears.
Ok, that's a valid point but it sounded like you were saying that a player is ranked differently because he has a worse team. Players who don't overlap (in chess ELO the problem is called selective pairing) cause problems because there're not enough samples to give valid data.
Like I said, stats-tracking would be provide an <i>alternative</i> between random-all and freely-choosen teams. Even if a traditional vet (which I hope you are not referring to the actual NS vets program) player joins a low-skilled server, a stats-tracking system with good convergence properties will adjust to his skill quite fast if he starts playing there, and if he doesn't, those few games ain't a problem. The other vet players may be ranked equally with the low-skilled players until there's enough overlap but it doesn't matter because these results don't have to be public. Most servers have a standard playerbase, who, with given some time will provide more than enough samples to accurately quantify his/her skill (and its distribution).
For example, I play pick-up games within our competitive community. We've got lots of regulars, so number of samples isn't a problem. Random non-regulars have little impact on the teams. The problem is balancing the teams with the regulars, because there's a lot of spread in the skill. This is exactly where a skill-tracking systems come in. However we actually don't use such because I have solved the issue with votable captains (less coding) which works fine, <b>even</b> though there're some exploits (people voting bad captains on purpose).
Look, I played on a server where there was first not any kind of team enforcement. Instead admins just banned people who they think caused bad teams. There're were lots of contronversial bans especially because some of there were done by face value than anything else, and I had to join aliens many times just because others were stacking (and even then you could get banned if you counter-stacked to balance teams out with friends). Well later they used random-enforcement, but changed that to stats-tracking system because people just didn't <i>want</i> random teams.
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->again, self-labeling. Simple implementation, intuitive, works.<!--QuoteEnd--></div><!--QuoteEEnd--> Self-labelling may help differently skilled players find servers, but stacking is a problem that happens on all servers because competitive players can be so skilled that even a few high-skilled players stacking can destroy the gaming experience of the rest, on a "vet" server. I can <b>tell you</b> that there would be a lot of stacking in our competitive pick-up games if there were no captains or anything to balance out teams. Labelling our gathers competitive wouldn't solve anything.
It should be noted that there are two distinct problems we're discussing: 1) intra-server skill-disparity (team stacking) 2) inter-server skill-disparity (noob vs. vet servers)
Random-all addresses #1 and self-labeling speaks to #2. Taken together they are a comprehensive solution. Likewise, a good match-making system will also attempt to deal with both; however a good MM system cannot prevent skill-stacking unless it manages teams. Thus the only disadvantage leveled against random-all (not being able to choose teams) is invalid since it is also inherent in a good match-making system.
Secondly, the only time a statistical system works is: a) on a large scale, globally, under a high number of heterogeneous samples; meaning you need a large userbase that does not exhibit "clustering" - i.e. the large majority of players are freelancers who frequent many different servers. b) on a small scale, per-server, in which the majority of players are "clusters" of regulars - i.e. very few servers jump servers.
For any situation in-between, you have a problem: stats are built-up in "clusters" formed around servers and their regulars. Intermittent freelancers who visit different servers will help to converge stats from each server; however in order for this to occur you will need either many freelancers or a long period of time. Neither of which is feasible for our community. We exist in such a scenario between a and b (or at least in our heyday we had equal parts regular and freelancer) , which means newcomers with no regular community will experience a wildly fluctuating variation in skill as they're matched amongst different servers. By contrast, regulars neither want nor need a match-making system (they use the server browser to find their communities). Despite our wildest hopes, I'm confident we will never see the numbers for convergence to be sufficiently accurate, nor is waiting for it to happen a viable option for a match-making system that needs to work on release. Thus a match-making system, I believe, is not suited for the NS community and only hurts the newcomers it seeks to help.
I don't know if you're advocating some kind of forced random, but if you are, that's stupid. Friends want to play together and team stacking happens. For anyone really concerned about team stacking I would just reply with "deal with it" as forcing friends to split up isn't what a game should do.
No I am not, but your idea of "just dealing with it" is quite the stupid you seem to be talking about, to say the least, primitive. Flick: random-all on Flick: random-all off
You want to play with your friend? Go find some willing server with team-stacking enabled. But I assure you the vast majority of the pub demographic is not comprised of clingy butt-buddies.
<!--quoteo(post=1762600:date=Apr 3 2010, 11:20 PM:name=R_e_n_e_g_a_d_e)--><div class='quotetop'>QUOTE (R_e_n_e_g_a_d_e @ Apr 3 2010, 11:20 PM) <a href="index.php?act=findpost&pid=1762600"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->No I am not, but your idea of "just dealing with it" is quite the stupid you seem to be talking about, to say the least, primitive. Flick: random-all on Flick: random-all off
You want to play with your friend? Go find some willing server with team-stacking enabled. But I assure you the vast majority of the pub demographic is not comprised of clingy butt-buddies.<!--QuoteEnd--></div><!--QuoteEEnd-->
People playing with friends really <i>isn't</i> stupid in anyway and it's half the reason why MMO's are so popular. Communities in FPS games are born and people stack the teams all the time to play with people they want. Sometimes this unbalances the skill levels, but like I said, deal with it.
Random all servers can exist and I'd have no problem with them, but your post implied forced random to be on at all times.
Comments
Who cares? This is a mathematical method and its completely irrelevant who made the calculation method.
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->I'm well versed in Bayesian networks and if you think for a minute this won't be abused within an inch of farcicality you're deceiving yourself. Abuse of Bayesian networks is as obvious as people who abuse relevancy measures when they mark an article as irrelevant simply because it did not solve their problem. Thus it's measure of relevancy is severely skewed by an incompetent userbase.<!--QuoteEnd--></div><!--QuoteEEnd-->
Skewed by incompetent userbase, wtf? Having a better player on your team increases team's probability of winning (otherwise he wouldn't be a better player) which can show up on statistics. What's the problem here?
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->Statistics are not the answer, lobbies are not the answer, anything that can be easily abused is not the answer. Random assignment is the only unabusable answer.,<!--QuoteEnd--></div><!--QuoteEEnd-->
What's wrong with lobbies? Go play some random games in any RTS game where there's no skill balancing and then tell me that mathematics cannot decrease the probability of facing an unequal opponent.
Is it necessary to formulate for you a list for you, or will <a href="http://overlordror.gamerlimit.com/files/2009/09/idle.jpg" target="_blank">a visual one suffice</a>? For anyway you can think of to quantize skill, there is a way it will be severely abused, and although it may work in the general case, it is well-known that only a small amount of griefing/abuse is required to ruin an entire experience. Don't waste devs' time & effort implementing a global skill rating system then engage them in guerrilla warfare trying to hotfix and blacklist abuses. We don't need that for them or NS2.
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->What's wrong with lobbies? Go play some random games in any RTS game where there's no skill balancing and then tell me that mathematics cannot decrease the probability of facing an unequal opponent.<!--QuoteEnd--></div><!--QuoteEEnd-->
I've done both, Starcraft and L4D. Players labeling their game with "pros only, or noobs" etc works a hell of a lot better than skill-matching in which I've yet to find a decent match.
In fact, self-labeling is the only form of "skill rating" I would suggest; a server/community can mark their server as designated for a certain skill level to which players decide if they are a part of. It's simple, intuitive, and doesn't waste development resources.
My point is that if in NS2 we'll have have to see half the servers using "rules against stacking", I'd say we have a flaw in the game. Stacked teams are actually a big problem in NS1. If people have to develop special plugins to deal with that, its even worse.
About self-labelling, while it may work to some extent, nothing stops a bunch of high-skilled players stacking on a beginner's server and killing everyone's gaming experience. That is something I'd like to address in NS2.
In any case, I don't think they should develop anything like this until final release.
Stacks are easily "smashed" (wee dev pun) by a simple random all + no team hopping; if NS2 employs this very simply solution I guarantee skill stacking will be a thing of the past. As for the "abuse" of a self-labeling system, the same point applies to skill-rating abuse: players who want to steamroll nubs will always find a way however you have to trust that there is very little incentive for a skilled player to <i>want</i> to do this. Granted, with NS2 becoming more "accessible" we will see an increase in the incidence of griefing; however I believe at its core, NS attracts skilled players who, more than anything else, want to play with other skilled players. A labeling system gives this power to the player and the community in an open and frank way, without worrying about hidden formulas.
This is was LoL does to do matchmaking using a modified ELO system. It also takes into account stuff like groups joining as a team and how that typically gives a boost to the overall team's skill rating. Work pretty well.
Basically, a persistent stat would be a good thing for those (perhaps a minority) who want automated balanced games, but keep it hidden so it's harder to exploit (unlike say HoN). And yes, this would require, as R_e_n_e_g_a_d_e points out, locking the teams so no hopping, no players join/leave, etc. otherwise most stat system crap out.
In general a standard server browser system for the generic pubs, and clan scrims should suffice. This whole stat system and autobalance is for the non-clan more skilled players.
However L4D kind of lobby would be awesome addition. They could use (like we do), votable captains who pick teams which will balance teams rather well. So first 12 (ie.) people will join, vote for captains, and captains will pick teams and they'll be like 2 rounds or whatever is set or agreed on. No need for random teams or whatever.
Actually quantifying skill is an interesting mathematical problem.
Not really. Win:Loss ratio is a pretty good indicator, even if it takes 20+ games to converge to something reasonable.
Granted, most systems get destroyed once you try to incorporate teamwork elements. One loser bringin' down the whole team.
Granted, most systems get destroyed once you try to incorporate teamwork elements. One loser bringin' down the whole team.<!--QuoteEnd--></div><!--QuoteEEnd-->
I like how you say "not really" then contradict that in the next sentence.
Win:Loss ratio is a <i>horrible</i> way to judge skill in a teambased game.
Then again, there's plenty of theory for trying to quantify skill using other means. K:D, "points", rating by team, etc. Most don't work, at least well.
Actually quantifying skill is an interesting mathematical problem.<!--QuoteEnd--></div><!--QuoteEEnd-->
Not really. Win:Loss ratio is a pretty good indicator, even if it takes 20+ games to converge to something reasonable.
Granted, most systems get destroyed once you try to incorporate teamwork elements. One loser bringin' down the whole team.
<!--QuoteEnd--></div><!--QuoteEEnd-->
<!--quoteo(post=1761096:date=Mar 24 2010, 09:38 PM:name=spellman23)--><div class='quotetop'>QUOTE (spellman23 @ Mar 24 2010, 09:38 PM) <a href="index.php?act=findpost&pid=1761096"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->I meant Win-Loss using something like a ELO system. Boil it all down to one variable, did what you do allow your team to win?
Then again, there's plenty of theory for trying to quantify skill using other means. K:D, "points", rating by team, etc. Most don't work, at least well.<!--QuoteEnd--></div><!--QuoteEEnd-->
<img src="http://trollcats.com/wp-content/uploads/2009/10/son_i_am_disappoint_trollcat.jpg" border="0" class="linked-image" />
Thous kpd'ish statistics are often used now...
For example there is DOTA like game - Heroes of Newerth where thous statistics are used to make teams more even(auto-balance) if the host wants to use that feature...
Same is true of K:D ratios.
If you want to do something like, I'd recommend assigning every new player a base rating (say 1000). Then for every event (structure distroyed, kill, etc) that furthers the team goals, all players on the event team are awarded points based on the relative value of the event (IE: more for killing a hive than an OC. Adjust for the ratio of points earned. So you earn points slower if your team is slaughtering the competition.) At the end of the match take the ratio of points earned and apply the ratio of the average of the team rating. Deduct the final from the losing teams score and add it to the winner.
That way L33t players are penalized if they don't protect thier mates. and teams can't spawn camp for more rating.
The only 'but' here is that people will have to be able to clear their stats and start over since if we use the system suggested by jiriki, the more matches we have played the harder it will be to move up or down in ranking. Either individual resets or global resets to accomodate for longer term trends.
Precisely. However, the time it takes to "even out" is... well... atrocious. And even longer the larger the team sizes.
There have been plenty of alternative systems (such as points for each thing done i.e. kill, buildings built, etc. or even an idea where your teammates give out points or demerits based on your performance) that, in theory, would converge faster, but are much more easily exploited than pure K:D.
Assuming random assignment and match-ups within a reasonable range, pure K:D is the only true measure. Too bad other stupid variables get in the way (I has team mates that I alway play with on my team and thus should get skill rating boosts) and it takes really long to converge properly.
Of course, one could game the system by deliberately joining the team that is more likely to win (You see this all the time already, without global stat tracking); as long as there is player choice on what team to join, I think this is always going be a possible way to exploit the ranking system. Also, if one player continually picks the same role, the ranking system is only really going to capture how well they play that role (This may or may not be a issue at all though).
Again, to any skill rating system I say "No no no." No to hidden formulas. No to exploitation and abuse. No to stat-hoarding. Simply label the server for a certain level and let the community decide openly.
The point of Trueskill or ELO is not rank it up the more you win, but calculate the how "surprising" the result is. Think about standard ELO, it has result subtracted from the expected value. If you stack teams, this team is more likely going to win anyway, and winning the game won't change your personal ranking at all.
There's one major flaw in this system. Normally people cannot adjust their skill level too much, sure you can play drunk with music on from time to time but that's not a big issue for the system as a whole. Bigger issue is that in NS there're different roles. You can be a terrible fade but a very good lerk etc. What kind of role distribution happens on public NS, is more of an empirical question but its not always random, meaning you can play either bad or good roles more than others. You would actually have to measure that to give a meaningful answer here.
If you want some serious fallbacks in W/L -system, you might wanna check the <a href="http://en.wikipedia.org/wiki/Elo_rating_system" target="_blank">Wiki page of ELO</a>. Some of these apply to Trueskill aswell and some (like deflation) are not much of an issue if the system is aimed at public games.
Renegade, if you are referring to team sizes, this is not an issue really as long as NS is atleast nearly balanced between different team sizes. Besides, why are you sure fanatic against this? If you make this an option for server providers, it would allow server <i>community</i> to <i>decide</i> whether they want to use that data to balance teams or not, that's how you get <i>the best from both worlds</i>. Now in NS1 there was not <i>any of these</i> methods available, instead you got lots of servers with stacked teams (annoying manual rule enforcement to "fix" this issue), or few servers with random teams (guess which ones are more popular), and like 1 server with stats-tracking to balance teams. How is that some kind of global stats-tracking feature going to make make this situation any worse?
Besides Lobby-system, with votable captains would be the ultimate team balancer solution.
P.S. I wonder how we got here from Organized NS.
I'd rather have server admins choose what type of skill level they want on their servers then have the players join whichever they want. The most obvious case of exploitation is a highly skilled player continually joining newbie geared servers. At that point, I'd leave it up to the players/admin to kick the individual if they're ruining the game. The above exploitation aside, I still think this is the best method and I hope to see something close to this in NS2.
No, not team sizes, I mean the difference a skilled player makes differs; think of it as a second derivative of "ELO" - the difference a single skilled player makes on some random pub is going to be much more than his or her difference on a server where vets regularly play. Not to mention as you say the problems of different roles.
I'm not so much a "fanatic" against it - if someone wants to implement this it's fine by me as long as I retain the option to use a server browser; however implementing a global rating system is no light work and being global implies that it must likely be done by UWE. The return (a sometime-accurate, sometime-exploitable match-making system only useful for the gaming inexperienced) is nowhere near worthwhile in comparison to the alternatives.
This not a valid point. If hes a just a random pub player, hes not going to make much of a difference even on this random pub server. If he however has some skill, its going to affect the expected outcome anyway and him/her turning the tide is expected with a specific probability. If you have a veteran server where teams are balanced, the same player may not affect the outcome, but this is expected because the calculated total probability of team winning is 50% etc. I'm sorry, but it seems you misunderstand the system.
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->Not to mention as you say the problems of different roles. I'm not so much a "fanatic" against it - if someone wants to implement this it's fine by me as long as I retain the option to use a server browser; however implementing a global rating system is no light work and being global implies that it must likely be done by UWE. The return (a sometime-accurate, sometime-exploitable match-making system only useful for the gaming inexperienced) is nowhere near worthwhile in comparison to the alternatives.<!--QuoteEnd--></div><!--QuoteEEnd-->
I think a server browser is a must, and what superior "alternatives" are you talking about ?
I wasn't even implying this would be necessary for match making, votable captains are a lot better way assuming people happen to know each other. This however could help both match-making and public servers (where you would join with a server browser) to balance teams as an extra feature.
Sure its up to devs whether using resouces on this is worth it, but in my opinion stacking was and is a serious problem in NS1, and I wouldn't want to see the same problem in NS2.
Pure W:L and K:D are indeed stupid and have no bearing on "skill". I thought we were talking about using Wins and losses in the scope of an ELO system.... sorry about the confusion.
And yeah, the point I was trying to make is that in the long run a ELO system works. However, in a team system it's really hard to combine individual ELO ratings and get a good guess on the team's ability to "win". Especially if one is a new account (low # of games) or 1 really, really bad player bringing the team down (see LoL and HoN). So, yeah, in that sense it's still a fun little (and ugly) math problem.
I understand it all too well, and its pitfalls: Noobs who frequent Server A and vets who frequent Server B will be rated equally by such a system. If players from each server should happen to cross paths you'll have a disparate situation that contradicts what the stats say. This may be necessary for proper convergence, but what has your system solved if several match-making errors need to occur before this convergence happens? It's no better or worse than the "trial and error" process a new player experiences upon trying different servers anyways; all that's been introduced is a layer of complexity.
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->I think a server browser is a must, and what superior "alternatives" are you talking about ?<!--QuoteEnd--></div><!--QuoteEEnd-->
again, self-labeling. Simple implementation, intuitive, works.
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->in my opinion stacking was and is a serious problem in NS1, and I wouldn't want to see the same problem in NS2.<!--QuoteEnd--></div><!--QuoteEEnd-->
Agreed, but you've confused the solution: you don't need a "system" to prevent skill-stacking; all that's needed is a simple random-all. If you can explain how this will fail to defeat skill-stacking or can be exploited (unlike your system which can be exploited and abused), I'm all ears.
Like I said, stats-tracking would be provide an <i>alternative</i> between random-all and freely-choosen teams. Even if a traditional vet (which I hope you are not referring to the actual NS vets program) player joins a low-skilled server, a stats-tracking system with good convergence properties will adjust to his skill quite fast if he starts playing there, and if he doesn't, those few games ain't a problem. The other vet players may be ranked equally with the low-skilled players until there's enough overlap but it doesn't matter because these results don't have to be public. Most servers have a standard playerbase, who, with given some time will provide more than enough samples to accurately quantify his/her skill (and its distribution).
For example, I play pick-up games within our competitive community. We've got lots of regulars, so number of samples isn't a problem. Random non-regulars have little impact on the teams. The problem is balancing the teams with the regulars, because there's a lot of spread in the skill. This is exactly where a skill-tracking systems come in. However we actually don't use such because I have solved the issue with votable captains (less coding) which works fine, <b>even</b> though there're some exploits (people voting bad captains on purpose).
Look, I played on a server where there was first not any kind of team enforcement. Instead admins just banned people who they think caused bad teams. There're were lots of contronversial bans especially because some of there were done by face value than anything else, and I had to join aliens many times just because others were stacking (and even then you could get banned if you counter-stacked to balance teams out with friends). Well later they used random-enforcement, but changed that to stats-tracking system because people just didn't <i>want</i> random teams.
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->again, self-labeling. Simple implementation, intuitive, works.<!--QuoteEnd--></div><!--QuoteEEnd-->
Self-labelling may help differently skilled players find servers, but stacking is a problem that happens on all servers because competitive players can be so skilled that even a few high-skilled players stacking can destroy the gaming experience of the rest, on a "vet" server. I can <b>tell you</b> that there would be a lot of stacking in our competitive pick-up games if there were no captains or anything to balance out teams. Labelling our gathers competitive wouldn't solve anything.
1) intra-server skill-disparity (team stacking)
2) inter-server skill-disparity (noob vs. vet servers)
Random-all addresses #1 and self-labeling speaks to #2. Taken together they are a comprehensive solution. Likewise, a good match-making system will also attempt to deal with both; however a good MM system cannot prevent skill-stacking unless it manages teams. Thus the only disadvantage leveled against random-all (not being able to choose teams) is invalid since it is also inherent in a good match-making system.
Secondly, the only time a statistical system works is:
a) on a large scale, globally, under a high number of heterogeneous samples; meaning you need a large userbase that does not exhibit "clustering" - i.e. the large majority of players are freelancers who frequent many different servers.
b) on a small scale, per-server, in which the majority of players are "clusters" of regulars - i.e. very few servers jump servers.
For any situation in-between, you have a problem: stats are built-up in "clusters" formed around servers and their regulars. Intermittent freelancers who visit different servers will help to converge stats from each server; however in order for this to occur you will need either many freelancers or a long period of time. Neither of which is feasible for our community. We exist in such a scenario between a and b (or at least in our heyday we had equal parts regular and freelancer) , which means newcomers with no regular community will experience a wildly fluctuating variation in skill as they're matched amongst different servers. By contrast, regulars neither want nor need a match-making system (they use the server browser to find their communities). Despite our wildest hopes, I'm confident we will never see the numbers for convergence to be sufficiently accurate, nor is waiting for it to happen a viable option for a match-making system that needs to work on release. Thus a match-making system, I believe, is not suited for the NS community and only hurts the newcomers it seeks to help.
Flick: random-all on
Flick: random-all off
You want to play with your friend? Go find some willing server with team-stacking enabled. But I assure you the vast majority of the pub demographic is not comprised of clingy butt-buddies.
Flick: random-all on
Flick: random-all off
You want to play with your friend? Go find some willing server with team-stacking enabled. But I assure you the vast majority of the pub demographic is not comprised of clingy butt-buddies.<!--QuoteEnd--></div><!--QuoteEEnd-->
People playing with friends really <i>isn't</i> stupid in anyway and it's half the reason why MMO's are so popular. Communities in FPS games are born and people stack the teams all the time to play with people they want. Sometimes this unbalances the skill levels, but like I said, deal with it.
Random all servers can exist and I'd have no problem with them, but your post implied forced random to be on at all times.