NS2 and Monitoring Game-Statistics

Triggerman

<div class="IPBDescription">Getting unbiased information about the game</div><b>TF2 death heat-map:</b>
<a href="http://tf2wiki.net/wiki/File:Valve_stats_cp_dustbowl_deaths.jpg" target="_blank"><img src="http://tf2wiki.net/w/images/thumb/c/c6/Valve_stats_cp_dustbowl_deaths.jpg/200px-Valve_stats_cp_dustbowl_deaths.jpg" border="0" class="linked-image" /></a>

<b>TF2 average assists/hour:</b>
<a href="http://tf2wiki.net/wiki/File:Valve_stats_avgassists.png" target="_blank"><img src="http://tf2wiki.net/w/images/thumb/f/fa/Valve_stats_avgassists.png/220px-Valve_stats_avgassists.png" border="0" class="linked-image" /></a>
Source: <a href="http://tf2wiki.net/wiki/Team_Fortress_2_stats" target="_blank">tf2wiki.net</a>

One thing I think NS2 will definitively gain from implementing sooner rather than later, is a good monitoring system that can gather good <i>statistics</i>.


Since UWE is developing their own engine, implementing this would make level-building as well as balancing a bit more manageable, and if they so desire they could hopefully release that to the community as well that goes hand-in-hand with their approach to moddable gaming as a part of the Spark's selling-point.

Personally, I think this is one of the most important things a multiplayer-game should have, since really it's all about balance that is so hard to obtain would be a bit easier to achieve with anonymous monitoring of games.
To take it down a level to where the gamers rather than the developers could use this... perhaps a server that is trying out a map that is a work-in-progress could show up a little icon that tells us that the server is trying to gather statistics for the host, that then can be collected after the server is brought down in a neat way.
Levels could get so much better faster too, showing where Lerks shoot Spores the most, where Skulks tend to stand still the longest, where Marines die, the Powernodes that gets taken down the most, where Sentries are placed and so on.

What is your opinion?

Tig

<!--quoteo(post=1822066:date=Jan 6 2011, 04:36 PM:name=Triggerman)--><div class='quotetop'>QUOTE (Triggerman @ Jan 6 2011, 04:36 PM) <a href="index.php?act=findpost&pid=1822066"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec--><b>TF2 death heat-map:</b>
<a href="http://tf2wiki.net/wiki/File:Valve_stats_cp_dustbowl_deaths.jpg" target="_blank"><img src="http://tf2wiki.net/w/images/thumb/c/c6/Valve_stats_cp_dustbowl_deaths.jpg/200px-Valve_stats_cp_dustbowl_deaths.jpg" border="0" class="linked-image" /></a>

<b>TF2 average assists/hour:</b>
<a href="http://tf2wiki.net/wiki/File:Valve_stats_avgassists.png" target="_blank"><img src="http://tf2wiki.net/w/images/thumb/f/fa/Valve_stats_avgassists.png/220px-Valve_stats_avgassists.png" border="0" class="linked-image" /></a>
Source: <a href="http://tf2wiki.net/wiki/Team_Fortress_2_stats" target="_blank">tf2wiki.net</a>

One thing I think NS2 will definitively gain from implementing sooner rather than later, is a good monitoring system that can gather good <i>statistics</i>.


Since UWE is developing their own engine, implementing this would make level-building as well as balancing a bit more manageable, and if they so desire they could hopefully release that to the community as well that goes hand-in-hand with their approach to moddable gaming as a part of the Spark's selling-point.

Personally, I think this is one of the most important things a multiplayer-game should have, since really it's all about balance that is so hard to obtain would be a bit easier to achieve with anonymous monitoring of games.
To take it down a level to where the gamers rather than the developers could use this... perhaps a server that is trying out a map that is a work-in-progress could show up a little icon that tells us that the server is trying to gather statistics for the host, that then can be collected after the server is brought down in a neat way.
Levels could get so much better faster too, showing where Lerks shoot Spores the most, where Skulks tend to stand still the longest, where Marines die, the Powernodes that gets taken down the most, where Sentries are placed and so on.

What is your opinion?<!--QuoteEnd--></div><!--QuoteEEnd-->

trigger, aren't you on the team? can't you just go over to charlie's desk and be all like "yo, do this."

Silverwing

This would be an excellent feature. I approve, make it so.

McGlaspie

+1

NS2 could definitely benefit from having a statistics mechanism.


On that note, be sure and take a look at <a href="http://www.unknownworlds.com/ns2/forums/index.php?showtopic=111390" target="_blank">this thread</a>! It's a start at the very least.

Martin

*cough* We are working on this right now, we've been working quietly on this since November. I can give a data dump of what we have if anyone is interested. I can make a CSV.

PaiSand

Nice to hear about it!!!
Once you have it done, please post pics, or even better, post the link to see it personally.

McGlaspie

@Martin: Just out of curiosity, how much of a performance hit is it on the server? In other words, have you noticed an impact on the server's tick rate while gathering stats?

Martin

Does anyone have a higher res overview of rockdown

_Thresh_

Think the rockdown map would be all red....

Martin

I'm going to work on the heatmap tonight, should have something up shortly. Currently we have about 4k data points.

Triggerman

<!--quoteo(post=1822069:date=Jan 6 2011, 12:46 PM:name=Tig)--><div class='quotetop'>QUOTE (Tig @ Jan 6 2011, 12:46 PM) <a href="index.php?act=findpost&pid=1822069"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->trigger, aren't you on the team? can't you just go over to charlie's desk and be all like "yo, do this."<!--QuoteEnd--></div><!--QuoteEEnd-->

Hah, yeah I wish.
The closest thing I got to that was my suggestion about the launcher that I think/hope was picked up by them, that was added some time after with the Launch Pad.
No smacking on anyone's desk though.

<!--quoteo(post=1822153:date=Jan 6 2011, 04:40 PM:name=Martin)--><div class='quotetop'>QUOTE (Martin @ Jan 6 2011, 04:40 PM) <a href="index.php?act=findpost&pid=1822153"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->I'm going to work on the heatmap tonight, should have something up shortly. Currently we have about 4k data points.<!--QuoteEnd--></div><!--QuoteEEnd-->

Sweet, seems like it's doable without UWE having to get their hands dirty then, guess that's a relief for them.
Going to be fun to see what you have gathered ;)

McGlaspie

@Martin: This a big enough image of rockdown for you?

<a href="http://i557.photobucket.com/albums/ss18/McGlaspie/ns2_rockdown_overview.png" target="_blank">http://i557.photobucket.com/albums/ss18/Mc...wn_overview.png</a>

Camron

<!--quoteo(post=1822146:date=Jan 6 2011, 08:19 PM:name=Martin)--><div class='quotetop'>QUOTE (Martin @ Jan 6 2011, 08:19 PM) <a href="index.php?act=findpost&pid=1822146"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->*cough* We are working on this right now, we've been working quietly on this since November. I can give a data dump of what we have if anyone is interested. I can make a CSV.<!--QuoteEnd--></div><!--QuoteEEnd-->
Can you give any more details on which data you are logging? I recently started working on a logging/stats mod but I will probably abandon it now.

Martin

Here is part of a simple data extract of how many kills per weapon type have occurred:

<b>Marines</b>
1466 Marine
203 Rifle
196 Flamethrower
162 Shotgun
87 Axe
46 Grenade Launcher
46 Pistol

<b>Aliens</b>
1505 Skulk
478 Fade
317 Lerk
107 Whip
78 Hydra Spike
36 Gorge
31 Hydra

*Kills include both player and structure kills

Martin

<!--quoteo(post=1822165:date=Jan 6 2011, 05:58 PM:name=McGlaspie)--><div class='quotetop'>QUOTE (McGlaspie @ Jan 6 2011, 05:58 PM) <a href="index.php?act=findpost&pid=1822165"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->@Martin: This a big enough image of rockdown for you?

<a href="http://i557.photobucket.com/albums/ss18/McGlaspie/ns2_rockdown_overview.png" target="_blank">http://i557.photobucket.com/albums/ss18/Mc...wn_overview.png</a><!--QuoteEnd--></div><!--QuoteEEnd-->


Yep that should be fine

Martin

<!--quoteo(post=1822166:date=Jan 6 2011, 05:59 PM:name=Camron)--><div class='quotetop'>QUOTE (Camron @ Jan 6 2011, 05:59 PM) <a href="index.php?act=findpost&pid=1822166"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Can you give any more details on which data you are logging? I recently started working on a logging/stats mod but I will probably abandon it now.<!--QuoteEnd--></div><!--QuoteEEnd-->


Kill location, if the killer was a structure, if the victim was a structure, when the event occured, how the killer is, who the victim is, how many points were awarded, killer's weapon type.

That is just the stats, there are other things that our utility does besides stats.

Triggerman

Nice work Martin, knew there was something diabolical with your servers considering I always end up playing on them >_>

McGlaspie

@Martin: I don't know how you are actually recording the data (where in code the data is copied/stored), but you should take a look at the approach <a href="http://www.unknownworlds.com/ns2/forums/index.php?showuser=73982" target="_blank">player</a> took in <a href="http://www.unknownworlds.com/ns2/forums/index.php?showtopic=112026" target="_blank">this thread</a>. Using the same method, you could potentially follow the <a href="http://en.wikipedia.org/wiki/Dependency_injection" target="_blank">Dependency Injection</a> pattern and get all kinds of interesting stats without impacting the base code all that much. In fact, with some (very) careful planning, you might be able to abstract the statistics mod out so it wouldn't be severely affected by new updates to the base code. Granted, that would be the case once it's more feature complete, but still worth looking in to.

Martin

<!--quoteo(post=1822180:date=Jan 6 2011, 06:19 PM:name=McGlaspie)--><div class='quotetop'>QUOTE (McGlaspie @ Jan 6 2011, 06:19 PM) <a href="index.php?act=findpost&pid=1822180"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->@Martin: I don't know how you are actually recording the data (where in code the data is copied/stored), but you should take a look at the approach <a href="http://www.unknownworlds.com/ns2/forums/index.php?showuser=73982" target="_blank">player</a> took in <a href="http://www.unknownworlds.com/ns2/forums/index.php?showtopic=112026" target="_blank">this thread</a>. Using the same method, you could potentially follow the <a href="http://en.wikipedia.org/wiki/Dependency_injection" target="_blank">Dependency Injection</a> pattern and get all kinds of interesting stats without impacting the base code all that much. In fact, with some (very) careful planning, you might be able to abstract the statistics mod out so it wouldn't be severely affected by new updates to the base code. Granted, that would be the case once it's more feature complete, but still worth looking in to.<!--QuoteEnd--></div><!--QuoteEEnd-->


The way I've written it doesn't affect the base code very much. I didn't want to have to write an injection if I didn't have to. The main reason was I didn't want to put a DLL out there for people to put on their machine for security reasons. I am a fan of open source, so thus the entire mod is written in LUA minus the listener which is .net but I will soon be moving to python.

Also note that we were writing stats to our database back in August ;), but they weren't as concise.

Hugh

Ahhh I love this kind of stuff! Stats was the one subject I had to repeat at uni but I find myself using it everywhere. Martin, it's awesome that you've gone and done so much work on this. I'd love to crunch some raw data some time!

player

I've already had something in mind in EmbdStats along Triggerman's suggestion, as it is exceedingly simple to retrieve every player's position at any given moment when you are completely embedded in the Lua-VM (now you know where the mod's name came from).

<!--quoteo(post=1822180:date=Jan 7 2011, 03:19 AM:name=McGlaspie)--><div class='quotetop'>QUOTE (McGlaspie @ Jan 7 2011, 03:19 AM) <a href="index.php?act=findpost&pid=1822180"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->@Martin: I don't know how you are actually recording the data (where in code the data is copied/stored), but you should take a look at the approach <a href="http://www.unknownworlds.com/ns2/forums/index.php?showuser=73982" target="_blank">player</a> took in <a href="http://www.unknownworlds.com/ns2/forums/index.php?showtopic=112026" target="_blank">this thread</a>. Using the same method, you could potentially follow the <a href="http://en.wikipedia.org/wiki/Dependency_injection" target="_blank">Dependency Injection</a> pattern and get all kinds of interesting stats without impacting the base code all that much. In fact, with some (very) careful planning, you might be able to abstract the statistics mod out so it wouldn't be severely affected by new updates to the base code. Granted, that would be the case once it's more feature complete, but still worth looking in to.<!--QuoteEnd--></div><!--QuoteEEnd-->
I've already back-ported GmOvrmind's build-independency to EmbdStats and as long as no significant architectural changes to the existing functionality are made it will function just fine on into oblivion. So it is indeed quite simple when you're sufficiently familiar with NS2's Lua-script.

<!--quoteo(post=0:date=:name=Martin)--><div class='quotetop'>QUOTE (Martin)</div><div class='quotemain'><!--quotec-->The way I've written it doesn't affect the base code very much. I didn't want to have to write an injection if I didn't have to. The main reason was I didn't want to put a DLL out there for people to put on their machine for security reasons. I am a fan of open source, so thus the entire mod is written in LUA minus the listener which is .net but I will soon be moving to python.<!--QuoteEnd--></div><!--QuoteEEnd-->
A very good point indeed. As I am a C-programmer through and through, whenever I get the chance I do things in C. Although with pretty much every mod I create I have good reason to do so (Lua's extremely limited sandbox, as well as C's raw speed, which is important when you are operating in the server's process-loop).

Martin

Proof of concept.

Player Kills:

<a href="http://ns2stats.net/heatmap.php?q=0" target="_blank">http://ns2stats.net/heatmap.php?q=0</a>

Structure Kills:

<a href="http://ns2stats.net/heatmap.php?q=1" target="_blank">http://ns2stats.net/heatmap.php?q=1</a>

I'm still working on the scaling issues but this is just a rough to show you it's possible.

McGlaspie

I think you oops'ed Martin ;)

The first link the the same URL as the second.

Regardless, this is very cool stuff. Love it! Good work and I'm excited to see more in the future.

Martin

Yeah I corrected the links as soon as I saw that ha

Asraniel

interesting. but i suppose thats not the real data (proof of concept as you said). i mean, no kills in west ? ;)

Martin

I'm still trying to get the scaling right, some of the X and Y coordinates are out in the "void", off the map essentially. And that is live data.

McGlaspie

Actually, the part that's in the lower-left area (I think) is in the vent connecting the hallway to West Hive to the Alien Start. A skulk or lerk could have gone into the vent on fire and died there, or a grenade could have been chucked up there from alien start. I've got no explanation for the bits around Marine Start tech point (to left of it).

Btw, the link to the first heatmap is this: <a href="http://ns2stats.net/heatmap.php?q=0" target="_blank">http://ns2stats.net/heatmap.php?q=0</a>

Martin

That is the correct link.

I'm trying to figure out what the orientation is for the world actually. It's not X:0 Y:0 when it comes to putting the heatmap as an overlay. 0,0 is actually the top left corner whilst the world has to be centered somewhere, still trying to figure that one out ;) . I have it centralized but now am trying to figure out scaling of the overview and world units.

Wheeee

that data can't be right.. are you sure the x-y coordinates correspond to where they are on the map? because it seems to me like the big blob should be oriented vertically centered at marine start. usually that's where 90% of the action is.

SN.Wolf

Was thinking on this. Not sure if it has any thing to do with real world coordinates but Maybe since the minimap relies on the minimap_extents those may play a role in it?

anyways just in case they did here are the numbers for them and if i am correct it uses center bottom as "0"?


<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->rockdown = 1261.21; -740.6; 1033.29
tram =281.13; 56.27; 279.86
junction =147.85; 43.08; 137.48<!--QuoteEnd--></div><!--QuoteEEnd-->

ScardyBob

Awesome work!

Can you log any of the following?
- Structure info (type, location, death)
- Research info (which items, how long into the match they are researched)