And since you seem to be fairly technically minded, the major challenge we face with LuaJIT is the Lua->C++ API. For LuaJIT to be performant, you have to architect that API in some pretty specific ways, otherwise you will always break the JIT compiles and get almost no benefit from it. Completely changing the API is not an attractive option (rewrite a lot of code..break all the mods..), so we're currently investigating ways of avoiding this.
The overall truth is that there hasn't been experiments on a stripped down codebase to see what would be possible in Spark with a vastly simplified game logic AFAIK, really that's something I would have expected to see from the combat team - they could remove massive amounts of game logic. I don't see there being notable improvements in frame rates unless you massively simply the game logic at this point, baring underlying engine changes that speed up the LUA/Renderer. I think they have optimized their current logic close to as far as possible.
Go and play the Fighter mod from the Mod Jam. In normal NS2 I get ~40 average fps and in the fighter mod I get 150 fps.
That's about the most damning comment you can possibly make about performance.
I didn't know minstrel was on teh forums
hi minstrel
On that note, I heard about the luaJIT too, and how it's a really fast scripting what not (sorry I don't know much about this)
But what about the openGL/DX11 things, what exactly does this do? (besides openGL possibly making it playable or mac or linux or something?) and dx11? No complaints as I'm running a card with dx11 though
edit: Although I too am not happiest with performance I still find it very playable in low playercounts and especially on the NSL maps, and I really hope it improves in the future because this is one of my favorite and only played games at the moment and also one of my favorite development teams
But what about the openGL/DX11 things, what exactly does this do? (besides openGL possibly making it playable or mac or linux or something?) and dx11? No complaints as I'm running a card with dx11 though
OpenGL and D3D are the lower level drawing APIs you use to program the graphics hardware. Stuff like loading textures into graphics memory, manipulating the drawing buffers, loading shaders and drawing the triangles. They are fairly similar in concept but with different function names and syntax.
There shouldn't be any difference in visuals going from DX9 to DX11 as all the art assets and shaders should be the same, unless UWE get a fat cheque from Microsoft and add DoF, motion blur and lens flare and put a DX11 sticker on the box.
But what about the openGL/DX11 things, what exactly does this do? (besides openGL possibly making it playable or mac or linux or something?) and dx11? No complaints as I'm running a card with dx11 though
OpenGL and D3D are the lower level drawing APIs you use to program the graphics hardware. Stuff like loading textures into graphics memory, manipulating the drawing buffers, loading shaders and drawing the triangles. They are fairly similar in concept but with different function names and syntax.
There shouldn't be any difference in visuals going from DX9 to DX11 as all the art assets and shaders should be the same, unless UWE get a fat cheque from Microsoft and add DoF, motion blur and lens flare and put a DX11 sticker on the box.
I don't care about visuals, and from what I've heard there is no developer who has properly applied direct x 10 into their games, let alone direct x 11 (visual wise of what they're capable of)
However if there's a performance benefit, then that sounds nice. I play any type of competitive fps game on lowest settings possible so textures are simple and not distracting, the only thing you want to see is your opponent and nothing else; plus you want performance
from what I've heard there is no developer who has properly applied direct x 10 into their games, let alone direct x 11
Are you currently living under a rock?
DirectX11 provides many neat new things like better multi-threading, tessellation and GPGPU...
Hell, check out this list of games with DirectX 11 support.
If they had licensed Valve's Source engine (maybe at a friendship price for an upcoming Indie developer) I bet there would already be a Linux version. Would not have minded that the tighter corset of the Source engine meant there would be fewer innovative features and harder modding. The artwork quality of Unknown Worlds is still extraordinarily and would definitely shine bright on Source 1, too. All I want is to play this game on mid-end hardware without choppy performance.
Does anyone know how I can return games on Steam and get a refund? I could than re-buy Natural-Selection 2 once they fixed their engine.
For anyone curious (and ScardyBob) i made a chart with some basic frame time data. I got this data from a listen server with some bots while i just ran around a bit, I am hoping to get some 'real' data later today from pubs. http://www.mediafire.com/download/x02f414sw7n55ji/NS2FrameTimes.xlsx is the spreadsheet.
But what about the openGL/DX11 things, what exactly does this do? (besides openGL possibly making it playable or mac or linux or something?) and dx11? No complaints as I'm running a card with dx11 though
The primary reason this is being done right now is to be able to port the game over to other platforms, like MAC and Linux. However, we are also hoping there will be some additional performance boost on the GPU side as well, though exactly how much is unknown at the moment. Between this and the LuaJITstuff that is being worked on, which will help out on the CPU side, there should be noticeable performance improvements coming.
But what about the openGL/DX11 things, what exactly does this do? (besides openGL possibly making it playable or mac or linux or something?) and dx11? No complaints as I'm running a card with dx11 though
The primary reason this is being done right now is to be able to port the game over to other platforms, like MAC and Linux. However, we are also hoping there will be some additional performance boost on the GPU side as well, though exactly how much is unknown at the moment. Between this and the LuaJITstuff that is being worked on, which will help out on the CPU side, there should be noticeable performance improvements coming.
This whole post sounds fantastic, hope it works out
And since you seem to be fairly technically minded, the major challenge we face with LuaJIT is the Lua->C++ API. For LuaJIT to be performant, you have to architect that API in some pretty specific ways, otherwise you will always break the JIT compiles and get almost no benefit from it. Completely changing the API is not an attractive option (rewrite a lot of code..break all the mods..), so we're currently investigating ways of avoiding this.
Tell us more! Reading about things like this gives me a "break out the profiler" itch. It's exciting. Is there a performance sensitive subset you could rewrite, and assume the remainder will fall back to being interpreted?
One challenge is even identifying the hotspots, so Dushan has been working on a sample-based Lua profiler to help with this. Think of it as Intel VTune for Lua scripts (cuz otherwise, VTune just tells us...Lua is slow). Once that is done, we can better assess if it's possible to address just that subset.
Does anyone know how I can return games on Steam and get a refund?
According to the Google search I just did, Valve's customer service manager is Helen Waite. So if you want a Steam refund you can go to Helen Waite for it.
Steve, Cory - as long as you're answering performance and technology questions:
Are you doing any work around splitting up the game logic code so that it could be handled in multiple lua threads? I know that's not easy (and I know that's an understatement) but it seems like the payoff could be huge.
matsoMaster of PatchesJoin Date: 2002-11-05Member: 7000Members, Forum Moderators, NS2 Developer, Constellation, NS2 Playtester, Squad Five Blue, Squad Five Silver, Squad Five Gold, Reinforced - Shadow, NS2 Community Developer
Steve, Cory - as long as you're answering performance and technology questions:
Are you doing any work around splitting up the game logic code so that it could be handled in multiple lua threads? I know that's not easy (and I know that's an understatement) but it seems like the payoff could be huge.
Right now the client uses slightly more than 2 cores at peak load, so further MT would not benefit 2-core machines much. Better to go for LuaJIT, which uses the CPU more efficiently.
I have one simple thing you guys could add that is not major but does help in late game performance... file appending. I know this crazy new age technology only recently developed by scientists sounds risky but it can and does effect performance. In addition the lack of file appending also hurts mods in that for example an admin mod can not have a single log file dedicated to each admin as over time the file would kill the server due to lack of file appending.
To test how the lack of file appending effects NS2 vs a game like cs:s I had a defrag operation run to simulate higher I/O load on the HDD. The cs:s server actually writes much more data to disk due to constant demo recording along with normal txt logging however it was totally unaffected by the increased I/O load. NS2 on the other had turned into a slideshow until logging was disabled at which time the game then ran as smoothly as it should. Given that we know that it was the logging in ns2 causing the lag and that there was no lag in cs:s from logging under the same conditions we can logically assume that the lack of file appending is the cause for this lag.
Steve, Cory - as long as you're answering performance and technology questions:
Are you doing any work around splitting up the game logic code so that it could be handled in multiple lua threads? I know that's not easy (and I know that's an understatement) but it seems like the payoff could be huge.
We do this for prediction already, which resulted in huge gains, but that's literally running two separate Lua worlds. Doing it within the same world is more complex, and much more risky (in terms of bugs, code maintainability, whether or not it would even work, etc. etc.). So, it's less likely we'll pursue this avenue.
from what I've heard there is no developer who has properly applied direct x 10 into their games, let alone direct x 11
Are you currently living under a rock?
DirectX11 provides many neat new things like better multi-threading, tessellation and GPGPU...
Hell, check out this list of games with DirectX 11 support.
I don't think you properly read my post, I said visually no game has done what direct x 10 or 11 is capable of doing, all I've seen is some fancy lighting with the suns rays and "tessellation" can you name a game that fully utilizes everything dx10 has to offer? Or dx11?
Not talking at all about games which "use" them or have one feature and try and market it as a selling point "tressFX, look at laras hair its so real!!1!"
Steve, Cory - as long as you're answering performance and technology questions:
Are you doing any work around splitting up the game logic code so that it could be handled in multiple lua threads? I know that's not easy (and I know that's an understatement) but it seems like the payoff could be huge.
Right now the client uses slightly more than 2 cores at peak load, so further MT would not benefit 2-core machines much. Better to go for LuaJIT, which uses the CPU more efficiently.
Majority of users are on direct x 11 capable cards, and majority have 4 cores or more (add up 4 cores + other) it's extremely close to 2 core users though, but will continue to trend only upward
Huh Ezekiel?
Majority has less than 4 cores you mean? It's 47% 2 cores, 4% 1core.
+2% every 4 months by the graph so the majority will be on 4 cores in December/january!
UWE it's ok you got time for the performance boost :>
Huh Ezekiel?
Majority has less than 4 cores you mean? It's 47% 2 cores, 4% 1core.
+2% every 4 months by the graph so the majority will be on 4 cores in December/january!
UWE it's ok you got time for the performance boost :>
Huh Ezekiel?
Majority has less than 4 cores you mean? It's 47% 2 cores, 4% 1core.
+2% every 4 months by the graph so the majority will be on 4 cores in December/january!
UWE it's ok you got time for the performance boost :>
matsoMaster of PatchesJoin Date: 2002-11-05Member: 7000Members, Forum Moderators, NS2 Developer, Constellation, NS2 Playtester, Squad Five Blue, Squad Five Silver, Squad Five Gold, Reinforced - Shadow, NS2 Community Developer
Steve, Cory - as long as you're answering performance and technology questions:
Are you doing any work around splitting up the game logic code so that it could be handled in multiple lua threads? I know that's not easy (and I know that's an understatement) but it seems like the payoff could be huge.
We do this for prediction already, which resulted in huge gains, but that's literally running two separate Lua worlds. Doing it within the same world is more complex, and much more risky (in terms of bugs, code maintainability, whether or not it would even work, etc. etc.). So, it's less likely we'll pursue this avenue.
The extra Lua VM running the Predict world is very much a special case; it it responsible for taking the server network updates and merge it with any missing client moves, then deliver that to the main thread. It reduced an O(fps^2) problem in the main thread to an O(network rate) in a secondary thread - a very good thing.
Also increased fps so much that Max had to add move rate limiters to avoid servers imploding under the load ... oh, fun times, good times :-)
Lua does not support true multithreading (yet), so if you want to split the logic work you would probably go for using multiple Lua VMs running in their own threads, and then have a merge phase in between... doable, but adding lots of complexity.
So going for LuaJIT first is the right way ... hopefully, it will be fast enough to tide Spark over until Lua supports true multithreading.
One challenge is even identifying the hotspots, so Dushan has been working on a sample-based Lua profiler to help with this. Think of it as Intel VTune for Lua scripts (cuz otherwise, VTune just tells us...Lua is slow). Once that is done, we can better assess if it's possible to address just that subset.
Wow, I can't live without my trusty pprof. It wouldn't surprise me if there were an obvious 20% gain that fell out of the first look at the graph.
ScatterJoin Date: 2012-09-02Member: 157341Members, Squad Five Blue
Is LuaJIT something that must be implemented game wide or can it be used in certain critical pathways? I have a basic coding understanding but not enough to really know this off hand.
It reduced an O(fps^2) problem in the main thread to an O(network rate) in a secondary thread
Oooh you sexy thing.
Lua does not support true multithreading (yet), so if you want to split the logic work you would probably go for using multiple Lua VMs running in their own threads, and then have a merge phase in between...
Yeah, that's what I was imagining.
doable, but adding lots of complexity.
I was imagining that, too.
So going for LuaJIT first is the right way ... hopefully, it will be fast enough to tide Spark over until Lua supports true multithreading.
Would in-VM multithreading buy you that much vs. using multiple-VMs? Isn't the hard part making the game logic multithreadable in the first place, i.e. separating out the parts that can run asynchronously from each other?
My apologies if I'm asking absurd questions here. I know just a little bit about these topics and nothing at all about your code.
Is LuaJIT something that must be implemented game wide or can it be used in certain critical pathways? I have a basic coding understanding but not enough to really know this off hand.
I think luaJIT splits the workload, so one part of the game goes through luaJIT, then another part goes through regular lua.. so making the work more evenly split on the cores already supported
or I'm an idiot and you can just disregard what I said because it's a complete guess
Comments
So we're basically boned for a really long time?
Go and play the Fighter mod from the Mod Jam. In normal NS2 I get ~40 average fps and in the fighter mod I get 150 fps.
I didn't know minstrel was on teh forums
hi minstrel
On that note, I heard about the luaJIT too, and how it's a really fast scripting what not (sorry I don't know much about this)
But what about the openGL/DX11 things, what exactly does this do? (besides openGL possibly making it playable or mac or linux or something?) and dx11? No complaints as I'm running a card with dx11 though
edit: Although I too am not happiest with performance I still find it very playable in low playercounts and especially on the NSL maps, and I really hope it improves in the future because this is one of my favorite and only played games at the moment and also one of my favorite development teams
OpenGL and D3D are the lower level drawing APIs you use to program the graphics hardware. Stuff like loading textures into graphics memory, manipulating the drawing buffers, loading shaders and drawing the triangles. They are fairly similar in concept but with different function names and syntax.
There shouldn't be any difference in visuals going from DX9 to DX11 as all the art assets and shaders should be the same, unless UWE get a fat cheque from Microsoft and add DoF, motion blur and lens flare and put a DX11 sticker on the box.
I don't care about visuals, and from what I've heard there is no developer who has properly applied direct x 10 into their games, let alone direct x 11 (visual wise of what they're capable of)
However if there's a performance benefit, then that sounds nice. I play any type of competitive fps game on lowest settings possible so textures are simple and not distracting, the only thing you want to see is your opponent and nothing else; plus you want performance
Are you currently living under a rock?
DirectX11 provides many neat new things like better multi-threading, tessellation and GPGPU...
Hell, check out this list of games with DirectX 11 support.
Does anyone know how I can return games on Steam and get a refund? I could than re-buy Natural-Selection 2 once they fixed their engine.
The primary reason this is being done right now is to be able to port the game over to other platforms, like MAC and Linux. However, we are also hoping there will be some additional performance boost on the GPU side as well, though exactly how much is unknown at the moment. Between this and the LuaJITstuff that is being worked on, which will help out on the CPU side, there should be noticeable performance improvements coming.
This whole post sounds fantastic, hope it works out
One challenge is even identifying the hotspots, so Dushan has been working on a sample-based Lua profiler to help with this. Think of it as Intel VTune for Lua scripts (cuz otherwise, VTune just tells us...Lua is slow). Once that is done, we can better assess if it's possible to address just that subset.
Are you doing any work around splitting up the game logic code so that it could be handled in multiple lua threads? I know that's not easy (and I know that's an understatement) but it seems like the payoff could be huge.
Right now the client uses slightly more than 2 cores at peak load, so further MT would not benefit 2-core machines much. Better to go for LuaJIT, which uses the CPU more efficiently.
Thanks for the reply.
To test how the lack of file appending effects NS2 vs a game like cs:s I had a defrag operation run to simulate higher I/O load on the HDD. The cs:s server actually writes much more data to disk due to constant demo recording along with normal txt logging however it was totally unaffected by the increased I/O load. NS2 on the other had turned into a slideshow until logging was disabled at which time the game then ran as smoothly as it should. Given that we know that it was the logging in ns2 causing the lag and that there was no lag in cs:s from logging under the same conditions we can logically assume that the lack of file appending is the cause for this lag.
http://store.steampowered.com/hwsurvey/cpus/
We do this for prediction already, which resulted in huge gains, but that's literally running two separate Lua worlds. Doing it within the same world is more complex, and much more risky (in terms of bugs, code maintainability, whether or not it would even work, etc. etc.). So, it's less likely we'll pursue this avenue.
I don't think you properly read my post, I said visually no game has done what direct x 10 or 11 is capable of doing, all I've seen is some fancy lighting with the suns rays and "tessellation" can you name a game that fully utilizes everything dx10 has to offer? Or dx11?
Not talking at all about games which "use" them or have one feature and try and market it as a selling point "tressFX, look at laras hair its so real!!1!"
According to steamstats hardware server http://store.steampowered.com/hwsurvey
Majority of users are on direct x 11 capable cards, and majority have 4 cores or more (add up 4 cores + other) it's extremely close to 2 core users though, but will continue to trend only upward
Majority has less than 4 cores you mean? It's 47% 2 cores, 4% 1core.
+2% every 4 months by the graph so the majority will be on 4 cores in December/january!
UWE it's ok you got time for the performance boost :>
The extra Lua VM running the Predict world is very much a special case; it it responsible for taking the server network updates and merge it with any missing client moves, then deliver that to the main thread. It reduced an O(fps^2) problem in the main thread to an O(network rate) in a secondary thread - a very good thing.
Also increased fps so much that Max had to add move rate limiters to avoid servers imploding under the load ... oh, fun times, good times :-)
(http://forums.unknownworlds.com/discussion/122189/224-tech-changes-part-1 for those interested in more details )
Lua does not support true multithreading (yet), so if you want to split the logic work you would probably go for using multiple Lua VMs running in their own threads, and then have a merge phase in between... doable, but adding lots of complexity.
So going for LuaJIT first is the right way ... hopefully, it will be fast enough to tide Spark over until Lua supports true multithreading.
Wow, I can't live without my trusty pprof. It wouldn't surprise me if there were an obvious 20% gain that fell out of the first look at the graph.
Yeah, that's what I was imagining.
I was imagining that, too.
Would in-VM multithreading buy you that much vs. using multiple-VMs? Isn't the hard part making the game logic multithreadable in the first place, i.e. separating out the parts that can run asynchronously from each other?
My apologies if I'm asking absurd questions here. I know just a little bit about these topics and nothing at all about your code.
I think luaJIT splits the workload, so one part of the game goes through luaJIT, then another part goes through regular lua.. so making the work more evenly split on the cores already supported
or I'm an idiot and you can just disregard what I said because it's a complete guess