This is why I really appreciate the developers of this game, they're being honest with us about everything (they admit when the game is unbalanced, and that performance blows, etc), and transparent about their inner workings. Indie studies ftw.
have a look the NS2HD article again <!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->The former is done constantly, through tweaks to the way the game thinks. For example, the original Hydra Lua code called on the Hydra to scan for targets several times a second, irrespective of whether there were any marines nearby. This caused immense lag in early NS2 builds (149, anyone?) because Hydras were constantly scanning for targets.<!--QuoteEnd--></div><!--QuoteEEnd-->
Buildings, there actions and the whole game logic for NS2 cost the most performance (all written in Lua)
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->Right now, Lua takes up around 50% of the CPU cycles that NS2 goes through to produce ever frame. This is, on its own, almost entirely responsible for the limited frame rates that people experience when playing in the NS2 Beta. Rendering in Spark is actually very fast. This is why turning down graphics options, or playing on a more powerful GPU, yields negligible performance benefits.<!--QuoteEnd--></div><!--QuoteEEnd-->
So if you would write a small CS like mod(without the complicated game mechanism) the game would run super smooth!
To use Lua as a scripting language way maybe the best decision UW did. Why? Well the dev. team of NS2 is small... really small compared to other game dev. teams where 50 to 100ppl work on one game over 2-3 years the goal they already reached in this "short" amount of time is amazing.
Lua is speeding up the process of development a lot! It make it easy for UW to try new things, play with values, without recompiling the whole game every single time(and even the community can help).
A lot of other games also use Lua as their script engines <a href="http://en.wikipedia.org/wiki/Category:Lua-scripted_video_games" target="_blank">http://en.wikipedia.org/wiki/Category:Lua-...ted_video_games</a>
To hardcode the whole game in C++ is not state of the art
The engine itself seems to be really well planned, new and efficient technologies are used.
Sure you guy expect a lag free game, but this is a BETA! I know most other beta games run great, but those betas are tested internal for months. UW doesn't have the capacity to do that.
If you develop software you first implement features and then you optimize them, that’s just how it works.
It will take couple of weeks, but NS2 is great and will be fast!
<!--quoteo(post=1925637:date=Apr 15 2012, 10:30 AM:name=F1ll3r)--><div class='quotetop'>QUOTE (F1ll3r @ Apr 15 2012, 10:30 AM) <a href="index.php?act=findpost&pid=1925637"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->A lot of other games also use Lua as their script engines <a href="http://en.wikipedia.org/wiki/Category:Lua-scripted_video_games" target="_blank">http://en.wikipedia.org/wiki/Category:Lua-...ted_video_games</a><!--QuoteEnd--></div><!--QuoteEEnd-->
I saw Operation Flashpoint in that list - I don't know how that game was, but it was definitely one of the best looking FPS games I had seen at the time. If that game ran well, then it's 100% possible for NS2 to run great.
<!--quoteo(post=1925561:date=Apr 15 2012, 09:19 AM:name=Yuuki)--><div class='quotetop'>QUOTE (Yuuki @ Apr 15 2012, 09:19 AM) <a href="index.php?act=findpost&pid=1925561"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->What about the dot functions (e.g. Alien.GetFrictionForce), are those written in lua ? I'm a bit confused because methods are normally called with colon in lua (Alien:GetFrictionForce).<!--QuoteEnd--></div><!--QuoteEEnd--> You can alternatively call methods by using a dot and simply passing 'self' as the first argument.
<!--quoteo(post=0:date=:name=w0dk4)--><div class='quotetop'>QUOTE (w0dk4)</div><div class='quotemain'><!--quotec-->So, at best, this will result in a 25% performance increase (btw. I thought this was already done). Of course this increases FPS and playability, but i.e. on the server side, at least a 100% perf. increase is needed (which is your goal for release apparently).
I hope you can pull it through but I also urge to think about more drastic actions. And the argument that being CPU bound is a good thing must be the joke of the week. CPU being the bottleneck is the worst thing that can happen to your game, since players can not "fix" it by lowering the graphics settings (especially important for competitive scene).<!--QuoteEnd--></div><!--QuoteEEnd--> The binding-layer was rewritten (back in B171), this would be the VM that translates bytecode into an actual instruction. I guess NS2HD's arithmetic is Lua being half of the frame-time, so a 50% increase * 0.5 = 25% FPS increase. For the server it would be more than half of the frame-time (maybe closer to 75%? who knows), so it'll be more than 25% there. And yes a CPU-bottlenecked game is pretty bad, because we're talking about a single-threaded bottleneck, and (aside overclocking severely) there's not much you can about it.
<!--quoteo(post=0:date=:name=rammaj)--><div class='quotetop'>QUOTE (rammaj)</div><div class='quotemain'><!--quotec-->On another note I am asking this because I have no background in programming but say if I wanted to create a game using this engine and LUA, and the game logic isnt as complex as the one in NS2 ( I am talking about the whole layer so including UI, how structures work, weapons etc ) lets say something like Counter Strike for argument's sake will the client/server run smoother if it has less lines of code giving that it is allso well optimized? and will it be a huge difference or very small and barely noticeable?<!--QuoteEnd--></div><!--QuoteEEnd--> It's true that NS2 is quite a bit more complex than say CS:S, so the things would not have been as problematic such a game. Lua can be used, but I guess not to such a degree as in NS2 (where indeed almost all of the game-logic is being handled by it). Take a look at how garry's mod does this, I believe he struck a good balance between moddability while retaining performance.
<!--QuoteBegin-'rammaj'+--><div class='quotetop'>QUOTE ('rammaj')</div><div class='quotemain'><!--QuoteEBegin-->if they decide to go with C++ may that be everything or just some parts would that mean they have to rewrite a certain part of the engine to call code from c++ instead of LUA? or can they just start doing it?<!--QuoteEnd--></div><!--QuoteEEnd--> Well certainly parts of the engine are geared to work specifically with that Lua-API, so some work would be in order there.
<!--quoteo(post=0:date=:name=F1ll3r)--><div class='quotetop'>QUOTE (F1ll3r)</div><div class='quotemain'><!--quotec-->To use Lua as a scripting language way maybe the best decision UW did. Why? Well the dev. team of NS2 is small... really small compared to other game dev. teams where 50 to 100ppl work on one game over 2-3 years the goal they already reached in this "short" amount of time is amazing.
Lua is speeding up the process of development a lot! It make it easy for UW to try new things, play with values, without recompiling the whole game every single time(and even the community can help).
A lot of other games also use Lua as their script engines <a href="http://en.wikipedia.org/wiki/Category:Lua-...ted_video_games" target="_blank">http://en.wikipedia.org/wiki/Category:Lua-...ted_video_games</a>
To hardcode the whole game in C++ is not state of the art
The engine itself seems to be really well planned, new and efficient technologies are used.
Sure you guy expect a lag free game, but this is a BETA! I know most other beta games run great, but those betas are tested internal for months. UW doesn't have the capacity to do that.
If you develop software you first implement features and then you optimize them, that’s just how it works.
It will take couple of weeks, but NS2 is great and will be fast!<!--QuoteEnd--></div><!--QuoteEEnd--> You shouldn't forget that the path of Lua has given Max (and assorted engine-programmers) a good amount of grief trying to cope with the performance-issues, so there's certainly a trade-off there. Anyway, if they can pull it off to yield good performance, then using Lua has been an excellent idea, but that is what is in question now. Your guess is as good as any at this point.
<!--quoteo(post=0:date=:name=Deadzone)--><div class='quotetop'>QUOTE (Deadzone)</div><div class='quotemain'><!--quotec-->I saw Operation Flashpoint in that list - I don't know how that game was, but it was definitely one of the best looking FPS games I had seen at the time. If that game ran well, then it's 100% possible for NS2 to run great.<!--QuoteEnd--></div><!--QuoteEEnd--> I do believe NS2 would be the first game to really use Lua to such an extend. I've peaked around at other games using Lua (FarCry, Garry's Mod), but they don't go as far in this as NS2 does (and as such, limiting moddability in that respect of course).
In case it seems like I'm coming across as overly critical: Join one of the top servers in NS2, punch in "net_stats" in the console, and wait until about 3/4 of the way into the game and watch the tickrate fall to nearly 10. Anything below 30 is unacceptable really (though right now you won't see it go above 30, due to it being capped there). Also remember that these (popular) servers are running on some serious hardware (between 4 and 5 GHz), which is far FAR higher than what public servers will be running when the game is released (and what all the vacant servers at the bottom of the server-list are running). Add to that that the amount of slots isn't as high as they have to be yet either (the largest is 20 I believe, while we should be aiming for 32). If you add all that together, it isn't odd to come to a number of 500-800% performance-increase required for the server at this point.
I think what he's getting at is that they can improve it without getting into a lot of GPU issues that differ much more from card to card (even manufacturer to manufacturer). When you write NS2 code that runs on the CPU, you're always writing for different flavours of x86, and you can ignore all the hardware features that aren't common to all your players.
As has been mentioned, the graphics engine in Spark is pretty robust, and performs/scales very well. A big push to optimize the GPU side of things would do very little for most players.
Hopefully at the end of things, the game will be GPU-bottlenecked for most players (after they all buy i5-2500ks to play it). When people say the game is CPU-bound, they mean that it is<i> right now </i>because the LUA integration work is unfinished. Not that it will be forever (but that's possible too).
<!--quoteo(post=1925641:date=Apr 15 2012, 11:35 AM:name=Deadzone)--><div class='quotetop'>QUOTE (Deadzone @ Apr 15 2012, 11:35 AM) <a href="index.php?act=findpost&pid=1925641"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->I saw Operation Flashpoint in that list - I don't know how that game was, but it was definitely one of the best looking FPS games I had seen at the time. If that game ran well, then it's 100% possible for NS2 to run great.<!--QuoteEnd--></div><!--QuoteEEnd-->
It probably didn't use LUA for the entirety of the game logic.
<!--quoteo(post=1925645:date=Apr 15 2012, 03:46 PM:name=player)--><div class='quotetop'>QUOTE (player @ Apr 15 2012, 03:46 PM) <a href="index.php?act=findpost&pid=1925645"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->You can alternatively call methods by using a dot and simply passing 'self' as the first argument.<!--QuoteEnd--></div><!--QuoteEEnd-->
Ho thanks, that's makes more sense. Another question, the lua hot-reload seems to be broken since a few builds, does that has something to do with optimization ?
<!--quoteo(post=1925657:date=Apr 15 2012, 12:10 PM:name=Yuuki)--><div class='quotetop'>QUOTE (Yuuki @ Apr 15 2012, 12:10 PM) <a href="index.php?act=findpost&pid=1925657"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Ho thanks, that's makes more sense. Another question, the lua hot-reload seems to be broken since a few builds, does that has something to do with optimization ?<!--QuoteEnd--></div><!--QuoteEEnd--> I think they said they changed it so it reloads every script now instead of just the changed one.
ok Thanks for the answers/explanations guys, I asked because I wanted to learn <b>not</b> because of doubts and I still have all the faith that UWE will do it in time.
Here's the thing. As the customer, I don't give a ###### why your framerate sucks. I don't have to care about LUA and coding levels and bottlenecks and tradeoffs. I'm the guy paying money. I don't care. All I know is your framerate sucks. I don't give a ###### why it sucks. I just know it makes me want to play something else and that your job is to make the framerates and server lag better - not to make excuses.
<!--quoteo(post=1925706:date=Apr 15 2012, 02:57 PM:name=jkflipflop)--><div class='quotetop'>QUOTE (jkflipflop @ Apr 15 2012, 02:57 PM) <a href="index.php?act=findpost&pid=1925706"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Here's the thing. As the customer, I don't give a ###### why your framerate sucks. I don't have to care about LUA and coding levels and bottlenecks and tradeoffs. I'm the guy paying money. I don't care. All I know is your framerate sucks. I don't give a ###### why it sucks. I just know it makes me want to play something else and that your job is to make the framerates and server lag better - not to make excuses.<!--QuoteEnd--></div><!--QuoteEEnd-->
They're not making excuses, though. They're explaining where they are, where they're going and so on. They're working on the problem and letting us in on that. If you don't want to take part in a real beta test, you're free to go with Guild Wars 2 or that sort of thing (where the game is basically finished and they're just using you as a marketing tool).
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->Lua takes up around 50% of the CPU cycles that NS2 goes through to produce every frame.<!--QuoteEnd--></div><!--QuoteEEnd--> So... it'd be equally efficient to work on the non-Lua parts? Are those less diverse than all the stuff coded in Lua? How much time is spent optimizing the Lua code rather than how the Lua code is handled by the engine? I mean, buildings "thinking" too often (like the hydras scanning for targets) seems like a fairly obvious issue, but maybe that's just hindsight...
IronHorseDeveloper, QA Manager, Technical Support & contributorJoin Date: 2010-05-08Member: 71669Members, Super Administrators, Forum Admins, Forum Moderators, NS2 Developer, NS2 Playtester, Squad Five Blue, Subnautica Playtester, Subnautica PT Lead, Pistachionauts
edited April 2012
<!--quoteo(post=1925710:date=Apr 15 2012, 11:09 AM:name=player)--><div class='quotetop'>QUOTE (player @ Apr 15 2012, 11:09 AM) <a href="index.php?act=findpost&pid=1925710"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Yea talking about being used as tools, this hasn't been a real beta either you know...<!--QuoteEnd--></div><!--QuoteEEnd-->
oohh its one of those FAKE betas, isn't it? damn, i didn't get the memo.
Some of you really just need to wait for 1.0 because your expectations for an unfinished product are laughable at best.
@jkflip: nothing is owed to you, read the hundreds of disclaimers; you are playing a work in progress, being allowed to play the game currently is a bonus from pre ordering. Also, keep it civil or don't post at all.
fsfodukJoin Date: 2004-04-09Member: 27810Members, NS2 Developer, Constellation, NS2 Playtester, Squad Five Blue, Squad Five Silver, Squad Five Gold, Subnautica Playtester, NS2 Community Developer, Pistachionauts
I bet Max's new Lua VM will never match the speed of LuaJIT when it's JIT is not hindered by Lua C functions causing JIT traces to abort. Changing the engine's api thats exposed to lua and the meta methods of the lua objects created by the engine into LuaJIT fast functions with corresponding trace recorder(generates the IR code for the Jit) functions like <b><a href="http://pastebin.com/6nUZYiAT" target="_blank">this</a></b> or cdata FFI functions would removes the main reason for traces aborting
To give an example of what this translates into. This piece of lua code which is basically freeview movement takes an average of 20k cycles(rdtsc on a 2600k@4.9ghz) to run in LuaJIT using custom fast functions and some FFI functions and cdata vs 34k in the normal lua vm and in a dedicated server Lua vm its 30k with LuaJIT vs 60k in the normal vm. The timer takes an average of 1k cycles on its own <!--c1--><div class='codetop'>CODE</div><div class='codemain'><!--ec1-->function Player:OnProcessMove(input)
local dotimer = input.move.x ~= 0 or input.move.y ~= 0 or input.move.z ~= 0
The fallacy of the scripting lanuage: 1. Engine programmer decides that the main engine language (C++) is to complicated for the people who make the game. He integrates a simple scripting engine for "non-programmers". 2. Non-programmers write unmaintanable and inefficient code because they have not much experience. 3. Whole thing blows up. 4a) Real programmer decides to rewrite game parts in C++ 4b) Real programmer fixes scripting code
Same thing happend with Far Cry. If you looked at it is was a complete mess: Parts of the game were in LUA others in C++. They did not have a clear interface or anything. For further modding 4b) is clearly the better choice and you maintain the advantages of going with the scripting approach.
In my opinion scripting languages should only be used for dead simple stuff like scripted sequences which overlapps with game content. Any kind of game logic should be done by programmers with experience.
<!--quoteo(post=1925720:date=Apr 15 2012, 09:22 PM:name=tobias3)--><div class='quotetop'>QUOTE (tobias3 @ Apr 15 2012, 09:22 PM) <a href="index.php?act=findpost&pid=1925720"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->The fallacy of the scripting lanuage: 1. Engine programmer decides that the main engine language (C++) is to complicated for the people who make the game. He integrates a simple scripting engine for "non-programmers". 2. Non-programmers write unmaintanable and inefficient code because they have not much experience. 3. Whole thing blows up. 4a) Real programmer decides to rewrite game parts in C++ 4b) Real programmer fixes scripting code
Same thing happend with Far Cry. If you looked at it is was a complete mess: Parts of the game were in LUA others in C++. They did not have a clear interface or anything. For further modding 4b) is clearly the better choice and you maintain the advantages of going with the scripting approach.
In my opinion scripting languages should only be used for dead simple stuff like scripted sequences which overlapps with game content. Any kind of game logic should be done by programmers with experience.<!--QuoteEnd--></div><!--QuoteEEnd-->
Let's not knock UWE too much - I know that coding is very artistic and open to critisism, but NS and NS2 are still the best games and the most addictive I have ever played. Keep it up and don't give up...
Still, I am running NS2 in lowest grapics settings on what I once thought was and absolute beast of a machine wondering why I am not just playing NS1 on it's highest... Ahh, I know why - flamethrowers, GL's and Alien Comm...
<!--quoteo(post=1925725:date=Apr 15 2012, 09:39 PM:name=profjekyll)--><div class='quotetop'>QUOTE (profjekyll @ Apr 15 2012, 09:39 PM) <a href="index.php?act=findpost&pid=1925725"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Sorry to be dense, but people keep saying "great read" etc... but I can't seem to see what you are all reading! Help?<!--QuoteEnd--></div><!--QuoteEEnd-->
Really though, I'm surprised they didn't just make the whole thing in C++ and then sell it with the source code included. Or do it UDK style where you can plug custom DLL files into the engine to get the desired results. I can see why they wanted to do what they wanted to, but it was an idealistic viewpoint and the game has paid the consequences. At this point, I'd have rather dropped modding support just to have the game play nice.
<!--quoteo(post=1925720:date=Apr 15 2012, 08:22 PM:name=tobias3)--><div class='quotetop'>QUOTE (tobias3 @ Apr 15 2012, 08:22 PM) <a href="index.php?act=findpost&pid=1925720"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->The fallacy of the scripting lanuage: 1. Engine programmer decides that the main engine language (C++) is to complicated for the people who make the game. He integrates a simple scripting engine for "non-programmers". 2. Non-programmers write unmaintanable and inefficient code because they have not much experience. 3. Whole thing blows up. 4a) Real programmer decides to rewrite game parts in C++ 4b) Real programmer fixes scripting code
Same thing happend with Far Cry. If you looked at it is was a complete mess: Parts of the game were in LUA others in C++. They did not have a clear interface or anything. For further modding 4b) is clearly the better choice and you maintain the advantages of going with the scripting approach.
In my opinion scripting languages should only be used for dead simple stuff like scripted sequences which overlapps with game content. Any kind of game logic should be done by programmers with experience.<!--QuoteEnd--></div><!--QuoteEEnd-->
Being a programmer with years of experience, I agree up to a point. Performance will often see drastic changes just because of the way the code is organized. Sometimes, a particular function is seen as very complex by a programmer, and coded as such. Another programmer can walk in, reevaluate the same problem with a different point of view, and rewrite that same function to be a lot simpler and get a very nice boost in performance along the way.
Years and years ago, people managed to make software that sent humans on the moon. The hardware behind that feat wouldn't even drive a calculator today. As programmers, we have access to excessive amount of hardware to accomplish our goals. The difference is, back in the days, we had to optimize the crap out of every function, or it wouldn't run (let alone fit in the available memory space). Nowadays, we can float with functions that aren't as optimized. The end result is that programming is now easier, because code that is, let's say, "sloppier", can still run acceptably in terms of performance thanks to the hardware driving it. This leads to more people being able to do software programming, and thus is driving the cost of software development down, which is the large goal. However, all that new shiny hardware we get is not used to it's full potential if the code you run on it isn't optimal. It's a matter of balance.
That being said, I do not believe UWE employs sloppy programmers. When you start optimizing code, you quickly get to patterns. The best way to optimize a function is to not run it at all. That's the idea behind the hydra code change (let's stop wasting time trying to find precise targets if nobody is even vaguely around, shall we?). And that's the kind of changes you can quickly make to code and get impressive returns, once it's completed. It's important to remember that time spent on optimizing a portion of code that won't be used is time wasted. 1 or 2 days of work playing around with some functionality that finally gets removed or changed significantly and/or rewritten entirely is not only costly in time, but money (most of the time, programmers are not working for free). And if there's one thing UWE cannot afford to waste if it wants NS2 shipped this year, it's time (plus, do I really have to make an argument as to why UWE cannot afford to waste money?).
As it is now, the game is (mostly) playable. Once it's gonna be feature complete, time spent optimizing the various parts will be time well spent. Until then, reorganizing LUA scripts is more of a gamble than an investment.
The community, on the other hand, can try to find those "weak spots" in LUA scripts and suggest new code to improve performance. If anyone is really fed up with the performance of NS2, they can fire up the scripts, start to guess where performance is sinking, find an alternate way of scripting the game so the same functionality is achieved with a better performance, and suggest the change to UWE. Bonus points : you get your name in the changelog, are credited for a performance improvement, and instantly become a NS2 community god :D
<!--quoteo(post=1925483:date=Apr 15 2012, 01:53 AM:name=cron)--><div class='quotetop'>QUOTE (cron @ Apr 15 2012, 01:53 AM) <a href="index.php?act=findpost&pid=1925483"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->I wonder how much work it would be to migrate parts of the game, that are rock solid decided and not subject to change, to C++ code. You already have the Lua scripts communicating with C++ code through some interface; in every program there are those functions that are called much more often than others and eat up much more time, I imagine that is very much so the case with game logic and I wonder how much of an impact it would have to migrate just the top 1% hungry functions to precompiled code. As I remember the Luajit approach did not work out and I see how migrating some parts of the game code to another language is neither a clean solution nor does it help the ease of modding that you want to pursue, but if there is no other way to squeeze out performance, would that be possible? Migrating just the most called functions like line of sight-tracing or the pathing of AI entities to precompiled code sounds like last measure method with a huge potential.<!--QuoteEnd--></div><!--QuoteEEnd-->
^This!
I don't know LUA, but Python. So here are some questions / ideas to possibly improve performance. <ul><li>Are there tools to automatically translate LUA code into C++? If so, this should be made for every beta-release. Adding a flag or param for modders, which tells the engine to use the lua files instead of the translated C++ code.</li><li>Is there another way to pre compile the LUA code (optimized?) in a released beta build? The slow runtime interpretation of LUA Scripts should only be activate-able (via param or flag) for modding / development, but be deactivated on default for normal users.</li><li>If non of the above is possible, you should profile which functions are called the most and translate them to C++ (sure in exchange of less modability) as cron said.</li></ul>
And last but not least: Thanks for the informational post to NS2HD. It was hell of a time to get a statement like this.
Thing that makes me wonder the most is that modders will generally only work on a game engine of a really successfull game, after all its no small thing to make a full featured mod and takes a lot of investment in terms of time and commitment... you wouldnt want to spend 2 years making a amazing mod and find there is no audiance as the core game that introduced the engine wasnt too popular.
Its great that UWE are really supporting modding but how big will the mod scene on the NS2 engine be if NS 2 doesnt have a smooth launch ? If the launch is less than ideal the audiance for mods on the NS2 engine will be small, thus limiting the intrest to mod on that engine. UWE will have to magically boost the performance radically so the current amount of LUA can be kept in, or sacrifice the ability to mod and get a smooth launch done.
... currently it seems UWE is too focused on the modding aspect and is risking not making a big enough entry with its core engine introduction game, I mean would the Source engine be much of a big deal if it wasnt for Half Life 2 ?
I am content to wait for UWE to deliver, after all this is a WIP and besides I got my moneys worth from NS 1, glad to help Flayra and team hopefully get NS 2 off the ground and get UWE on the map.
<!--quoteo(post=1925800:date=Apr 16 2012, 11:42 AM:name=Scrogneugneu)--><div class='quotetop'>QUOTE (Scrogneugneu @ Apr 16 2012, 11:42 AM) <a href="index.php?act=findpost&pid=1925800"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Being a programmer with years of experience... NS2 community god :D<!--QuoteEnd--></div><!--QuoteEEnd-->
Great post Scrogneugneu. Anyone that read my blog post should then come and read your post here.
I know it may be a bit troubling to have poor performance. Still I would like to point out that the guys at UWE are not infallable - they are just human. There is nothing like experience to refine things...
I expect it must be pretty frustrating for THEM that the many hours of patient development that has been put into this iconic game (for me at least) has perhaps had a percentage of time poorly invested. I am also aware of the disappointment that this may have caused them.
Lets encourage them - I sincerely hope they suceed (one day) of getting this awesome concept to what their goal was.
Having said that, I am concerned about scalability. What happens on bigger maps people develope? What happens when someone does place 10 hydras? What happens when someone does make an ARC train?
Perhaps a partial rewrite is the final destination, but lets enjoy the game as best we can for the time being. Even a partial rewrite wouldn't void much of the work UWE has invested, many of the classes can be pretty easily transferred...
InsaneAnomalyJoin Date: 2002-05-13Member: 605Members, Super Administrators, Forum Admins, NS1 Playtester, Forum Moderators, NS2 Developer, Constellation, NS2 Playtester, Squad Five Blue, NS2 Map Tester, Subnautica Developer, Pistachionauts, Future Perfect Developer
<!--quoteo(post=1925720:date=Apr 15 2012, 08:22 PM:name=tobias3)--><div class='quotetop'>QUOTE (tobias3 @ Apr 15 2012, 08:22 PM) <a href="index.php?act=findpost&pid=1925720"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->Any kind of game logic should be done by programmers with experience.<!--QuoteEnd--></div><!--QuoteEEnd-->
All of our game programmers are talented and experienced.
Without backtracking too much, just the other day I was slagging off high level programming languages. You can always tell which download from downloads.com is going to run faster - the one that is less than a meg big (C / C++) or the one that is 50+ meg big (java etc).
Comments
have a look the NS2HD article again
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->The former is done constantly, through tweaks to the way the game thinks. For example, the original Hydra Lua code called on the Hydra to scan for targets several times a second, irrespective of whether there were any marines nearby. This caused immense lag in early NS2 builds (149, anyone?) because Hydras were constantly scanning for targets.<!--QuoteEnd--></div><!--QuoteEEnd-->
Buildings, there actions and the whole game logic for NS2 cost the most performance (all written in Lua)
<!--quoteo--><div class='quotetop'>QUOTE </div><div class='quotemain'><!--quotec-->Right now, Lua takes up around 50% of the CPU cycles that NS2 goes through to produce ever frame. This is, on its own, almost entirely responsible for the limited frame rates that people experience when playing in the NS2 Beta. Rendering in Spark is actually very fast. This is why turning down graphics options, or playing on a more powerful GPU, yields negligible performance benefits.<!--QuoteEnd--></div><!--QuoteEEnd-->
So if you would write a small CS like mod(without the complicated game mechanism) the game would run super smooth!
To use Lua as a scripting language way maybe the best decision UW did.
Why?
Well the dev. team of NS2 is small... really small compared to other game dev. teams where 50 to 100ppl work on one game over 2-3 years the goal they already reached in this "short" amount of time is amazing.
Lua is speeding up the process of development a lot!
It make it easy for UW to try new things, play with values, without recompiling the whole game every single time(and even the community can help).
A lot of other games also use Lua as their script engines
<a href="http://en.wikipedia.org/wiki/Category:Lua-scripted_video_games" target="_blank">http://en.wikipedia.org/wiki/Category:Lua-...ted_video_games</a>
To hardcode the whole game in C++ is not state of the art
The engine itself seems to be really well planned, new and efficient technologies are used.
Sure you guy expect a lag free game, but this is a BETA! I know most other beta games run great, but those betas are tested internal for months. UW doesn't have the capacity to do that.
If you develop software you first implement features and then you optimize them, that’s just how it works.
It will take couple of weeks, but NS2 is great and will be fast!
<a href="http://en.wikipedia.org/wiki/Category:Lua-scripted_video_games" target="_blank">http://en.wikipedia.org/wiki/Category:Lua-...ted_video_games</a><!--QuoteEnd--></div><!--QuoteEEnd-->
I saw Operation Flashpoint in that list - I don't know how that game was, but it was definitely one of the best looking FPS games I had seen at the time. If that game ran well, then it's 100% possible for NS2 to run great.
You can alternatively call methods by using a dot and simply passing 'self' as the first argument.
<!--quoteo(post=0:date=:name=w0dk4)--><div class='quotetop'>QUOTE (w0dk4)</div><div class='quotemain'><!--quotec-->So, at best, this will result in a 25% performance increase (btw. I thought this was already done).
Of course this increases FPS and playability, but i.e. on the server side, at least a 100% perf. increase is needed (which is your goal for release apparently).
I hope you can pull it through but I also urge to think about more drastic actions. And the argument that being CPU bound is a good thing must be the joke of the week. CPU being the bottleneck is the worst thing that can happen to your game, since players can not "fix" it by lowering the graphics settings (especially important for competitive scene).<!--QuoteEnd--></div><!--QuoteEEnd-->
The binding-layer was rewritten (back in B171), this would be the VM that translates bytecode into an actual instruction. I guess NS2HD's arithmetic is Lua being half of the frame-time, so a 50% increase * 0.5 = 25% FPS increase. For the server it would be more than half of the frame-time (maybe closer to 75%? who knows), so it'll be more than 25% there. And yes a CPU-bottlenecked game is pretty bad, because we're talking about a single-threaded bottleneck, and (aside overclocking severely) there's not much you can about it.
<!--quoteo(post=0:date=:name=rammaj)--><div class='quotetop'>QUOTE (rammaj)</div><div class='quotemain'><!--quotec-->On another note I am asking this because I have no background in programming but say if I wanted to create a game using this engine and LUA, and the game logic isnt as complex as the one in NS2 ( I am talking about the whole layer so including UI, how structures work, weapons etc ) lets say something like Counter Strike for argument's sake will the client/server run smoother if it has less lines of code giving that it is allso well optimized? and will it be a huge difference or very small and barely noticeable?<!--QuoteEnd--></div><!--QuoteEEnd-->
It's true that NS2 is quite a bit more complex than say CS:S, so the things would not have been as problematic such a game. Lua can be used, but I guess not to such a degree as in NS2 (where indeed almost all of the game-logic is being handled by it). Take a look at how garry's mod does this, I believe he struck a good balance between moddability while retaining performance.
<!--QuoteBegin-'rammaj'+--><div class='quotetop'>QUOTE ('rammaj')</div><div class='quotemain'><!--QuoteEBegin-->if they decide to go with C++ may that be everything or just some parts would that mean they have to rewrite a certain part of the engine to call code from c++ instead of LUA? or can they just start doing it?<!--QuoteEnd--></div><!--QuoteEEnd-->
Well certainly parts of the engine are geared to work specifically with that Lua-API, so some work would be in order there.
<!--quoteo(post=0:date=:name=F1ll3r)--><div class='quotetop'>QUOTE (F1ll3r)</div><div class='quotemain'><!--quotec-->To use Lua as a scripting language way maybe the best decision UW did.
Why?
Well the dev. team of NS2 is small... really small compared to other game dev. teams where 50 to 100ppl work on one game over 2-3 years the goal they already reached in this "short" amount of time is amazing.
Lua is speeding up the process of development a lot!
It make it easy for UW to try new things, play with values, without recompiling the whole game every single time(and even the community can help).
A lot of other games also use Lua as their script engines
<a href="http://en.wikipedia.org/wiki/Category:Lua-...ted_video_games" target="_blank">http://en.wikipedia.org/wiki/Category:Lua-...ted_video_games</a>
To hardcode the whole game in C++ is not state of the art
The engine itself seems to be really well planned, new and efficient technologies are used.
Sure you guy expect a lag free game, but this is a BETA! I know most other beta games run great, but those betas are tested internal for months. UW doesn't have the capacity to do that.
If you develop software you first implement features and then you optimize them, that’s just how it works.
It will take couple of weeks, but NS2 is great and will be fast!<!--QuoteEnd--></div><!--QuoteEEnd-->
You shouldn't forget that the path of Lua has given Max (and assorted engine-programmers) a good amount of grief trying to cope with the performance-issues, so there's certainly a trade-off there. Anyway, if they can pull it off to yield good performance, then using Lua has been an excellent idea, but that is what is in question now. Your guess is as good as any at this point.
<!--quoteo(post=0:date=:name=Deadzone)--><div class='quotetop'>QUOTE (Deadzone)</div><div class='quotemain'><!--quotec-->I saw Operation Flashpoint in that list - I don't know how that game was, but it was definitely one of the best looking FPS games I had seen at the time. If that game ran well, then it's 100% possible for NS2 to run great.<!--QuoteEnd--></div><!--QuoteEEnd-->
I do believe NS2 would be the first game to really use Lua to such an extend. I've peaked around at other games using Lua (FarCry, Garry's Mod), but they don't go as far in this as NS2 does (and as such, limiting moddability in that respect of course).
In case it seems like I'm coming across as overly critical:
Join one of the top servers in NS2, punch in "net_stats" in the console, and wait until about 3/4 of the way into the game and watch the tickrate fall to nearly 10. Anything below 30 is unacceptable really (though right now you won't see it go above 30, due to it being capped there). Also remember that these (popular) servers are running on some serious hardware (between 4 and 5 GHz), which is far FAR higher than what public servers will be running when the game is released (and what all the vacant servers at the bottom of the server-list are running). Add to that that the amount of slots isn't as high as they have to be yet either (the largest is 20 I believe, while we should be aiming for 32). If you add all that together, it isn't odd to come to a number of 500-800% performance-increase required for the server at this point.
I think what he's getting at is that they can improve it without getting into a lot of GPU issues that differ much more from card to card (even manufacturer to manufacturer). When you write NS2 code that runs on the CPU, you're always writing for different flavours of x86, and you can ignore all the hardware features that aren't common to all your players.
As has been mentioned, the graphics engine in Spark is pretty robust, and performs/scales very well. A big push to optimize the GPU side of things would do very little for most players.
Hopefully at the end of things, the game will be GPU-bottlenecked for most players (after they all buy i5-2500ks to play it). When people say the game is CPU-bound, they mean that it is<i> right now </i>because the LUA integration work is unfinished. Not that it will be forever (but that's possible too).
<!--quoteo(post=1925641:date=Apr 15 2012, 11:35 AM:name=Deadzone)--><div class='quotetop'>QUOTE (Deadzone @ Apr 15 2012, 11:35 AM) <a href="index.php?act=findpost&pid=1925641"><{POST_SNAPBACK}></a></div><div class='quotemain'><!--quotec-->I saw Operation Flashpoint in that list - I don't know how that game was, but it was definitely one of the best looking FPS games I had seen at the time. If that game ran well, then it's 100% possible for NS2 to run great.<!--QuoteEnd--></div><!--QuoteEEnd-->
It probably didn't use LUA for the entirety of the game logic.
Ho thanks, that's makes more sense. Another question, the lua hot-reload seems to be broken since a few builds, does that has something to do with optimization ?
I think they said they changed it so it reloads every script now instead of just the changed one.
They're not making excuses, though. They're explaining where they are, where they're going and so on. They're working on the problem and letting us in on that.
If you don't want to take part in a real beta test, you're free to go with Guild Wars 2 or that sort of thing (where the game is basically finished and they're just using you as a marketing tool).
So... it'd be equally efficient to work on the non-Lua parts? Are those less diverse than all the stuff coded in Lua?
How much time is spent optimizing the Lua code rather than how the Lua code is handled by the engine? I mean, buildings "thinking" too often (like the hydras scanning for targets) seems like a fairly obvious issue, but maybe that's just hindsight...
oohh its one of those FAKE betas, isn't it?
damn, i didn't get the memo.
Some of you really just need to wait for 1.0 because your expectations for an unfinished product are laughable at best.
@jkflip: nothing is owed to you, read the hundreds of disclaimers; you are playing a work in progress, being allowed to play the game currently is a bonus from pre ordering. Also, keep it civil or don't post at all.
To give an example of what this translates into. This piece of lua code which is basically freeview movement takes an average of 20k cycles(rdtsc on a 2600k@4.9ghz) to run in LuaJIT using custom fast functions and some FFI functions and cdata vs 34k in the normal lua vm and in a dedicated server Lua vm its 30k with LuaJIT vs 60k in the normal vm.
The timer takes an average of 1k cycles on its own
<!--c1--><div class='codetop'>CODE</div><div class='codemain'><!--ec1-->function Player:OnProcessMove(input)
local dotimer = input.move.x ~= 0 or input.move.y ~= 0 or input.move.z ~= 0
if(dotimer) then
StartTimer(MT_Names[MT_I])
end
self.viewPitch = input.pitch
self.viewRoll = input.roll
self.viewYaw = input.yaw
if(input.move.x ~= 0 or input.move.y ~= 0 or input.move.z ~= 0) then
local velocity = self.velocity
local angles = Angles(input.pitch, input.yaw, 0)
local viewCoords = angles:GetCoords()
// Apply acceleration in the direction we're looking (flying).
local moveVelocity = viewCoords:TransformVector(input.move) * kAcceleration
velocity = velocity + moveVelocity * input.time
// Apply friction.
local frictionForce = Vector(-velocity.x, -velocity.y, -velocity.z) * 5
velocity = velocity + frictionForce * input.time
// Clamp speed.
local velocityLength = velocity:GetLength()
if velocityLength > kMaxSpeed then
velocity:Scale(kMaxSpeed / velocityLength)
end
local offset = velocity * input.time
local newOrigin = Vector(self:GetOrigin()) + offset
self:SetOrigin(newOrigin)
self:SetVelocity(velocity)
StopTimer(MT_Names[MT_I])
end
end<!--c2--></div><!--ec2-->
1. Engine programmer decides that the main engine language (C++) is to complicated for the people who make the game. He integrates a simple scripting engine for "non-programmers".
2. Non-programmers write unmaintanable and inefficient code because they have not much experience.
3. Whole thing blows up.
4a) Real programmer decides to rewrite game parts in C++
4b) Real programmer fixes scripting code
Same thing happend with Far Cry. If you looked at it is was a complete mess: Parts of the game were in LUA others in C++. They did not have a clear interface or anything. For further modding 4b) is clearly the better choice and you maintain the advantages of going with the scripting approach.
In my opinion scripting languages should only be used for dead simple stuff like scripted sequences which overlapps with game content. Any kind of game logic should be done by programmers with experience.
1. Engine programmer decides that the main engine language (C++) is to complicated for the people who make the game. He integrates a simple scripting engine for "non-programmers".
2. Non-programmers write unmaintanable and inefficient code because they have not much experience.
3. Whole thing blows up.
4a) Real programmer decides to rewrite game parts in C++
4b) Real programmer fixes scripting code
Same thing happend with Far Cry. If you looked at it is was a complete mess: Parts of the game were in LUA others in C++. They did not have a clear interface or anything. For further modding 4b) is clearly the better choice and you maintain the advantages of going with the scripting approach.
In my opinion scripting languages should only be used for dead simple stuff like scripted sequences which overlapps with game content. Any kind of game logic should be done by programmers with experience.<!--QuoteEnd--></div><!--QuoteEEnd-->
Let's not knock UWE too much - I know that coding is very artistic and open to critisism, but NS and NS2 are still the best games and the most addictive I have ever played. Keep it up and don't give up...
Still, I am running NS2 in lowest grapics settings on what I once thought was and absolute beast of a machine wondering why I am not just playing NS1 on it's highest... Ahh, I know why - flamethrowers, GL's and Alien Comm...
<a href="http://www.ns2hd.com/2012/04/performance-where-is-it-now-and-where.html" target="_blank">http://www.ns2hd.com/2012/04/performance-w...-and-where.html</a>
Really though, I'm surprised they didn't just make the whole thing in C++ and then sell it with the source code included. Or do it UDK style where you can plug custom DLL files into the engine to get the desired results. I can see why they wanted to do what they wanted to, but it was an idealistic viewpoint and the game has paid the consequences. At this point, I'd have rather dropped modding support just to have the game play nice.
1. Engine programmer decides that the main engine language (C++) is to complicated for the people who make the game. He integrates a simple scripting engine for "non-programmers".
2. Non-programmers write unmaintanable and inefficient code because they have not much experience.
3. Whole thing blows up.
4a) Real programmer decides to rewrite game parts in C++
4b) Real programmer fixes scripting code
Same thing happend with Far Cry. If you looked at it is was a complete mess: Parts of the game were in LUA others in C++. They did not have a clear interface or anything. For further modding 4b) is clearly the better choice and you maintain the advantages of going with the scripting approach.
In my opinion scripting languages should only be used for dead simple stuff like scripted sequences which overlapps with game content. Any kind of game logic should be done by programmers with experience.<!--QuoteEnd--></div><!--QuoteEEnd-->
Being a programmer with years of experience, I agree up to a point. Performance will often see drastic changes just because of the way the code is organized. Sometimes, a particular function is seen as very complex by a programmer, and coded as such. Another programmer can walk in, reevaluate the same problem with a different point of view, and rewrite that same function to be a lot simpler and get a very nice boost in performance along the way.
Years and years ago, people managed to make software that sent humans on the moon. The hardware behind that feat wouldn't even drive a calculator today. As programmers, we have access to excessive amount of hardware to accomplish our goals. The difference is, back in the days, we had to optimize the crap out of every function, or it wouldn't run (let alone fit in the available memory space). Nowadays, we can float with functions that aren't as optimized. The end result is that programming is now easier, because code that is, let's say, "sloppier", can still run acceptably in terms of performance thanks to the hardware driving it. This leads to more people being able to do software programming, and thus is driving the cost of software development down, which is the large goal. However, all that new shiny hardware we get is not used to it's full potential if the code you run on it isn't optimal. It's a matter of balance.
That being said, I do not believe UWE employs sloppy programmers. When you start optimizing code, you quickly get to patterns. The best way to optimize a function is to not run it at all. That's the idea behind the hydra code change (let's stop wasting time trying to find precise targets if nobody is even vaguely around, shall we?). And that's the kind of changes you can quickly make to code and get impressive returns, once it's completed. It's important to remember that time spent on optimizing a portion of code that won't be used is time wasted. 1 or 2 days of work playing around with some functionality that finally gets removed or changed significantly and/or rewritten entirely is not only costly in time, but money (most of the time, programmers are not working for free). And if there's one thing UWE cannot afford to waste if it wants NS2 shipped this year, it's time (plus, do I really have to make an argument as to why UWE cannot afford to waste money?).
As it is now, the game is (mostly) playable. Once it's gonna be feature complete, time spent optimizing the various parts will be time well spent. Until then, reorganizing LUA scripts is more of a gamble than an investment.
The community, on the other hand, can try to find those "weak spots" in LUA scripts and suggest new code to improve performance. If anyone is really fed up with the performance of NS2, they can fire up the scripts, start to guess where performance is sinking, find an alternate way of scripting the game so the same functionality is achieved with a better performance, and suggest the change to UWE. Bonus points : you get your name in the changelog, are credited for a performance improvement, and instantly become a NS2 community god :D
As I remember the Luajit approach did not work out and I see how migrating some parts of the game code to another language is neither a clean solution nor does it help the ease of modding that you want to pursue, but if there is no other way to squeeze out performance, would that be possible?
Migrating just the most called functions like line of sight-tracing or the pathing of AI entities to precompiled code sounds like last measure method with a huge potential.<!--QuoteEnd--></div><!--QuoteEEnd-->
^This!
I don't know LUA, but Python. So here are some questions / ideas to possibly improve performance.
<ul><li>Are there tools to automatically translate LUA code into C++? If so, this should be made for every beta-release. Adding a flag or param for modders, which tells the engine to use the lua files instead of the translated C++ code.</li><li>Is there another way to pre compile the LUA code (optimized?) in a released beta build? The slow runtime interpretation of LUA Scripts should only be activate-able (via param or flag) for modding / development, but be deactivated on default for normal users.</li><li>If non of the above is possible, you should profile which functions are called the most and translate them to C++ (sure in exchange of less modability) as cron said.</li></ul>
And last but not least: Thanks for the informational post to NS2HD. It was hell of a time to get a statement like this.
Its great that UWE are really supporting modding but how big will the mod scene on the NS2 engine be if NS 2 doesnt have a smooth launch ? If the launch is less than ideal the audiance for mods on the NS2 engine will be small, thus limiting the intrest to mod on that engine.
UWE will have to magically boost the performance radically so the current amount of LUA can be kept in, or sacrifice the ability to mod and get a smooth launch done.
... currently it seems UWE is too focused on the modding aspect and is risking not making a big enough entry with its core engine introduction game, I mean would the Source engine be much of a big deal if it wasnt for Half Life 2 ?
I am content to wait for UWE to deliver, after all this is a WIP and besides I got my moneys worth from NS 1, glad to help Flayra and team hopefully get NS 2 off the ground and get UWE on the map.
Great post Scrogneugneu. Anyone that read my blog post should then come and read your post here.
I expect it must be pretty frustrating for THEM that the many hours of patient development that has been put into this iconic game (for me at least) has perhaps had a percentage of time poorly invested. I am also aware of the disappointment that this may have caused them.
Lets encourage them - I sincerely hope they suceed (one day) of getting this awesome concept to what their goal was.
Perhaps a partial rewrite is the final destination, but lets enjoy the game as best we can for the time being. Even a partial rewrite wouldn't void much of the work UWE has invested, many of the classes can be pretty easily transferred...
All of our game programmers are talented and experienced.