Development Blog Update - 15th podcast
Flayra
Game Director, Unknown Worlds EntertainmentSan Francisco Join Date: 2002-01-22 Member: 3Super Administrators, NS2 Developer, Subnautica Developer
Game Director, Unknown Worlds EntertainmentSan Francisco Join Date: 2002-01-22 Member: 3Super Administrators, NS2 Developer, Subnautica Developer
Please post comments on the topic Development Blog Update - <a href="http://www.unknownworlds.com/ns2/news/2008/01/15th_podcast" target="_blank">15th podcast here</a>
<a href="http://www.unknownworlds.com/ns2/forums/index.php?s=&showtopic=103563&view=findpost&p=1666902" target="_blank">Transcript </a>- Thanks afratnikov!
<a href="http://www.unknownworlds.com/ns2/forums/index.php?s=&showtopic=103563&view=findpost&p=1666902" target="_blank">Transcript </a>- Thanks afratnikov!
Comments
Don't get too far into technical detail, it bores the non-programmers, unless you want to add a tag warning people to not waste their time listening to something they don't understand/care about.
Flayra, you created something strange, a phenomenon ... but you need to feed the original beast then see to the offspring, the new cubs will treat this as a ned mod, the old will revere it. I suggest lose the CO portion for now, NS cant support 2 communities.
I suggest you create a NS source, then add the new stuff. NS was a great game, but it is old news...bring it back to the headlines then create the ultimate NS.
i know it was probably just an example, but please please don't remove dymanic infestation - even if its competley nerfed to the point where it doesnt affect gameply and only adds to atmosphere.
also i don't quite understand why you can't have unit testing and player testing at the same time? i don't know much about it - do they conflict with each other ?
First, I love the tech talk, the blathering on. Being candid is very cool, it helps me empathize a lot with your processes.
Second, I want NS2, not NS:S. Anyone who has noticed my thoughts and ideas in my various posts would recognize that I feel there are great things about NS1 but serious flaws as well to one's potential enjoyment in the game, which as I understand it was at least partly do to the limitations that were found in being a mod for a existing game. I waited patiently for HL2, I can wait patiently for the next big thing, NS2 ... and Spore.
First, I love the tech talk, the blathering on. Being candid is very cool, it helps me empathize a lot with your processes.
Second, I want NS2, not NS:S. Anyone who has noticed my thoughts and ideas in my various posts would recognize that I feel there are great things about NS1 but serious flaws as well to one's potential enjoyment in the game, which as I understand it was at least partly do to the limitations that were found in being a mod for a existing game. I waited patiently for HL2, I can wait patiently for the next big thing, NS2 ... and Spore.<!--QuoteEnd--></div><!--QuoteEEnd-->
I agree totally. I love the tech talk, can’t get enough of it.
More media would be awesome. Would love to see another DI video, and any other new features you are adding (IE alien tunnel thingies). Can’t wait to see this stuff hopefully this year: D
Keep up the great work/POD casts/etc
Don't get too far into technical detail, it bores the non-programmers, unless you want to add a tag warning people to not waste their time listening to something they don't understand/care about.<!--QuoteEnd--></div><!--QuoteEEnd-->
*sigh* god i'm getting sick of this troll, yet i feel compelled to keep feeding him...
i disagree totally domining. i'm not a programmer, but i still find their conversations about programming or the game industry very interesting. its good to hear them thrashing out ideas, sharing their thought processes with the community - it makes the game development process heaps better from a consumer perspective.
and for the record, right under the heading, and just above the podcast recording, they do actually post a brief description as to what each podcast will contain... it cant get any more obvious than that.
great to hear you guys are planning on releasing some screenshots and more videos in the not to distant future, i cant wait. keep up the good work!
*edit* ASnogarD, dont be to worried about NS2 not being NS:S. i'm sure once it is released someone will tinker with LUA to create a version of NS:S to play on custom servers. but i reckon there's going to be so much more cool stuff in NS2 that no one would bother to play NS:S...
Warning: The following post is techy!
<b>Unit testing:</b>
I agree that unit testing can be a pain in the butt to implement, especially because some things are just damn near impossible to encapsulate in a unit test. What I often find most difficult about it is that you need to know exactly what your new method/function is going to produce before you've written it, and that can sometimes fly in the face of the concept that you can't know how to build something in software development until <i>after</i> you've built it. If you don't know how it will work, you can't fully know what it will do, and so how can you hope to put that in a unit test?
That being said, I still believe it is worth the effort. One thing you guys mentioned, that I find is the greatest strength of unit testing is peace of mind. It's not just that you can run a battery of tests and be happy that all the lights are still green, it's also that the unit test also defines the constraints of the method you are implementing, and as you say, with constraints comes freedom. Freedom to change. When every object becomes a black box, with only a set of inputs and outputs exposed to the outside world, and your unit tests reflect this by only throwing input at your box and ensuring that the output is as expected, you suddenly find that you are forced to conform to those constraints in every other part of your program, that you force yourself to use the inputs and outputs only as intended. With this knowledge, when you go to refactor your little black box, you <i>know</i> nothing else will break well before you run any tests on it.
Unfortunately however, I've personally found that unit testing is a square-shaped hole. Not every peg will fit inside it. It is excellent for providing verification (it works), but not validation (it's what you wanted). You can't exactly unit test for fun, or aesthetics. On top of that, unit tests can still only check for what you can imagine. I've found unit testing can suffer from the same problems as any other form of testing. When the same person writes the test and implements the code, it is very easy for them to forget about the edge cases that may seem obvious to someone else. Writing the test in advance isn't going to going to suddenly gift you with the mindset of an exploiter out to ruin other people's fun.
<b>Estimating:</b>
Don't beat yourself up about this! Listen to the wisdom of <a href="http://en.wikipedia.org/wiki/Hofstadter's_Law" target="_blank">Hofstadter</a>!
Look at what you're estimating: How long it will take you to gain an understanding of a problem and then create a unique solution for it. How can you know how long it will take to implement a solution when you don't even know what the problem actually <i>is</i> yet? You are trying to estimate the construction time for a house before you have even looked at the land you are going to build it on.
And so ofcourse once you start building, and you immediately start gaining insights on where you went wrong in your initial design. This isn't a bad thing! This is where a common bedfellow of unit testing comes into play: <a href="http://en.wikipedia.org/wiki/Iterative_and_incremental_development" target="_blank">Iteration</a>. Design a small chunk of your software, implement it, see what you learned from that and then redesign that chunk armed with your new knowledge. In effect, you start <a href="http://en.wikipedia.org/wiki/Software_prototyping" target="_blank">prototyping</a> everything in tiny little chunks, with each new chunk giving you lessons that help you build a better end product.
What is the potential cost of not proactively prototyping and iterating? Look at Warhammer Online. They created the entire Ork/Dwarf area. They were <i>finished</i> with that part of the game, and moved on to creating the Human/Chaos areas. Once they were halfway done with that, they looked back on what they had done with the Orks & Dwarfs and realised they could have done so much better. As a result, they started a full third of their game all over again. The same thing happened with <i>both</i> Half-Life games. Hunt around for old Gabe Newell interviews, and you will eventually find him saying that both games were delayed a year because by the time the release date had come around, they were looking at a nearly finished product that they simply weren't happy with. Both times, they started from scratch, armed with the lessons they had learned and produced ground-breaking games.
Waiting until the end of your development life to realise that you've merely completed your first prototype leaves you with exactly these sorts of decisions: Do I release a sub-par product on time, now that I know exactly how to make it all so much better? Do I start all over again, costing me or my producers a fortune and infuriating my expectant customers? Actively acknowledging this property of software development, and capitalising on it produce a better product, can turn what was previously a pitfall into a huge boon.
This does ofcourse all tie back into estimation, and why many developers have stopped doing it. Once you are working iteratively, your plans and predications can't reliably extend past your current iteration. Mind you, you haven't really lost anything, your predictions for the full life cycle were bogus anyway. This does lead to some major friction with the business world however, and this is where the guys at Valve and Mythic are really lucky to be paid by people who demand quality above all else. The business world revolves around long term planning. Budgets, fiscal forecasts and what-not, and that currently isn't compatible with <i>quality</i> software development. Few investors are willing to take bets on uncertain time-frames, just as they are often unwilling to take bets on new genres and mold-breaking gameplay.
This also ties into your final point on industry new year's resolutions too. I would suggest that the new year's resolutions you have proposed for the industry would have to be taken to heart at the investment level before it could trickle down into the development level.
Don't get too far into technical detail, it bores the non-programmers, unless you want to add a tag warning people to not waste their time listening to something they don't understand/care about.<!--QuoteEnd--></div><!--QuoteEEnd-->
I second that.
Thanks to this design you can foresee a lot of problems that may rise in the future of the development process. So I in my opinion you can work faster using unit testing design.
Another advantage is that when you have written the fundamentals of you application using unit tests, you also have a better idea how much it would take to write code for every part. So your estimation can become more correct.
In reply to Revenge:
When I start out creating an application, I would write unit tests for every piece of the program. Therefore you get a decent idea what every method is going to produce. This eliminates the common problem that a methods signature would change like a billion times during development.
EDIT: I should probably mention that regardless of how I test/implement, I always work out my stable abstractions and map out the higher-level aspects of my design before I get started on anything, so I still have a fairly good picture of the overall shape of my program.
Id also like to see someone attempt to build a fpv 3d internet browser. its a pretty cyber punk idea but its turned up in enuth fiction and tv -Snow crash, neuromancer, and they have it in an episode of futurama. a fully adaptable and modable fpv interface that allowed you to view 3d webpages, communications systems, and games. If uve played the mod dystopia they have a 3rd cycber space in that but its still jsut a shooter in disguse. And there have been hacking simulators like uplink but theyve been a bit dull too.
any way i agree that the big budget game industry could try something different
...it's also that the unit test also defines the constraints of the method you are implementing, and as you say, with constraints comes freedom. Freedom to change.
...
Unfortunately however, I've personally found that unit testing is a square-shaped hole. Not every peg will fit inside it. It is excellent for providing verification (it works), but not validation (it's what you wanted). You can't exactly unit test for fun, or aesthetics. On top of that, unit tests can still only check for what you can imagine. I've found unit testing can suffer from the same problems as any other form of testing. When the same person writes the test and implements the code, it is very easy for them to forget about the edge cases that may seem obvious to someone else. Writing the test in advance isn't going to going to suddenly gift you with the mindset of an exploiter out to ruin other people's fun.<!--QuoteEnd--></div><!--QuoteEEnd-->
I'm glad to hear that the concept of unit testing (which at this point I think is established as a best, if not <i>ubiquitous</i>, practice) is being brought to the gaming industry (which from my external observations seems much for focused on time to market than quality, for sometimes understandable reasons...games have a limited shelf life and code is often not reusable).
My point of view is that in addition to "just" testing the code, the <i>process</i> of writing unit tests actually <i>tests the developer</i>, i.e., it tests and exposes hidden assumptions that live in the developers mind. Many many times just in the <i>process</i> of constructing a unit test I have exposed corner cases and unexpected behaviors in my code that as a programmer my mind just swept under the carpet, and have gone back and actually changed code as part of the unit testing process. It doesn't just happen after the fact. (advocates of "Test Driven Development" say to write the tests first and use them to guide design...I don't fully buy that dogma)
As to the difficulty of applying unit tests to everything, sure that is a problem. Unit tests test "units". This is most applicable for classes and library code. The idea is that if all your "units" are solid than any composition of them will also be solid. We all know that is not the case so there are multiple layers of testing: unit testing, integration testing (testing larger combinations of units, subsystems, etc.), functional testing (end users test the product to see that it meets their functional needs), quality assurance testing (diabolical testers do diabolical things to make sure the product doesn't break in undesired ways).
My conclusion is that unit testing cannot, and is not designed to be, absolutely comprehensive. Something like that requires formal proofs and is essentially a computer science research area and isn't all that applicable to commodity languages and products people build in the real world. Instead, unit testing is about covering your code in some probabilistic manner, such that <i>if something can go wrong, chances are you'll discover early</i>, and not in a live system where 1000 different variables are involved. Drive a unit testing stake through your systems, and, granted that it doesn't cover anything, you know for sure that the test should pass indefinitely...if not something is wrong, fix it, and elaborate the test at that point. Quite often a unit test serves simply to confirm that an externally identified bug is fixed ("STAY FIXED DAMMIT!") and regressions don't show up.
Unit testing also has ramifications for estimation, etc. This gets more into "agile" programming. In fact, maybe games are an ideal case for agile programming as you typically want to turn around features, balance changes, etc., very quickly, and often the testers are savvy enough to provide a quick, perhaps real time, feedback cycle.
Don't get too far into technical detail, it bores the non-programmers, unless you want to add a tag warning people to not waste their time listening to something they don't understand/care about.<!--QuoteEnd--></div><!--QuoteEEnd-->
I still enjoy them talking about whatever, even technical stuff, but it is kinda hard to stay interested in such verbose ramblings. The podcast could have probably been half the length if they had it layed out like Tomshardware does. *shrug* But even if it doesn't change, I will still listen to it cause it's my "crack" <img src="style_emoticons/<#EMO_DIR#>/biggrin-fix.gif" style="vertical-align:middle" emoid=":D" border="0" alt="biggrin-fix.gif" />
Or do the discussion over IRC and post the log! :O
In regards to the industry insight, I would say that there's definitely a lot of non-violent, more creative, especially musical games out on the horizon. Many of those you already mentioned. Ultimately, as I'm sure many understand the gaming industry serves up interactive fantasies and one of the biggest things, especially for predominantly male players is the expression of violence. (something that is obviously not socially acceptable). I don't believe though that it's necessarily violence for violence's sake, but rather a means to some other type of fantasy. Whether it's being a hero, stalwart soldier or other typically glorified status. At it's most simple level these involve physically dominating someone or something.
Of course, that's not always the case, and it's not the only desire of male gamers, but it's certainly a dominant one. Games like Guitar Hero / Rockband have done a great job of discovering other fantasies that are non-violent. The really great thing is that the desire to be a rock star goes beyond just males and guitar hero has definitely opened a lot of doors for females with any gaming interest.
Portal was mentioned, and I have to say I certainly enjoyed it but within the scope of other games, but I didn't see it as more than a big tech demo. I would definitely agree that it would be great if several developers moved away from mega blockbuster hits and created games with limited but innovative concepts, but I think to a degree a level of replayability is critical. In many ways the shortening of games has concerned me, and I'm not sure if that's a reflection of the fact that many games are still priced the same, but I think there's a legitimate complaint when a game has only 3 hours of playtime.
However these non-violent games are absolutely vital to the industry, and violence has definitely been a crutch for too long. Analysts and audience's favor of these games are evidence of this fact. Michael Pachter of Wedbush Morgan believes Guitar Hero to be one of the biggest games of the years, and even N'gai Croal of Newsweek (Levelup) thinks Desktop Tower Defense is one of the best games of the year. Penny Arcade gave a huge nod to Portal and Joystiq put Peggle in it's Top 10 games of the year.
Even within a limited scope of features, enabling players to create content, modify or at least experience content in different ways is critical. (something I know is important to the developers)
But in general, I agree, despite playing devil's advocate. By avoiding violence in games, developers have created some absolutely intriguing concepts. By doing so developers are able to create innovation that would likely have never been discovered within a typical shooter. (<a href="http://="http://www.nekogames.jp/mt/2008/01/cursor10.html" target="_blank">Cursor*10</a>, Little Big Planet, EchoChrome (Similar tech <a href="http://"http://="http://www.nhk.or.jp/digista/blog/works/20070517_fujiki/index.html" target="_blank">demo</a>), Portal) In the same stroke, I don't want to diminish violent games because I think they have a legitimate place within gaming although it's been horrendously over-represented. Some of my favorite games of the year have been Guitar Hero and Puzzle Quest.
In short: Non-Violent games are and should be an important focus to the gaming industry. However, they need to be held to the same standards as other games in terms of replayability and duration. Violent games are over-represented but still a legitimate genre. In addition, avoiding violence in games has definitely led developers to amazing concepts that would have likely never been discovered within a violent game.
/end messy reply
Why? If you really go into great detail above the model for NS, then the release time grows a lot. The change to source will effect a lot of gameplay with physics and guns. Since it has already been very long since an NS update you should release when you have NS:S + Dynamic infestation. Then you can hammer away new ideas that work in source and test them.
Why? If you really go into great detail above the model for NS, then the release time grows a lot. The change to source will effect a lot of gameplay with physics and guns. Since it has already been very long since an NS update you should release when you have NS:S + Dynamic infestation. Then you can hammer away new ideas that work in source and test them.<!--QuoteEnd--></div><!--QuoteEEnd-->
I think this is the direction they are heading in with the dynamic hive locations. I suppose there would be several potential rooms for hive locations, or marine bases for that purpose, and the start locations for both teams could be randomized.
edit: I didn't really address your last part. Developing a releasable client is a lot of work and detracts from the ultimate goal of the developers of releasing a standalone for-sale game. As lovely as a retuned NS would be on a new engine with limited features, it is neither wise, nor financially responsible and would likely detract from the final game due to time / financial constraints.
Unit Testing:
I understand your concern about Unit Testing. I am a good debugger though, so I usually debug my code after every sub routine (I haven't ever debugged on a project as large as NS 2 though <img src="style_emoticons/<#EMO_DIR#>/sad-fix.gif" style="vertical-align:middle" emoid=":(" border="0" alt="sad-fix.gif" /> I think that Unit tests would be a good idea for critical parts of the game code (THink render loop, and all the routines that handle all the sideways special cases(Stuff that isn't called very often) as these are where bugs can sneak in and not be found till way later, and are difficult to find) but don't spend time writing a unit test on every single line of code. A combination of Good algorithmic planning, unit testing, and just plain old debugging would produce the best results I believe.
Estimation:
When I write code I find it difficult to plan even 15 minutes ahead, I can only really give my self general Guide Lines. I have spend time on Loops that I thought would take a few seconds to snap the code in that took me hours, and I have implemented complex algoritihms in a only a few minutes before. When I deal with alot of code, I just give loose guidelines about what I am going to do, and way overestimate, and I usually finish just in time. I code better when I don't feel rushed so I get done alot fast. I don't know if that will work in the business world though, Mabie if you guys kept track of what you do with all your time you could figure out where you are not being very productive and improve, and also figure out how long it takes to get things done in code.
Less Shooting:
Are shooting games really the problem, or is the problem that all major games are shooting and violence games? This and your last point seem to be one in the same, not really differentiated.
Many FPS'ers went to mindless, b/c they were shifted into jobs after college that made them tired, imho, or they just don't have the time to invest in them to be uber.
Some prefer strategic.
I prefer complex, strategic FPS's. This is NS2, and I think many people would agree that this is your core audience. If NS2 is complex and strategic, then you have a slam dunk imho. Don't make it laggy, don't make simple (you can make the basic skills easy though <img src="style_emoticons/<#EMO_DIR#>/tounge.gif" style="vertical-align:middle" emoid=":p" border="0" alt="tounge.gif" />), don't make it overly ridiculous. "Keep it simple, stupid" rings true here, except for the fact that you want the elements to be simple but many elements combined making a complex gameplay system.
----
To da Devs:
The sooner you get the gameplay done, the sooner the -final- maps are going to be done. #$%# unit testing. You have a huge f'ing testing base. Even if the game is buggy to hell (but fixable), I'm sure you can get peeps to help you for free. GOGOGOGO FFS. Work 80 hours a week TESTING and CODING the f'ing SKILLS and movement and stuff. Slap it into a random CS map and go at it. Whatever, it doesn't even matter. Get the gameplay and skills roughly cut out and slap it into your orange-skinned boxy maps. Then 'take it up a notch', and get the maps done. Seriously, GOGOGO. You guys are poor. Necessity is the mother of invention, and heaven knows you guys need money...
You guys are getting -old-. Time is cruel. GO! Don't you want to be driving a BMW on a private yacht? (aka, yacht as big as aircraft carrier w/ BMW driving on it). Laughing about how you were poor? Cmon... Your legacy is -now-.
Oh, and BTW: <b>ZONK</b> -- Rad icon.
or something like that.
Anyway, I'm glad to hear that we'll be getting more screenies/artwork and such.
I'm assuming that modelling and textures and such are already into development, since you know what you can do with/put into the Source engine.
You seam to be talking about Unit Testing in terms of generally testing to find any bugs when in fact you should specificly ask testers to tests the current changes only. By looking for any bugs at all, testers might actually never tests some changes. I did game testing for EA (not for long but enough to see how game testings go) and it is done that way. You do not waste time just running around or submitting bugs (unless major ones) which have nothing to do with the current changes being tested. Testing for changes done should be done first, then do general testing for anything else (if times permits it).
Estimating in game MODing, is rather difficult to do. You have to take in account how long each tasks are going to be done by people not working full time on this. Each task assignees have their own life issues, own work and do not generally have a garanteed schedule to follow every week. So compared to a full team working 40 hours per week, it is much more difficult to do. Some people might postpone their tasks or drop out of the project leaving the task unassigned. So estimating a 40 ressource hours task that might actually take anywhere between 2 to 12 weeks is hard when your staff is spreaded around the world. You have mentioned "Your tiny team" meaning what ? You are lacking people ? By the way, I have seen the same 3times rule applied on several projects I have worked on.
Focus content, non violent games, serious constraints on game contents, smaller games, I'm not sure what was your point talking about this related to NS. Why did you needed to talk about these ?
Planning/resolution to release more medias, yes I agree more would be good for the community moral but carefully selected as to not reveal too much and with a note indicating this does not represent the final product.
"If you look at the last couple of years at titles that have sold well...". You mention that the Wii and DS games have sold better, that's because they are casual platforms as well as casual games. Using a pen or a remote is much more accessible than a mouse and keyboard. I think your talk about Wii games being very successful sort of misses the point that half of their success is their increased ability to reach a larger audience through the simplified and more intuitive interface.
Portal was successful because there were no time constraints on aiming, so it's immediately a non-twitch game. In any type of multiplayer shooter game (excluding perhaps co-operative, which is more similar to single-player in this respect) there will always be a need for the player to perform at their quickest in order to stand a chance against other players.
Next up bear in mind that Portal was 1 weapon, so the interface even on a keyboard and mouse was really simplified down to just movement, jump crouch and two portal buttons. Then the levels were designed to be achievable for most people. If more people can access the game through its interface you automatically have a larger amount of more casual players interested. Portal was so piss easy that without the dark, quirky humour and the final <span style='color:#000000;background:#000000'>behind the scenes twist on the</span> level design it really would have been an unfulfilling and unsatisfying experience for hardcore gamers. I was getting bored by the 13th level because of all the hand-holding.
Why am I saying this? I just want to make sure you don't try to cram too much 'casual' into NS2 believing you'll be able to cash in on the sort of widespread appeal is has given dedicated casual games over the past few years. The reason they've been popular is not because casual has been integrated in, but because they have been purpose-buit for the casual gamer. So save 'casual' for the next game, NS survives to this day because of a mod-downloading, mouse and keyboard-using hardcore audience.
Just for the record, 'hardcore' != 'pro'. Hardcore is how invested in games you are. E.g. Are you willing to spend considerable time on learning a control scheme for a game? Are you willing to spend considerable time on the completion of a game? If a multiplayer game - have you learnt or are you prepared to learn all the maps inside out? Are you reluctant to pick up your phone if in the middle of (a good part of) a game / would you tell someone to phone back later because you're 'busy'? It's the lengths to which you will go to play games. For NS this means learning or already being familiar with a mouse and keyboard control system (which is NOT intuitive for a beginner who doesn't know how to adjust sensitivity, however simple it may seem to you now), searching for and downloading updates to the game, managing their own resources, managing multiple weapons, learning multiple weapons and multiple classes with radically different movement styles, etc. etc.
Don't get too far into technical detail, it bores the non-programmers, unless you want to add a tag warning people to not waste their time listening to something they don't understand/care about.<!--QuoteEnd--></div><!--QuoteEEnd-->
I like it, a lot.
Keep it up.
Keep it up.<!--QuoteEnd--></div><!--QuoteEEnd-->
Boring podcast. Or is this your way of saying "we haven't worked on NS2 so we aint got nothing to talk about"?
I don't want a buggy game release, do you? With comments like these, sometimes I wonder if gamers deserve what we get when buy something that is still Beta rather than Gold. Try to have a little understanding of the process but then again, that would mean actually listening, rather than just hearing, the podcasts and blogs and posts and videos and concept art...
They are doing everything short of inviting us all over to look over their shoulder every second of every day. Give em a break, you can always go experience the other side of game development with some announced game development in a big producer bought developer's unresponsive forums.
Its fairly hard to listen to amateurish podcasts, especially when they contain almost nothing interesting.
<!--quoteo--><div class='quotetop'>QUOTE</div><div class='quotemain'><!--quotec-->They are doing everything short of inviting us all over to look over their shoulder every second of every day.<!--QuoteEnd--></div><!--QuoteEEnd-->
That wouldn't be very exciting, unless you're a programmer I'm sure.