So, rather randomly, I was discussing with @PetMac about the merits of a particular engine being split up into multiple libraries. We’d suffered from the other extreme: one gigantic project that contained everything (and had several minute link times as a result). I opined that it was, by and large, a good thing, even if it was inevitable that a lot of time be spent splitting off chunks of functionality and pushing them up or down the hierarchy of libraries to avoid circular dependencies. The alternative being of course that libraries end up tightly coupled, and even though they are two separate units, they are effectively indivisible. That is, not quite spaghetti code, but certainly a tightly snarled up ball of functionality that it would take many man-hours to pull apart. And as soon as libraries start to stick together like that, the rot sets in quickly; one  reference turns to dozens, and even if it might have been possible to separate them again before, it isn’t feasible any longer. I think (and he can correct me if I’m misstating his opinion) that Pete agreed on that front.

Why is that relevant to object orientation? Well, because the means by which you most commonly ‘fix’ a circular dependency is to abstract one side so that it becomes unaware of the other. So your character system knows about your audio system because characters carry around audio references; but instead of your audio system being aware of characters (so that they can, say, position sounds), you rewrite your audio system in terms of ‘character-like things’. Or more cleanly, ‘things with position’. Because that’s all the audio system really needs to know about. In an object oriented system, you’d use an interface definition to say that characters can satisfy the definition of a ‘thing with position’; of course that’s not the only way to achieve the same goal, but it’s certainly a nice easy way to do it. What’s important is that the library has a nice clean interface, that expresses exactly what it needs, in a succinct way. Ideally, it is also written without explicit knowledge of any other libraries. Having a clean and clear interface is what helps you keep that lovely de-coupled code, and lets you re-use it elsewhere.

Personally, I’ve never had a problem with using interfaces or other object-oriented mechanisms. But recently Pete has been trying to persuade me that object orientation is the dark side, and that our code would be much better if we only thought about things in terms of data transforms. There’s been a lot of eminently sensible stuff written on it, including stuff by @noel_llopis over on his blog, and by @TonyAlbrecht in a talk for GCAP. I’ve read their pieces, and don’t really disagree with most of it. If I have an issue at all, it is that their concerns about OO (and C++ specifically) primarily relate to performance, and when I’m coding, performance is only one factor; an equally pressing factor is how easy the code is to write and maintain.

Here’s the thing though; object-orientation can be really bad for performance, sure. And used badly, it can be really bad for design as well. C++ has a whole lot of cruft that means that expressing the code design you want, without locking yourself into bad assumptions, is hard. Not impossible, just hard. But there are a whole lot of code design needs I have which are very hard to satisfy without the basic features of C++. Interfaces and polymorphism straight off, and probably more. Really though, my problem lies with anyone that tells me that we should all go back to using C instead of C++, because it will avoid all of that bad stuff. Well, sure. I could go back to writing in assembly and never worry about variable aliasing as well, but I’m not going to. I’ll use C-style interfaces when they help, and C++ when they help, thank you very much. Whatever gets me the simplest, cleanest, most maintainable interface, that still lets me do the work.

I have no doubt that using C-style library interfaces would avoid a lot of unnecessary object-orientation. @PetMac is trying to persuade me though that a C-style interface is just plain better, and not only that, but that the inputs and outputs should only be structures defined in the library interface. So an audio transform would be ProcessAudioEmitters, and if you want to process a bunch of positional audio emitters, one for each character, you have to marshal an array of audio emitter structures, and copy the position from your character into its audio emitter. Which doesn’t sound so terrible, if it leads to a cleaner interface. I’d probably be fine with that. At a simple level, for core systems like audio or rendering, where the inputs and outputs are clear and rarely change, I think that would probably work well. Best of all it makes the audio library completely independent – it knows nothing of the things that it’s working with, except the data the other systems choose to feed it.

My problem comes when I consider how I would make that approach scale to all the other systems I need to build. The example I posed to Pete was one of an AI system. To use Pete’s preferred paradigm, and think of data transforms, the AI system would be a DecideWhatToDo transform. Great. What are the inputs and outputs? Well, that depends on the AI. One type of AI might want just a list of character types and positions. Another might want to know about the environment as well. Smarter AI might want character positions, damage levels, movement histories, factional allegiances; as well as the ability to co-operate with other characters. The outputs of the AI are just as bad – they can affect everything from the desired target position, to queuing animations, in fact pretty much anything a character can do might be an output of the AI.

I would describe Pete’s system as a ‘push’ system. Everything the system needs has to be fed to it explicitly, in terms it can understand. The problem with push systems though is that when the number of inputs goes up, the amount of code you have to maintain just for the marshalling of the push grows with it. You find yourself implementing the same code several times: you add the notion of damage to the character, then you have to add the ability to marshal the damage information into a structure the AI system would understand, then you have to add the notion of damage to every single AI interface that wants to know about damage. And in a system with dozens of different sorts of AI, that might be a lot of interfaces.

To me that smells wrong. It means that you’re baking implementation details (like AI ‘X’ cares about damage) into the interface. Conversely, the ‘pull’ system stays relatively clean. You simply pass the list of characters to the AI, or the environment, and allow the AI system to ask the character interface for only the data it needs. Characters might provide a vast array of query-able state, and the AI can pick and choose what it asks for. Of course this comes with a down side. The AI system now has to have knowledge of the character system (or at least, provide abstractions which the character system can fulfil). It’s no longer truly independent. The performance impact of lugging over the entire character object, when perhaps you only want to access a few small parts of it, is very real. But in terms of the ability to write clean code, without a massive amount of interface book-keeping, it’s a big win. That said, I’m open to persuasion. If someone can describe to me how they would write a succinct AI library interface in a C-style, for a few dozen varied and sophisticated character AI, without giving the AI library knowledge of the character interfaces, I’d be happy to change my point of view.

There will be those who say that if your structures are that complex, you’ve already done something wrong. That’s very idealistic thinking. The simple fact is that we are often writing fantastically complex simulations. Sometimes the ‘pure’ systems that you’d need to build to support the level of complexity the design calls for are just far more effort than the benefits they would give. When it comes down to it, we need to write code effectively more than anything else. We need to be able to code quickly, cleanly, and flexibly; especially when the game design is changing quickly as well. It’s of no benefit at all to spend months building a fantastically clean engine to support one game design, only to find that in the time it took you to build it, design changes have rendered it obsolete.

To sum up, because I’ve gone on for a long time: the one thing I like less than being accused of ‘drinking the OO kool-aid’, is the notion that there’s only one right way to do things. As a coder, you should be constantly and critically evaluating all your systems and interfaces. Sometimes a data oriented approach is better: consider the purity of the interface and the vastly improved ability to parallelise and minimise your memory accesses. Other times the structures and inter-dependencies are simply too complex, and object orientation is the most effective tool at keeping your code clean and versatile. I won’t claim to always get it right (as Pete and Tim have both at various times pointed out, I tend to over-structure my code), but I’d hope I always aim for clean code as best I can.

There are two main areas where amateur game development happens. The first is the Indie scene, which encompasses most forms of game development done by a single developer. This can mean cheap action games on Steam, cult hits like Minecraft and Braid, flash developers working on Newgrounds or mobile and smart phone developers. These games are often simple, 2D, and tend to be creative or puzzle type games with accessible graphics.

The second area is in the modding scene. This consists of unofficial add ons, changes and modifications to the games called mods. The modding scene has existed since PC gaming began. It has had wavering popularity but game developers such as Epic with the Unreal series still herald their games “moddability” as a selling point.

The interesting thing about these two scenes is their complete isolation from each other. It sometimes even goes as far as hostility. Modders can see indie developers as stuck-up and pretentious – working on mediocre puzzle platformers with pixel graphics. Indie developers can see modders as simple fan boys, making Call of Duty machinima videos set to “let the bodies hit the floor” and yet more tedious realism mods.

On course, in reality, neither of these are exactly true. My background is the modding scene, so perhaps I have a natural bias toward it; but I’ve been lingering in the indie gaming scene and increasingly I’m seeing the void between the cultures as doing increasing damage.

Artists and Programmers

One of the major differences is that the indie scene is largely constituted of programmings – often contracting out artwork for games. The modding scene, on the other hand, has an abundance of artists, across all skill levels, willing to get their hands dirty.

It isn’t really the obvious benefit that could be gained by a more balance skill set that bothers me most. What I find most annoying is the disjointed and boring artistic direction in both scenes. In the modding scene I don’t want to see another Star Wars mod, another Lord of the Rings mod, another Graphical Enhancement mod – and this is coming from someone who loves all these things like no one else.

In a similar way, in the indie scene I’m so bored of pixel graphics, cartoon graphics, crappy looking 3d games, terrible assets.

In both scenes we have very uninspired and boring artistic direction, with poor technical features due to the tiny number of graphical programmers working with, or being, artists.

More collaboration, sharing of ideas and passions, would be amazing great!

Individuals and Teams

For the modding scene the de-facto standard is to bring a team together to put our your mod. This is seen as essential on anything large at all, and timescales are assumed to be as short as possible. Indie development is the opposite.

I’m not going to discuss if team development or individual development is better or worse. That is one for another time. They clearly have their strengths and weaknesses. Team development is all but useless without someone in charge who is organised and knows what they are doing and individual development is powerful but suffers from burnout.

I think both sides just need to view (or experience) the benefits of the other. Indie developers seem unkeen to share their vision with untrusting individuals, missing out the benefits of a shared workload and new and interesting contributions. Modders set their sights too high, assuming the team will carry the weight, just to fall at the unreliability of others.

More importantly though, I think there needs to be more communication from those with successful and released modding projects and indie games – giving insights into what is needed to finish a product.

Money

For many indie developers making games is a way to make their living and the idea that they are developing for money is a no-brainer. modding on the other hand is almost always free and has a feel about it akin to the open source community.

The result of this is that modding can be more fun – you don’t feel the pressure, and legal complications are greatly reduced. The problem of course is that there is a huge uproar when someone wants to charge for their mod – they are often benefiting off much previous development made by other teams in tools, tutorials and tips. There are strong feelings of betrayal and greediness.

The ideal situation would be that modding remains fun, with reduced legal issues, but developers are more motivated due to potential of making money. The modding community needs to have a serious think about this if it wants to progress, and there are lessons to be learnt from indie developers. I don’t want to make games if it isn’t fun, but I, like everyone else, am sick of failed projects.

Unity

So this is my modest proposal: A community for indie developers and modders to get together and find ways of working on projects that everyone is excited about.

It’s often the case that in the industry,  people will do research on particular software engineering methodology, or a team will publish a post-mortem in which they talk about a particular style of working and how successful it was for them. And the discussions following those posts will usually descend into an argument, with different people chiming in on how they tried that methodology, and it was rubbish, or how their own methodology (or ideal methodology) is better.

This sort of debate annoys me, because it’s always couched in absolutes. In software engineering there are no absolutes. So I felt I had to respond when someone declared, without much in the way of context:

No they shouldn’t. At least, not unless your team ethos supports it.

Just like all of these statements about how things should or shouldn’t be done, there is a whole bunch of context needed before you can say whether or not a strategy is successful or not. You can’t just look at those stats and say “TDD is the way to go”, or “asserts should be everywhere and always on”.

Every last one of these tenets of development requires a particular way of working before it is viable and/or usable. Asserts are great, as long as the team ethos is to never (or nearly never) allow them into a live build. What do you think you get if you just turn asserts on everywhere, when the build is riddled with conditions which aren’t show-stoppers, but which result in asserts? You get designers and artists that can’t use the build any more, and everyone gets pissed off.

Similarly, what do you get if you have a team which is notionally doing TDD, but in fact many of the developers aren’t structuring their code to support complete tests, or have incomplete test coverage where it counts? You get slower development, without a great decrease in the number of defects, and now you’ve got less time to fix them when it comes time to ship.

People should stop looking for one-stop, quick fix solutions to the problems which all development teams have. Every last one of these solutions will a) make things worse if applied in a half arsed way, and b) stem from an underlying mentality which is “what can we do to improve maintainability, increase coder efficiency, and smooth out problems in our day to day development?”

Sure, read the stats, read other people’s techniques, but for the love of Mike, don’t try to just stamp a particular technique on your development team and expect it to improve your lot. Instead, take a long hard look at your day-to-day process, identify the root of the problems that your team actually has, and take small incremental steps to fix those problems. Repeat ad-infinitum (or until you ship).

More than anything though, make sure any steps you do take work with the team you currently have, not with some ideal team that you’ve read about. If you find yourself applying a solution which will only work if everyone has a certain mentality or set of skills, then you’d better make damned sure that your team has those things before you try to apply the fix.

Finally gotten around to putting VS2010 on this machine, and this time around I’m breaking with tradition and simply not putting MSDN on there, at all. It used to be a no-brainer, put the reference libraries on as you’re going to be looking stuff up all the time. But these days it was always more an exercise in frustration than a useful tool.

Many topics are just “not there”. Huge swathes of really basic stuff are just missing (basic date formatting string specifiers – that’s pretty low level!), so that when you navigate to them it tells you the page is missing. Go online to the MSDN reference there, and you’ll find the page, just not in the locally installed copy. I thought to begin with it was just because I’d installed it badly, but even from a clean install it was still just not there. I’ve since concluded that it must be the Express versions are just subsets of the full documentation, to keep the downloads small. It’s certainly not a functionality split – like they’re only putting in help topics for things in the Express editions – because the Express editions are really quite close to fully functional. No, this is stuff that’s core to .NET and the language.

So, since I’m having to fall back on searching the internet anyway, I figure I might as well have my hard disk space back. The online resources available now are fantastic anyway, and it’s rare that I’m not connected when developing. Most often it’s the online MSDN references that show up first in the search, so in the end it’s much of a muchness – except I don’t have to use the horrible HTML help interface which has been getting steadily worse with every revision of Visual Studio.

I guess this is just another nail in the coffin of the disconnected computer: so many things now expect/assume/require you to be connected to the Internet. Which wouldn’t be so bad, but even with 3G connectivity, a network connection while on the move still isn’t something that can be taken for granted. But I’ll stop grouching about it like an old man, and go with the flow…

Picked up on when reading through some old posts on the sweng_gamedev list, and had to be shared.

I must admit, I’m a bit of a CruiseControl.NET fan. It drives most of our automation systems, and provides us a backbone from which we can hang many different systems. I use it both here at the Company, and when I’m doing tools consulting with other teams. That said, it’s not without it’s flaws and limitations; not least of which is the lack of documentation on custom plug-ins.

There are a wide variety of pre-built plug-ins already available, for source control systems like Perforce and Subversion, and build systems like Nant and MSBuild. These usually work pretty well straight out of the box, if you’re building a vanilla continuous integration server. That is, every change made to source control results in a build of some sort. I’m not going to dwell on why that’s a good idea (it is), I’m only going to say that there are times when you need something different. There are inelegant ways around this, but in truth CC.NET has all of the customisability to let you do this properly within the system – by defining your own plug-ins.

Reading the documentation on CC.NET, you get only a single page of documentation on custom plug-ins, and that page is pretty spotty at best. It shows you a “hello world” plug-in, with no clues as to what information you get into the plug-in, when your plug-in’s methods will be called, or how you’re supposed to pass either status information or logging back to CC.NET itself. In short, it’s pretty useless, other than to highlight that the facility is there, and give you enough of a pointer to get you into the code and poking around.

Sure enough, with Auto-complete on, you find that there are lots and lots of information passed to you, and by debugging and breakpointing inside your plug-in, you can get a feel for when your code is called. Making CC.NET load and unload your plug-in is refreshingly easy and fairly robust, and once you figure out how to pass parameters to your plug-in by specifying them in the config file, you can see the possibilities open up. More importantly than the sample building plug-in though is that, through a bit of digging, you find you can also define your own custom source control plug-ins, and with the combination of those two things, you can do pretty much anything you want.

In one particular situation, I’ve been working with a Perforce (p4) source control system, and a rather black-box build system for the game itself. Rather than wanting a build made of every single commit to the p4 repository, I needed it to make only particular builds – those marked as verified by the developer’s internal test team. This is a pretty common situation when you’ve got an automated build system – you have a raw source control system that lies underneath, which operates at the level of atomic commits. But above that is a logical structure, which only people understand – that operates at the level of ‘nightly builds’ and milestones. You have custom logic which you can apply to the system, using some basic rules. So in essence you have a virtual source control system, built on top of the raw version. By writing a custom source control plug-in for CC.NET, you can expose an interface to your automated build server, so that it recognises when something new is available from that virtual source control, and only builds exactly what you want.

CC.NET offers great flexibility, and those people who develop it, know all about that flexibility. But information on what you can do, and how, is rarer than hens teeth. So over the next few weeks, I’m going to write up and publish here some examples of real-world plug-ins that I’ve written. That should hopefully give readers enough context to go off and write their own plug-ins, to suit their own needs.

Disclaimer: all of the points made here refer to CruiseControl.NET 1.4.x, not the later versions of the system. There are some big and eagerly awaited advancements in the newly released 1.5 version, that many people like myself will have to avoid for now, until it’s bedded in.

Amongst various things I had to sort out today, I was asked to write out a blurb for a potential client about improving build processes and automating/scripting things in the development pipe-line. It’s a subject I get quite passionate about, because unlike so many things in games development, it’s a nice task to do. There are clear, quantifiable goals (“make creating a build a one click process”, “speed up turn-around times for artists by 50%”), and usually plenty of options about how to get there. It is also a nice, self-contained task that you can just wade into and make progress on, unlike for example gameplay coding, where you can often get blocked on feedback from the creative team, having to rework things and so on.

I think that’s possibly why I like to spend time on improving our pipe-line at the weekends or in my off-time; even though I could spend more time on the big pile of client work that needs done, I find myself tackling little bits of our own pipe-line because I know it’s a task I can get done without any other input.

On that note, with some more collaboration with the developers on some little niggles, we finally switched our creaky old makefile based system over to using JamPlus properly. Both build processes still run side-by-side, but the JamPlus version has a fraction of the number of lines in the makefile, runs much faster doing dependency checking etc., and in general is much cleaner and will be more maintainable going forward. I’ll have to walk the guys through what’s there so they can maintain it too, but after that I should be able to scrap the makefiles altogether.

Next step is the art/audio asset to platform binary conversion process, and this is why I really wanted to switch over to JamPlus. Our previous art pipeline would always rebuild platform assets, even if the source assets hadn’t changed. That was fine early on, when all of our tools ran lightning fast and we had few source assets, but very quickly it grinds when you introduce slow tools (such as our font encoding tool that does smart packing of glyphs and colour conversion), or many assets. Also the build scripts which make those assets are all Lua based, and so we have different technology for building code than for building art and audio. I’m pretty hopeful that we can make JamPlus fulfill both functions, and in the process get fast dependency checking for our art assets so that only the assets that have changed get rebuilt. But for that I’ll need a free day, and those are few and far between right now.

There are few things worse as a programmer than to know that you shouldn’t do something, know all the reasons why you shouldn’t, but yet you do it anyway. You tell yourself that it’s more important to get things in quickly, that the code is only preliminary or temporary. You tell yourself that it’s better to get things in so you can see whether or not the whole approach will work, and that taking time on the details now would be premature optimisation.

Of course you know, deep down, that the code that is supposed to be temporary will stay, because it works. Any bad things you put in there will be left with a @todo comment next to them, and shuffled down the priority list. Which might seem like a good thing on the run-up to a deadline, but is so many times more irritating when it comes back to bite you later on.

In this particular case, the standing rule we have at the Company about programming for a fixed memory environment, even on PC, was the one I was ignoring. We were initially developing for PC, and memory handling there can be handed off to the operating system heap management and forgotten about. On every other platform though (especially PSP for which we are now porting things), you don’t have that luxury – every drop of memory has to come from the single small fixed arena, any allocations from outside of that will just fail.

So I had to spend a couple of days, going through all the allocations in the code and plumbing through proper fixed heaps so that it was clear where all the memory was coming from. Two days, and probably that in itself wiped out any gains we made in the original PC development by working fast and loose.

In summary then, no lying to yourself when it comes to weighing up pros and cons of doing something in a way that you know isn’t going to be good enough in the long term. Oh, and work to a fixed memory environment. Even on the PC. It will make your code tighter and faster, give you a clear idea of what your memory footprint is, and help reign in the inevitable problems when your memory usage runs away with you and you have to spend time tracking down who is wasting memory and where.

Virtual memory hides a lot of sins from you as a developer, but it is a crutch which you can all too easily become addicted too. It doesn’t take much to engineer systems to avoid memory fragmentation and waste, but re-engineering them once they are embedded is difficult and problematic.

So I must admit to being a bit of a luddite when it comes to embracing the new languages and technologies available for developers now. Partly this is because I’ve read (and heartily agreed with) No Silver Bullet by Fred Brooks, and partly it is because I’m much more comfortable knowing exactly what is going on ‘under the hood’ when I write software.

That being said, having good tools is a vital part of games development, and to write good tools you have to build good user interfaces on top of functional software. No amount of clean, efficient and well structured code is going to get you past that final hurdle, which is to interact with the user. I have spent too much time on too many different projects faffing around with inadequate UI libraries to want to spend any more on it now. I would say I am comfortable with MFC based development, but I would never claim that it was easy or pleasant.

So when I keep hearing other developers evangelising the merits of tools based on managed code (C#, etc.) and the .NET platform. Apparently it should take the pain out of making tools, and user interfaces, and should let me concentrate on the important things instead. Well, that was enough to tempt me in, and to give it a try.

The thing is though, our engine and game code is all based on C++, simple and clean, and we’re not going to change that (no matter how much XNA and Microsoft try to tempt us otherwise). So any tools we build have to be able to leverage all that pre-written code, and play nice with the other parts of our engine. So we needed a way to make the managed tools work with the un-managed engine, and that was where my headaches began.

Straight out of the gate, building a tool application with Visual Studio 2005 was simple and easy, and it took less than 5 minutes to have a skeleton UI that had all the right hooks for exercising some of the engine functionality needed to pre-process our assets. But then I had to figure out how to link those hooks to our pre-existing code, and that wasn’t nearly so simple. The problem was this – a C++/CLI based application (i.e. our tool UI) needs to jump through a few hoops to talk to native (C++) code. The documentation rabbits on about mixed and pure assemblies, DLL building, COM linking and a whole ream of pages about how to build a managed application. All of which is total overkill for what we needed – a simple wrapper layer between the managed UI application, and the native core code.

Now that I’ve found out how, it’s not as hard as I thought; as I work through the details, I’m going to note them down and post them here, because it was immensely frustrating to continually search for tutorials and references (I gave up on the MSDN documentation), only to find lots of people talking about how simple it was, but no-one bothering to let on how it was done.

Anyway, in lieu of a later, better post, here is what I have so far:

• Make a library DLL, making sure that it has managed (/clr) support enabled. This will form the wrapper layer
• The library DLL can statically link to the native libraries you have.
• Build a wrapper class which uses pointers to your native classes to route commands/requests to the native code. Make sure this is a managed class (it should take the form “public ref class WrapperClass” if you’re using C++/CLI)
• NB: You will have to follow the managed rules about not having native value types in your wrapper classes, but you are allowed to have pointers to native types and use new/delete as normal
• In your fully managed UI application, use the Reference system in the Common Properties for the project to add a reference to the wrapper library. This will automagically allow use of your wrapper layer classes, no need for headers or static linkage. [this is the one annoying step that really wasn't clear from the documentation]

Anyway, a better picture should emerge from this experimentation, and I’m hopeful that once the basic pattern for managed/unmanaged tools emerges, that we’ll be massively more productive and be able to build up a nice tool-set using this new technology.