Ubuntu 14.04 upgrade

Posted in Technical Guidance on August 21st, 2014 by MrCranky

After the kerfuffle with Heartbleed earlier in the year, and finding out our server installation was way out of date, I resolved to keep it more current. That meant upgrading from 12.04 to 14.04. Not as painless as previous upgrades sadly, and left me with three notable problems. I’m posting my notes here in case they’re helpful to anyone else upgrading. Basically our server box acts as a DHCP server, file server (using Samba) and gateway for the internal network, as well as hosting a couple of websites which we use both internally and externally. After upgrading, I noted:

  1. Two of the hosted websites were no longer working: they were giving 404 not found messages.
  2. A persistent message being posted at startup and various points during shell sessions: “no talloc stackframe at ../source3/param/loadparm.c:4864, leaking memory”
  3. DHCP server stopped responding properly the day after the upgrade

I’d had to merge a few configuration files, of which Samba and dhcpd were one, so my initial thought was that I’d botched the merge. However there wasn’t anything obvious in the merge results that would explain why. Anyway, issue by issue:

No talloc stackframe

This one was the most easily resolved. This post points the finger at libpam-smbpass, a Samba module which seems to have an outstanding bug. Fine, it’s not functionality we rely on, so uninstalling the module makes the problem go away:

sudo apt-get remove libpam-smbpass

DHCP server stopped responding properly

This one didn’t bite me until mid-way through I was trying to diagnose the Apache issues, my DHCP lease ran out and suddenly the laptop I was remoting in to the server was without network. Super annoying. Setting the IP address / DNS of the laptop manually got me network connectivity to search for solutions, otherwise it would have been guessing blindly from the server terminal (which doesn’t have web browsing).

While there wasn’t anything hinting at a DHCP problem in the logs, I noted at server reboot time a line along the lines of the following:

Apparmor parser error for /etc/apparmor.d/usr.sbin.dhcpd at line 69 Could not open /etc/apparmor.d/dhcpd.d

I couldn’t find that line anywhere in my logs, but I probably wasn’t looking in the right place. The configuration file that is complaining is basically trying to #include the dhcpd.d subfolder, and failing. Still, it suggested some sort of permissions or configuration problem with AppArmor and dhcpd. Oddly though, DHCP had been working after the upgrade the afternoon before, and I could see successful DHCP negotiation going on from this morning, but it all ceased an hour or two before my DHCP lease expired. All of my searches were throwing up results for the package isc-dhcp-server though, whereas I was pretty sure the package was dhcp3-server. On checking, isc-dhcp-server was not installed. Installing it:

apt-get install isc-dhcp-server

Lo and behold, DHCP was functional again, using our already existing configuration. So, I’m guessing, the packages on our legacy machine (upgraded using do-release-upgrade from 10.10) aren’t properly handled by the release upgrade procedure, and were left with folder permissions set incorrectly; which was fixed by installing the correct DHCP server package.

Apache website issues

Ubuntu 14.04 brings with it a fairly major upgrade from Apache 2.2 to 2.4. While the web server was still functional, and I could access pages resting directly under the DocumentRoot, our two sites set up using Alias directives were no longer accessible. Both returned 404 errors. Using a symbolic link in the filesystem under the DocumentRoot would allow them to be accessed, but that wouldn’t allow us to enable/disable the site at will. While there are changes to the permissions system in 2.4, we don’t use those with our sites. So all very odd.

Our setup was very simple: each site had a configuration file that only contained a single Alias line, remapping the appropriate site folder to the folder on the local disk. Further experimentation showed that we could shift the same Alias line into the default site configuration, and have it work. It gave a 403 Forbidden error, but not a 404 any more. Adding an appropriate Directory element with a “Require all granted” directive inside fixes the 403. So presumably the default permissions for an aliased directory have changed to deny by default instead of grant.

So from that I can only conclude that Apache 2.2 was more forgiving of having Alias directives standing alone in their own site .conf files, for whatever reason. I’m probably missing some nuance of the setup as to why it worked before. Rather than spend too much time figuring it out, I’m going to just go with having the sites as a part of the main site instead of as sites on their own.

Visualisation

Posted in Tales from the grind-stone on August 14th, 2014 by MrCranky

One of the most interesting things about the field in which I work is the sheer range of topics I get to work on. Not just on different platforms or in different languages, but the actual subject matter of the projects. The project I’ve just completed, again working with Eutechnyx, manage to exercise parts of my skillset that I haven’t had to use for a while. Sometimes, working in games, you find yourself working on ostensibly the same problems. Different skin, different IPs, different engine, but really it’s the same fundamental concepts and functions that you’re re-implementing in a new project.

So when you get challenging problems, it’s really quite refreshing, because you have to go back to first principles of analysis and visualisation techniques to solve them. Where you’re presented with an overwhelming amount of information, and you have to design and implement something which wrangles that raw data into something coherent. Where you have to figure out a way of presenting that information in a way that is visually compelling and conveys that information in a useful, concise form. They are fundamentally hard problems, sometimes solvable, sometimes not, and finding the right solutions, if they exist at all, takes a level of concentration somewhat higher than the usual required to bring our games to life.

This is the state of mind I’ve been in for the last few months. For confidentiality reasons I can’t say anything very much about the project itself (it’s not the lovely visualisation linked above of course), but I wanted to talk about the satisfying nature of developing visualisations in general.

Often when you’re developing code, your debugging tools are limited to logging and step-by-step debugging. But for complex data sets, especially those dealing with spatial data, it’s far more useful to display that data visually. The same is true for end users – a sequence of numbers means very little; a dumped spreadsheet of data, while accurate, doesn’t let you see shapes or patterns. Turn those numbers into 2D graphs, and you can discern patterns, noise and trends. But that still may not be enough. You can make a line graph of each component of a 3D position that changes over time, but in 2D those graphs make little sense. Allow it to be viewed in true 3D space however, and suddenly you can see the shapes. But a line covering every point that 3D position visited tells you nothing about how quickly it moved between those points. So you introduce animation over time, or colour, and suddenly the data makes sense. When writing processing code, a visual representation of the outputs lets you pick out flaws that would otherwise be hidden. Spikes where there should be smoothness, patterns where there should be only random noise, correlations that hint at relationships you didn’t realise existed.

Of course it isn’t as simple as layering in more and more information. With too much visual clutter, it becomes impossible to discern any useful patterns from the data. So knowledge of what data is important becomes vital; ways of filtering information to show only what is relevant allow you to show information where it is needed, and hide it when it is not.

This, again, is one of the reasons why I’m so enthused about VR development. Being able to visualise spatial data is very dependent on good camera work – you have to be able to look around the visualisation. If the camera is out of your control, then you’re utterly reliant on the generated camera. If that is poor, then you might as well have a 2D visualisation of the data, because you need to have some useful spatial context to be able to process what you’re actually seeing. It’s for that reason that many optical illusions that rely on a particular perspective are defeated by moving your viewpoint; if you viewed the same illusion from a static perspective, you wouldn’t be able to tell it was an illusion at all.

Floating cube

So the introduction of VR allows us to get great flexibility of viewpoint, while not requiring the user to learn a cryptic set of control inputs to gain full 3D control over their viewpoint. That opens up great possibilities for exploring even more complex datasets and 3D structures. We’re living in an age where technology has advanced massively and the integration of computing into our everyday lives has resulted in masses of new information becoming available, in overwhelming amounts. Being able to visualise and process that information is the first step to being able to make use of it, and we’ll need new tools and tricks to do that.

Scottish Games Network

Posted in Random Stuff on December 4th, 2013 by MrCranky

What I didn’t take the time to blog about last month was my attendance at the Scottish Games Network launch event held in Edinburgh. I’ve been broadly supportive of the idea of making SGN official since Brian announced it in October. I’m sure it must be a little disconcerting for him to think that simply declaring that SGN is now the official games industry trade body for Scotland is enough to make it happen, but it’s not really as simple as that. All a trade body really needs to be taken seriously is the support of the companies it purports to represent. Officially or not, I think SGN has been doing a pretty good job of representing our interests, without being asked, or paid. The proof is in the pudding as they say, and so we will judge it on the work it does.

What I’ve said, both here on the company blog and in person when I’m out and about amongst the rest of the industry, is that communication is key. As an industry we’re not generally competing with each other. We gain a lot by collaborating, sharing knowledge, ideas and inspiration. Many if not most of the client relationships we have were started by going out, seeing what the rest of the industry was doing, and letting them know who we are and what we do. Without a focus point, to do that we’d all have to be contacting each other, and that is time-consuming and not very practical. The simple fact is: locality is important. I know what many of the game developers in Edinburgh are up to, because I meet them. Either at @GameDevEd, or at other industry events around town. I know what some of the developers in Dundee are up to, but generally only because I’m in touch with individuals at various studios up there and we chat regularly. I’d love to know more about what’s going on up there, as it’s very easy for me to lose touch, especially when we or they get busy.

So for me there’s a definite niche to be filled, that of a locus for information, someone or something capable of routing information around. That’s especially true for those outside the industry. I’m sure there are many, many Scottish organisations that are interested in interactive digital entertainment, with ideas and projects just waiting to be made. I don’t know who they are. I’d love to talk to them though. They don’t know who we are. Few outside the industry do. It’s just as infeasible for them to cold-contact every games developer in the country as it is for me to cold-contact random organisations to offer our services. But if there were a central point of information, obvious and high profile, those two organisations can be connected together. They can go to that central provider and say “we’ve got a budget and an idea, but we don’t know who can help us,” and be told “Well, Black Company makes games about that size, or Proper, or Storm Cloud. Here are their details, I’ll introduce you.”

More importantly I feel that the government bodies here in Scotland, the Parliament, Creative Scotland, the many media departments, could all be engaging with the creative digital interactive media talent here in Scotland much more, if they had a reliable conduit into the industry. Scotland is a country exploding at the seams with culture and history, and I feel it’s crying out to be exploited in interactive media. I’ve long chafed at the need to globalise and homogenise our games to appeal to the world-wide audience. We should be embracing our heritage and making games that tap into our local culture. Such as Beeswing, a lovely little project, set in rural Scotland.

Beeswing

It’s fantastic that the Kickstarter for this was successfully funded (with a few of my pounds as well). Instinctively though I looked at it and thought – this is the sort of stuff that the Scottish government should be actively encouraging. I believe they would too, if they had a practical way of engaging with the Scottish games development community to start these discussions. So again, a central focal point can enable those two sides to get together and make amazing things happen.

Visibility is one of the main reasons we are members of TIGA, and is why we’ll be happy to become paying members of the Scottish Games Network as well. Not because one is better than the other, but because they serve different localities. The TIGA folks are lovely, and very efficient. They give us a presence in Westminster that I feel is important. They cover the UK industry and beyond, and that is also an area in which we are very interested. We don’t cut our business dealings off at Hadrian’s wall. But like it or not, Black Company isn’t well placed to attend events in London, and so a sister organisation that can provide even more coverage in Scotland seems like a very good idea to me.

Winter warming

Posted in Tales from the grind-stone on November 20th, 2013 by MrCranky

You can tell when the November cold snap comes around to Edinburgh. Here in the office, it means that the winter charcoal hand-warmer comes out…

Winter hand-warmer

Virtual Reality

Posted in Tales from the grind-stone on November 13th, 2013 by MrCranky

So, it seems in a staggering degree of competence, the three lengthy rants on crunch I wrote and queued for posting didn’t in fact get posted, and just sat around in WordPress till now. So much for my good intentions on blog posting. They’re up now though, so please do scroll back and take a look. Those who know me know I’ve a bee in my bonnet about both crunch and the underlying viability of games development, so those thoughts have been bubbling up for a while.

Since NASCAR: Redline shipped I’ve been busy with various different things that I’ve been neglecting, not least of which is looking ahead to see where to go next. While I’m still positive about smartphone development, I’m conscious now that the market is rather saturated, and I feel like we have missed the window where a small team can do great things and get noticed for it. That’s entirely my fault, focusing too much on work-for-hire development and neglecting our own projects, but I stand by the choices good or bad.

Photo 11-11-2013 16 31 19Recently though I’ve been investigating the resurgence of Virtual Reality tech, specifically the Oculus Rift project and castAR. Both are using modern technology to revisit the old holy grail of immersive digital realities, but unlike previous attempts at it, these project really feel like they are breaking through and making this a real possibility. The enthusiasm around both projects is real and infectious, and having gotten a hold of a Rift prototype in October, I confess that I joined in. Potential projects, interesting research opportunities, titles that you just couldn’t do before, they’re all bubbling up and out of me, and I’m enthused about getting my hands dirty in a way I haven’t been for games development in a long time. What’s clear to me is that many or most of the existing techniques we use, both for user input and for user interfaces, just don’t work in a VR setting. We’ll need to throw a bunch of our old preconceptions about how to build games out and learn them anew; as well as conquer a few more problems which are unique to VR. I have plenty of ideas on that front, but will need to try them out to see if I really understand the problems, let alone the solutions.

My Rift prototype turned up earlier this week, and I find myself massively frustrated that I’m busy with other more pressing projects and can’t get started. I’ll hopefully rectify that soon enough, and you should probably expect a bunch more posts around my experimentation with the kit. I’ll have to wait until next year for a castAR kit, but that will open up even more possibilities in a different way to the Rift, and I think the final solutions will draw on the lessons learned from both systems.

A typical crunch story

Posted in Industry Rants on October 31st, 2013 by MrCranky

Following on from my previous two posts about why crunch happens, the last of my crunch posts (for a while at least) focuses on the developer, and why crunch happens even when projects are started with the best of intentions.

For most developers, the underlying business reality is that the deadlines are fixed, the budget has little room to grow, and the scope is broadly fixed when the title is green-lit. The only axis with any real wiggle room is quality, but dropping your title’s quality will hurt sales, and even if it doesn’t cost you this time, your next contract will suffer because you let the quality bar slip. But it’s that inability to shift any of the parameters which is the reason why crunch is so common in our industry. Starting off with an unrealistic schedule is what causes crunch. Failing to respond to external or internal factors that have increased the project cost, either by shifting the deadline, cutting scope or increasing the budget causes crunch. If a team is closing in on a deadline they can’t make, and the developers can’t shift the deadline or cut scope, then of course they’re going to try crunch, ineffectual as it is. They’re stuck. Why? Because the entire thing was unrealistic in the first place.

Most big games seem to involve crunch in some way (whether they turn out good or bad). But we all know, management included, that crunch is something to be avoided. At some point, the management and/or the team, voluntarily or not, decide that crunch is the least bad of all their available options. Given how bad crunch can be, and how many bad experiences we’ve all had, I don’t believe that smart, capable people would make that decision for no good reason. So I want to explore that reasoning, and perhaps bring it out into the open.

I’d like to posit an example that I’ve seen a few times, obviously it’s not the only case. The developer is mid-way through their project. Two weeks from a big milestone, the time for what needs to go in doesn’t fit into two weeks. Publisher won’t budge on dates or features, and there’s no more people to put on it. But maybe it’s only three weeks worth of work. So the team does 60 hour weeks, but they still don’t quite get it all done. But they were close enough that the publisher accepts it, and the work still left to do (lets say a couple of days) gets rolled over, because you’ve claimed to deliver it, right? You can’t get it cut later, the work still needs done. But hey, only two weeks of crunch is productive, right? And it felt productive – you got 2 and 3/4 weeks done in the space of two. And the crunch is ‘done’. Only now you’ve just cut two days out of your budget for the next milestone. And even if you hadn’t the next milestone was actually a week over budget as well.

Chain a few of those milestones together, and not only have you been alternating between fortnights of crunch and 40 hour weeks, but your actual feature set / quality is lagging behind the milestone list, and the publisher and their QA team know it. For milestone one the decision seemed obvious – it was only an extra week of work, and you pretty much nailed that. For milestone two, well, you knew there had to be a bit of knock-on when you slipped the first milestone a little. Third and fourth? Now the publisher is on your back, and things are getting awkward. Now it’s not “we need to somehow get an extra week’s work done to make this the game we want it to be,” it’s “we need to get an extra fortnight’s work done just to avoid the publisher canning us for breach of contract.” They’re running just to stay upright.

At that point, the management are sitting there with a pretty rubbish choice. If they do crunch, well then perhaps those work-time studies were right, and the team will actually get less than 40 hours done in a 60 hour week. But if they don’t crunch, then they know they’re going to fail. The milestone won’t be hit, the bills won’t be paid, and it all goes south really fast. The only hope they have is that the studies were wrong, that their team is at the top end of that bell curve, and that they can still be more productive than normal even though they’re pushing harder. But the fact is, they don’t really know. There’s no control group to compare themselves against, there’s no equivalent game being made without crunch. So they crunch and hope, while they try to dig their way out by other means (pleading with the publisher for more leeway, slashing the quality bar below where they’re happy with it, stealing resources from other projects / teams).

Thing is, the alternative: no crunch, and hope that by not crunching you actually do more already assumes that you’re so far down the road of crunch that even with >100% effort you’re doing <100% actual work. And most teams aren’t prepared to admit that. Not the managers, the teams. They know that the shit is hitting the fan, and they want to bail the team out, they don’t want to be the ones saying “actually guys, I was zoned out for a whole bunch of last week and maybe did 30 hours of actual work in my 60.” They see their bosses sitting in the meeting rooms with the publisher with all serious looks on their faces, and a lot of them (usually the younger ones who haven’t been through the wringer quite as often) feel guilty that they couldn’t be more effective, that they’re struggling after a few long hard weeks.

Worse, if the managers did say “no crunch, and we’ll do better work,” they’d have to admit to the publishers that they’ve been barely able to hit the milestones they agreed on, making them look like a poor developer. Because even if the publishers aren’t aware of the crunch before, you’ve got to explain why crunching now isn’t even an option. Now if there’s been shifting milestones or external factors that can be argued around a bit, but fundamentally the developer is having to admit to the publisher that they’re not good enough at development to deliver on what they’ve promised, for whatever reason. That’s a bitter pill, and not one that most developers want to swallow.

Again I think it’s stemming from the harsh financial conditions and unfounded optimism: the budget is fixed low due to market expectations, but the feature set / quality bar doesn’t shift; the developers agree to the optimistic assessment because it’s sign this gig or go hungry. Then everybody loses. The team gets burnt out, the developer loses money and their team, the publisher gets a shit game if they get a game at all, and the customer gets delays on their game and a poorer experience. It just isn’t as simple as those who’ve been burnt by crunch saying “it simply never works.” Long term we know that’s true. Even short term it’s not great. That doesn’t mean it won’t happen, or that sometimes it doesn’t need to happen.

But just because I understand the reasoning, doesn’t mean I agree with it. Management shouldn’t be burying their heads in the sand. They should be honest about their teams situation and performance, and they need to know that the very real costs of crunch on the staff aren’t something they can just ignore. If workers aren’t shouting against crunch, management are all too likely to forget that it’s not just the productivity on the game that matters, but the well-being of their staff and team, up to that deadline and beyond it. We absolutely should not be accepting the word of management teams that are conflating crunch with ‘passion’, and suggesting that crunch is a natural, positive part of game development. It’s not. Mandated crunch indicates a severe, uncorrected failure from somewhere along the line. Maybe it was the planning, maybe it was the publisher, maybe it was the team, maybe a combination of all three. But it’s always a failure.

The real cost of making games

Posted in Industry Rants on October 24th, 2013 by MrCranky

The last time I talked about inaccurate estimating, and the dangerous road publishers and developers are heading down by lying to themselves and each other about the real cost involved in making their games. To me, the arguments about crunch and contingency are looking in the wrong place. They’re a symptom, not the root problem in themselves. Crunch happens, because there aren’t any tenable options left to the developer that is mid way through a title, and has a fixed deadline to hit. To appreciate why it’s the only option left, you have to step back a bit.

Most developers are pitching for business from publishers. A few get their finance from a non-publisher entity, but the relationship is effectively the same. Publisher-owned studios are in much the same situation, it’s just that the pitch and negotiation stage isn’t between two distinct businesses, but between units in the same business; so the negotiation is less antagonistic, but the basic relationship is the same. One side provides the finance, and gets the revenue/profits from selling the game; the other provides the game for some cost. The financier is buying a title that it can sell on for a profit. The console market has moved to a place where to make a profit, you have to hit a certain level of quality and have a game of a certain level of scope. So there is a minimum viable product for the financier, and an effective market size that means it’s not cost-effective to make a title unless it costs little enough that it can make its costs back. Most titles cost is proportional to the number of man-months involved, so shifting a deadline out doesn’t really save any money, quite the opposite – the developer staff need paid more for that extra time. So generally, the deadline is fixed.

It’s with that price in mind the only variable left gets decided: scope. How big a game will it be? How complicated? Will it break new ground, or go with a safe mechanic or style that the developer is confident of delivering for the budget? Here’s where the problem comes: how big does it need to be to make its money back? I think we’ve got to face the very real possibility that the effective cost of making the games the console market expects outstrips the likely revenue you’ll get from those titles. If it does, the difference has to come from somewhere.

From the developer’s point of view, it is hard to get a publisher to sign on to what you think is a reasonable price for making the game they’d like. Of course they want more for less; their margins have been squeezed to the bone as it is. But if you have a team of staff waiting to make a game, the cost of refusing to make a game because the publisher is only prepared to pay 80 or 90% of what you think it will actually take to make their game may be that you fold altogether. At least if you take the 80% deal you can argue the scope down later, or find some other way of making it work.

That’s where the trouble kicks in. If your company is bidding low to get financing, then making up the difference through crunch (which is effectively asking the employees to subsidise the project cost through ‘free’ labour), then it’s screwed. But the alternatives aren’t much better for the company, although they’re clearly better for the staff:

  1. Don’t make the game at all. Company has no business, shuts. Financiers get no games, can’t make a profit.
  2. Make a smaller game. Market rejects it due to unrealistic expectations, financiers lose out, next title doesn’t get funded, company shuts.
  3. Bid low and try to make the game for less than it costs, through crunch. Company and financiers do okay on this title. Staff get burnt out, next title costs even more to deliver (through reduced efficiency/quality), repeat this choice scenario again but with worse numbers to start with.
  4. Bid low and manage to raise the price later. Company does okay, but financier loses out when revenue doesn’t match cost. Next title doesn’t get funded, company shuts.
  5. Bid realistically, financier knows the numbers don’t work. Company loses out, shuts. Financier either gets no games, or finds some company willing to choose scenario 3.

You can probably see why companies choose option 3, even when they know what the consequences are. Because it’s the least-bad option available to them. And they can persuade themselves that this time will be different, this time they’ll work smarter, and they’ll hit those lower costs without crunching, because they’re good at what they do. When that works out, everyone’s happy. When it doesn’t, there are lots of factors they can blame. NB: “Bid realistically” here means hiring great planners, and adopting a sensible, reactive planning approach like I described last time. A company can be bidding low without even realising it, but that doesn’t make their situation any better.

When the fundamentals of it are that it costs that particular developer more to make that particular game than they thought, that’s a business doomed to extinction. Crunch is a side issue, one of many symptoms, of which the root cause is denial about how much it actually costs to make the games we are building. The only way out is to make different games, maybe in different markets, which actually cost less to make than they take in revenue. Maybe I’m wrong, maybe the console games business is eminently viable. But the reality of difficult financial conditions and the developer’s strategy for dealing with that is the core problem underlying crunch. Railing against crunch is going to do little to help us, if we don’t address the underlying business conditions that cause the unrealistic expectations in the first place.

Crunch vs. Contingency

Posted in Industry Rants on October 17th, 2013 by MrCranky

So the PlayStation 4 and XBox One are soon to be released, launching us into another console generation. This time around, it’s not just me that is cynical about the prospects for the ‘traditional’ games industry. The ecosystem of games has been changed irrevocably by the advent of smartphones, tablets, and a resurgence from PC gaming. It’s no longer a given that there is a niche for console gaming large enough to support the costs of developing those games. But I’ve certainly been wrong before, and I don’t want to call console gaming dead before its time.

Recently, in response to this article on crunch, I found myself  coming at this tired old debate from another angle. Many in the industry, generally not management types, are frustrated by the management’s inability to put in sufficient contingency, resulting in an almost inevitable period of crunch, where the developers put in overtime far over and above their expected working hours, to try and get the title out  for its fixed deadline. Typically, when the ‘more contingency’ argument is rolled out, it is countered with “game development is hard, and unpredictable,” and “you can’t schedule for ‘fun’.” The counter-counter argument to that is typically that other software industries deal with equally unpredictable factors, and they don’t have to crunch in quite as pathological a way as we do. The core of these arguments is really this niggling underlying sense that crunch is a natural consequence of not being quite good enough at making games, and that’s problematic.

Thing is, being bad at making games is a cause of crunch. But not because the people making the games are bad at what they do. Because part of making games is estimating how long it will take (and correspondingly how much it will cost) to make the game, given the team you have. Not an ideal team, not the team you’d like to have, the team you have actually got. Planning is hard. Some game-devs, usually the ones who’ve not had to make a plan for any sort of sizable project, think that all that is needed is ‘more contingency.’ This is waved around as if it was really simply to do, and that the management / planners are not doing it deliberately so that crunch is required, because crunch is cheap, and contingency isn’t. But anyone that has to make a plan, and more importantly anyone that has to sell a plan to the game’s financiers, knows that simply whacking on a bigger and bigger percentage figure for contingency doesn’t work. It is admitting that you don’t know how things are going to go, and trying to pick a single large fudge factor that insulates you against bidding too high or too low. We almost never make the same game twice; previous games aren’t much help at predicting how long future games will take. You can break things down to estimable components, but the way those components interact, in ways which may or may not work, which may or may not be fun, is what turns a project from under-budget to over-budget.

That’s not to say we can’t get a lot closer than we do, with better planning. Game-devs in my experience are almost always hopelessly optimistic, even though project after project teaches them that requirements do change, designs do change, and that a sizeable software project invariably has nuances that couldn’t reasonably be predicted at the start. Fundamentally though, there are two changes that need to happen before we’ll stop seeing regular, mandated crunch.

Firstly, we need to accept that the scope, design and timetable for the development is flexible. Trying to nail down the plan up front is foolish and naive. Either the developer does stick to the plan, and the game is crippled because it didn’t respond to the practically inevitable changes that were needed to make it the game it should have been; or the developer diverges from the plan, and either the publisher has to pick up the cost (from the deadline slipping) or the developer does (either by paying for more development time, or by burning out their staff with crunch). As the development continues, the plan should become more and more clear, but it won’t be clear up front. A good developer, and the publisher/financier that is bankrolling the development, will be continually re-assessing the plan as to what is feasible, and what is desired. The publisher will always be pushing for more for less money, and the developer will be pushing for less, but it needs to be accepted that the ‘plan’ is a continually shifting thing, that is going to end up being a comprimise, negotiated by both sides.

Secondly, both the financier/publisher and developer need to be honest about how much it actually costs to make the games that are being made. Hiding the real development cost of a title by burying it in crunch is effectively passing off some of the cost of development onto the staff, and that is fundamentally bad for all concerned. But more importantly, it’s leading both developer and publisher down the road to bankruptcy, from sticking their heads in the sand. More on that next time.

NASCAR: Redline

Posted in Games on October 10th, 2013 by MrCranky

Finally! The fruits of our labour since November last year have made it to the app store, and soon enough the Android marketplace. Ladies and gentlemen, I give you:-

NASCAR: Redline

I’ve shown screenshots there that give you a sense of how great the Eutechnyx art team got the cars and tracks looking, but at its core this is more about the tactics and strategy of real NASCAR racing than it is about twitch driving skills. Not to diminish the thrill of watching the lovely 3D segments where you see your driver slipping through the tiniest of gaps or avoiding a big pile-up, but the off-track decisions play as much a part in your final position as the driving. When to pit, how far you can stretch your tire wear, choosing the right parts for the track you’re racing on, you need to get all those things right to come out on top. For the real NASCAR fans they’ll love competing against their favourite drivers, on all the real NASCAR tracks, to really feel like they’re part of the Chase for the Sprint Cup.

I had a great time working with the team at Eutechnyx to help build this title, so it’s a real thrill to finally see it out there on the app store making NASCAR fans happy. As for me, I’ve been taking a well-earned break, to try and get the constant thrum of highly tuned engine noise out of my head. 🙂

Suddenly, melon.

Posted in Random Stuff on September 10th, 2013 by MrCranky

A post, or possibly series of posts, on reasons behind crunch in the works now I have a bit more slack, but until then, I thought I’d share something that made me smile this morning.

 

Suddenly, melon.

Suddenly, melon.

Dean Village is a lovely place to have an office, but the locals are… Well, let’s just say that random large fruit by number 12 doesn’t surprise me any more.


Email: info@blackcompanystudios.co.uk
Black Company Studios Limited, The Melting Pot, 5 Rose Street, Edinburgh, EH2 2PR
Registered in Scotland (SC283017) VAT Reg. No.: 886 4592 64
Last modified: February 06 2020.