10-Tative Code
Back at [company X], I think it was near the end of [the project], we had an object in one of the levels that needed to be hidden. We didn't want to re-export the level and we did not use checksum names. So right smack in the middle of the engine code we had something like the following. The game shipped with this in.
if( level == 10 && object == 56 )
{
HideObject();
}
Maybe a year later, an artist using our engine came to us very frustrated about why an object in their level was not showing up after exporting to what resolved to level 10. I wonder why?
- Anonymous
All Signs Point to "No-No"
This issue came up during Raven Software's new Wolfenstein, as I was setting up controller support for the 360. It turned out that for the Live integration, we had to know which controller was sending input events. The Doom 3 input code we were using was derived almost straight from Quake 3, so it was a pretty simple system.
In the existing event system, each event was passed around with two integer arguments and a void pointer in case you needed extra parameters. So the goal was to associate a controller ID with each input event. No problem -- just pack the controller ID into one of the event's integer arguments, right? Of course, they were both already in use.
Then came the next idea: Let's just use our fast no-fragmentation per-frame allocator to reserve some memory, plop the controller id in there, and then pass a pointer to it in the event's pointer slot.
It turns out that the event system would take it upon itself to free() the event's void pointer after processing the event. Of course this was totally incompatible with the multiheap approach that we were using. Moreover, there were a few legacy Doom 3 chunks that actually relied on the event code calling free(), so we couldn't just remove the call without non-trivial changes across the codebase. What about adding a third integer parameter? Well, that would involve changing several hundred function calls.
The deadline is looming, I can't spend much more time on this. So, I did the unthinkable -- I packed the controller id into the pointer parameter. I marked it as a horrible hack in a 4-line all-caps comment, and checked it in. This worked fine for a while, and lasted until the whole input system was replaced with something a little better down the line.
- David Dynerman
Velcro Sneaks
Here's a tale from a project in the early PS2 days. We had a ton of collision/bounding issues that were largely solved at the last minute by a rewrite of character collision to use a "collider" model -- a stack of spheres that were way better at resolving bumps vs. boxes than our hierarchical tree of oriented boxes ever could be (ah, the Land Before Havok).
However, we had a rare bug for ages that was pretty maddening -- we called it Velcro-ing -- where every now and then the player character would be up against a wall and sliding along it, but then a sphere of the collider would suddenly decide it was on the wrong side of the wall and wouldn't let you get away from it (i.e. you'd be stuck to the wall's surface).
This was one of those god-awful "sounds easy" bugs -- just figure out why the sphere thinks it's on the wrong side, and fix it. Problem is, you had to catch what happened in all the collision resolutions before it happened to figure this one out.
When you'd drop in things like handy conditional breakpoints to look for a weird response, or a push that made it get confused where it was on a skinny box or something similar, it'd bog the frame rate down and the issue simply wouldn't happen. (The real fix to this problem took blood, sweat, tears, and I think other fluids not usually required, but that is another story.)
Since we were trying to submit, and regularly would get this issue as a blocking bug, one "solution" was to buff out the bounding on whatever was causing the problem, which was guaranteed to make it not happen, but was clearly not the right fix. We'd forward the bug to our art team, they'd fix up that case, and in the ensuing turn-around time, we bought ourselves precious days to keep trying to track down the actual deep, subtle problem.
Testers would typically send back a "player hit invisible wall, player must not hit invisible wall" bug of roughly equal severity (bounding the wall tightly again fixed that, as Velcro-ing was really rare and hard to reproduce anyway), but after a couple cycles of this as we cleaned up the rest of the game, the blood, sweat, and tears fix was eventually found and off we went to duplication and shelves.
This wasn't the proudest cycle of bug-turn-around-stalling, but it got the upper-ups off our backs, because we could say "oh, no, we released that to testing yesterday," and keep scrambling to figure out that damned flipping issue.
- Anonymous
|
en.wikipedia.org/wiki/Checksum
I was just about to tell the team the good news when the producer pulled me aside, and mimed a "shh!" gesture.
The team continued to crunch, none the wiser, until we reached the now fake deadline. After that, the producer acted dissappointed in us, and told us to keep going.
Boy, that producer really knew how to get work out of people who were demoralized and exhausted already! HAH HAH!
I feel strangely more human all of a sudden!
As for 'The Programming Antihero" - Well on that one I actually thought *everyone* did that. The real trick when coding solo is to make yourself forget about that array until the very last minute of the project.. ;)
There is a special place in hell for producers that give resources fake deadlines. It's a trick bad producers use to make themselves look good, at the expense of everyone else. In this case, your team suffered so your bad producer could be the hero to the client (and upper management).
Way to cover up for him/her!
Did you like seeing your other team members get extra demoralized and extra exhausted? I guess letting them also be "super relieved" was not your problem. They must have been amazed at your ability to handle the stress of the "unrealistically aggressive" fake deadline. Hopefully that "can-do" attitude got you a promotion.
Good producers work honestly with their team, and they get far more accomplished as a result.
The fake deadline trick tends to only work once (it always gets discovered).
Ever notice that teams miss deadlines for some producers more than others?
Ever wonder why that is? It's all about trust and respect.
HAH HAH!
As some have already mentioned, this is done fairly often with deadlines in quite a few fields. I've not ever seen it done with memory constraints, though!
I can see I'll be looking forward to more in the comments, too. ;)
As far as that goes, we were recently having problems where the player avatar would mysteriously disappear and become unresponsive due to the screen clipping against a non-square map. We found the fix readily enough, but the performance cost was so high, that we ended up just forcing the player to render if they were alive with the comment:
// HACK
// Players should ALWAYS be on screen if they exist.
There were also some related to how we had to implement our own renderer to get half-decent performance, but the real ugly stuff was in the grid optimisation. We lost about two solid weeks ironing out the bugs in that (which, considering our deadline, was actually about a quarter of the total development time; madness), and we still ended up with another avatar-specific hack so players wouldn't be dropped from the game state if they crossed a grid boundary. The code for this is still enclosed in
// HACK HACK HACK LOUSY HACK
and
// End of lousy hack.
But I love the trick in "The Programming Antihero"; I think I'm going to start using it. :D
Employees is a little vague though. Need a way of distinguishing between management, account managers, and senior leads (techincal or creative). Unless the producer is insane, those employees aren't given fake deadlines to motivate them. It's only the people underneath the producer, the creative and technical resources, that get abused. Not sure how else to group them.
What could be more obvious? Only render every second letter! But alternate, so that on even numbered frames, you render even-numbered letters, and on odd-numbered frames, you render odd-numbered letters!
This leaves you a flickery, but fast and usable, debug menu.
Stuff like this is a great read. Not only is it amusing but it also teaches you tricks you can implement when in a crisis, or more importantly what kind of problem might blow up in your face and how you can avoid them to begin with.
Sir, do not worry! You may have missed my sarcastic tones, and also I missed a key plot point: I did the exactly opposite of cover for him.
I was as shocked as you, and told my comrades the story behind the producer's back, because I'm a LOOSE CANNON. It was still a hard push, but they were able to pace themselves.
"static char buffer[1024*1024*2];"
That's just priceless.
i really enjoyed every one. I would love to see the second edition of this, or maybe more!
Profuse apologies, I TOTALLY did not realize you were joking.
Tone totally changes everything.
I applaud your approach.
I deem thee "Sir" Loose Cannon.
@Brandon
Forgot to mention I loved the article.
Great reading. I have gained wisdom.
Major kudos to Noel Llopis' old hand coder for that memory trick!
At university there was a team (not related to me, but these guys are the perfect example :P) that made a FPS flash game...
For some bizarre reason, the programmer instead of checking if you was colliding with the wall and not allow you go there, he made the inverse, he checked if there was a wall, and allowed you to move parallel to it...
This sparked a bizarre bug: In crossings, you could not actually cross, only turn to the passage on your left or right.
The deadline was closing, and they had no idea on how to fix it...
Then the team writer fixed the issue! He told the artist to draw a animation of hands touching the walls, and then he wrote in the story that the protagonist was blind and needed to touch the walls to know where he was going.
//@hack
//@remove
//@fixme!
And, well... they almost never get changed :)
Turned out the flash init code was dead and the carts could not save games properly!!! Studio went into meltdown trying to figure out how to ship 250'000 broken carts. Suggestions of production lines adding extra resistors and other hacks to every cart were tried and failed , then some figured out if you played some games in a weird order the flash memory would sort of work. So i extra leaflet was added to every box explaining how to use this "feature".
Job done
Chris Kirby
That's the best thing I've ever heard!
Yes, of course...if you are in need of many megabytes of memory, inspecting the code-size is definitely not the most obvious thing to do ;) but, at least for console/handheld-projects, it is somewhat of a standard procedure.
I have found similar (but unintended) stuff in earlier projects while inspecting the map-file once in a while.
------
Don't know how many remember Force 21, but it was an early 3D RTS which used a follow cam to observe your current platoon. Towards the end of the project we had a strange bug where the camera would stop following the platoon -- it would just stay where it was while your platoon moved on and nothing would budge it. The apparent cause was random because we couldn't find a decent repro case. Until, finally, one of the testers noticed that it happened more often when an air strike occurred near your vehicles. Using that info I was able to track it down.
Because the camera was using velocity and acceleration and was collidable, I derived it from our PhysicalObject class, which had those characteristics. It also had another characteristic: PhysicalObjects could take damage. The air strikes did enough damage in a large enough radius that they were quite literally "killing" the camera.
I did fix the bug by ensuring that cameras couldn't take damage, but just to be sure, I boosted their armor and hit points to ridiculous levels. I believe I can safely say we had the toughest camera in any game.
------
When I was a lead on R6 Lockdown PS2, I also used the "The Programming Antihero" trick -- whenever an engineer found a significant memory savings I told them to release enough to get the game under budget for the time being, but hold some back for the eventual future overrun. In the end we had a 1-2K buffer (IIRC) that we released just before ship. Most of the team never knew.
Well, this is basically what Microsoft uses as their WPARAM and HPARAM in WinProc, so it wasn't that bad =)
@Evan Bell: I am a programmer in the healthcare industry and hit that lottery once some years ago. I had to load a text file with over 100k lines, one per medicine, and two lines gave the same CRC32. Fortunately the update routine was run in our server and the clash was easily found. For two files with the same CRC32, check http://www.allegro.cc/forums/thread/585925
Although not game derived, I got a couple to share. Around 5 years ago we had to change the L&F of the application. That included switching menus over, changing background images, reshaping controls, etc. It went pretty well (at 16 hours per day, took us just a week). However, testers complained that the application started to randomly crash after some time of use, with an out of memory error. Just 4 hours before shipping we discovered the error: the new rounded buttons didn't free the normal/pushed bitmaps correctly, eating memory every time they appeared on screen. So we had to remove the new buttons, and shipped a Mac-like rounded application with gray square buttons.
Regarding fake deadlines, we once decided to fulfill it. Indeed, we reached it, had the product uploaded to our servers, fully tested and waiting for the approval of the CEO to be released. But he decided not to launch it. The thing was that he had put a very impossible goal so that we would be forced to delay the package, giving him time to suggest more features before the real launch date arrived.
Also, whenever the physician chose a drug, it would be stored into the database, linked to the patient and a recipe within a visit. However, one of the programmers really messed up and, instead of storing the NDC (national drug code as set by the FDA), he stored the drug index in the drug database. The drug database was usually modified once per month (removing drugs that had expired, adding new drugs alphabetically, etc) so that, when I checked, the IDs in our database were usually pointing to drugs that were not the chosen ones. Even though the contraindication check used names instead of
NDC (which made the routine work still), it was too risky to have random numbers as medicine codes, so we had to clean them up. None of them was salvageable, and I had to wipe out the entire column, but there was no way to explain that to the users without making them think they were in great risk. So, I placed a progress bar that lasted around 7 minutes with something like "Upgrading drug information" which did nothing at all, and notified the user that he would be forced to relink some (all!) of the medicines in the program database with the ones from the medicine database as he starts reusing them. None of the doctors complained, though, and everything has been going smooth since then.
Chris Kirby's one reminds me of a couple. A programmer had just installed a spell checker into our program, with the ability to add your own custom words, and he started testing with words like a**, f*ck, and similar. Unfortunately, he didn't clear the table and we got angry calls from one doctor who was showing his secretary the new spell checking functionality when f*ck popped up. So, always clear your databases before shipping the product!
Great feature! I look forward to "Dirty Coding Tricks II"!
One of the "impossible" errors were: "OH MY GOD, THE GAME IS TOTALLY FUCKED AND THE WORLD WILL EXPLODE BECAUSE SOME IMPOSSIBLE SHIT HAPPENED AND ACTIVATED THE IMPROBABILITY DRIVE CAUSING A TIME PARADOX THAT WILL SCREW THE UNIVERSE IN ALL FUCKING SEXUAL POSITIONS POSSIBLE"
After I typed that, I plainly forgot it, it was embedeed deeply in some initialization routines of the API, a code that once done I would never peek at, also it was the only case that I used profanity or that sort of stuff (I am a person that is not much into profanity, but that day was a bad day, so I ranted on that error message).
Unofortunally, one of the testers actually made that error happen... He called me all confused, and me too (since I don't remembered it) and I tought that he was joking, but I searched and actually found it...
Then I had two questions on my head:
How that error happened? (awnser: a mistype that I introduced on the lastest build caused a nasty chain reaction of bugs that made the execution stack go awry, making the impossible actually possible).
Why I did wrote so much profanity? (awnser: In fact I am still wondering the awnser for that...)
So this is a way to access private/protected members if there are no getter/setter.
Now I'm not saying I'd do that but in the case you're working on a program with an API and you only have access to the .h file, you could get around it this way. =)
I don't know why this ever worked but I had read that if you shift the mouse position left and then right it solved the problem. So I added that and it worked.
Later a programmer looked at this and thought it was just self cancelling and commented out the code and you could no longer click the button.
I explained the real problem was that we were not counting "Mickeys" and should move to a relative system. He asked if I knew how and that is how I moved from being a tester to a programmer.