Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
July 22, 2014
arrowPress Releases
July 22, 2014
PR Newswire
View All
View All     Submit Event





If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 
A Look at Kickstarted Video Game Delivery Rates
by Un Subject on 02/25/14 12:55:00 pm   Featured Blogs

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.

 

Back about two years ago, someone challenged my wariness about video games funded through Kickstarter. “Where was the evidence that these crowdfunded games weren’t going to deliver? Wasn’t I just being a jerk by being so doubtful?” they asked me in broad terms.

This was possibly a rhetorical question, asked during the 2012 Kickstarter boom, but it struck me that no-one else had a hard answer to those questions.

So I did the passive aggressive thing, collected the data and analysed it. I did some (pretty rambling) analysis then, and recently went back and looked again at how Kickstarted video games were doing.

As a short description of my approach, I collected a list of the successfully funded Kickstarter video game projects from 2009 through to the end of 2012, then waited to the end of 2013 to see how many projects across this time frame officially delivered the title (mostly video games, with a smattering of other video game-related things, like books and OUYAs) as promised to their backers. I say ‘officially’ because some projects (example: OUYA again) may have formally launched into the market, but individual project backers may not have received the rewards they were promised.

It was thought that a year was a long enough period to wait to test out what the delivery rates for 2009 to 2012 – out of the 366 projects in the analysis, only 9 indicated they were targeting a 2014 release date. I’ve tried to ignore these 9 titles in the rest of the analysis.

Overall Delivery Rate is Worse Than Shaq’s Free Throw Percentage

The major finding of this analysis was that only around a third of successfully funded Kickstarter video games have fully delivered on their title over the indicated time period. If you include the number of partially delivered titles, such as the first episode of five being released or an iOS version being released with a PC version still on the way, this delivery result increases to about half of all Kickstarted video games.

A pie chart showing that 37% of Kickstarted video games fully deliver on their title promised in the pitch.

 

 

 

 

 

 

 

 

(I’ll make the note that I haven’t included the release of alpha / beta versions or launch on Steam Early Access as a partial delivery, unless that was what the original Kickstarter pitch said was the end result. Titles in alpha / beta / Early Access are by definition unfinished and development on these games is hopefully ongoing.)

I’ve seen some comments that this delivery rate is pretty good, or at least in line with expectations about software development. Maybe that’s true, but I wonder how all those software project failures are dealt with by the people who funded them. Formal investors are likely to have some kind of contractual protection against a project being a write-off and I doubt software developers often just get to shrug their shoulders, go, “Eh, we tried,” and leave a failed project without any consequence.

Crowdfunding is a bit different in that it is a trust exercise. A Kickstarter backer trusts the project to deliver on its promises and ‘buys in’ at a reward level. Very, very few backers put money into a project but forgo the rewards offered – for example, only 244 out of the 34397 backers for Star Citizen aren’t assigned to a reward level. That’s less than 1% of backers who appear to be only putting in money to support a project without expecting anything in return, and it’s a figure that appears consistent with other Kickstarter projects I’ve looked at.

Failure to deliver violates this trust and damages the possibility of that person backing future projects. That’s why I consider a full delivery rate of only around 1 in 3 projects pretty poor – in the longer term, it makes crowdfunding video games a lot less viable as backers get burned and drop out, or only back projects that are ‘guaranteed’ not to fail.

The Waiting Game

Delays are perhaps to be expected among Kickstarted video game projects and a question was asked about how long backers have to wait for their project to release something. In short, only around one in ten Kickstarted video game delivers ‘on time’ (i.e. roughly within their Kickstarter estimated launch month) while almost half of projects have left their backers waiting a year or more over the original date listed in the pitch.

A chart showing 10% of Kickstarter video game projects are delivered on time and more than half leave backers waiting at least 4 months.

 

 

 

 

 

 

 

 

For those wondering how an undelivered game can be ‘on time’, those 2% of titles were those promised at the end of 2013, when the analysis period ended.

In this instance, only full delivery is being counted, so there are some titles like Kentucky Route Zero that has released some episodes but hasn’t yet delivered its full amount. (It’s arguable that someone backing Kentucky Route Zero back in 2011 was aware that it was going to be an episodically released title – that change was first announced via Kickstarter in October 2012. Backers may have been made aware of this change earlier through direct newsletters though.) However, only a small proportion of Kickstarted video games have been classified as partially delivered (8%), so the majority of titles that look delayed above are exactly that.

All in all, if you are evaluating a Kickstarter project before you back it, it’s pretty safe to add another 12 months to that estimated delivery date and consider if you’re willing to wait that long before receiving what you contributed money towards.

Is It The Money?

A question was asked if projects with larger funding targets and / or higher achieved amounts took longer to deliver. There’s some evidence to support that idea, with titles that received $100k or more being disproportionately undelivered (admittedly on a small numerical base).

My last chart on this for a wihle, I promise.

 

 

 

 

 

 

 

 

I was interested to see that the majority of successful Kickstarted video games from 2009 to 2012 received $20k or less from crowdfunding – that’s not a lot in game development terms.

If you combine these dollar figures with the delays over the estimated launch times, it opens a question about how well these projects can really plan their budgets. A title that receives crowdfunding covering four months of development but actually takes ten months to complete potentially leaves a big funding hole that the developer(s) need to fill, which is a risk to the successful delivery of the project.

Conclusion: I’m Still A Jerk About Kickstarter For Video Games

Following all this analysis, I’m still pretty comfortable being wary about Kickstarted video games. I’m not willing to spend money and accept the risk of a project failing. If a project succeeds, then I can always purchase it through the normal channels.

Part of the point behind crunching these numbers is to provide some hard(er) figures around the risk of being a backer. Too often it is only the successful titles that get the attention, which I believe distorts how people see risk around crowdfuning. For all those who are happy to accept that risk – and a lot of backers ask for refunds when a project fails, so don’t think that all those put money into a Kickstarted game will accept losing it – then hopefully the above at least has served as something to think about.

Caveats

You can find most of the data I used to develop this analysis here (although the recent date-based analysis isn’t there year and will go up across the course of the week). The vast majority of the data from Kickstarter was manually entered and I’ve done my best to minimise any errors in my work.

Probably the largest contention with this analysis will be around the classification of the titles’ completion states. I’ve done my best to find out how each of the 366 projects have been going, but that’s not easy when updates are hidden behind backer-only protections or developers have stopped updating the Kickstarter site and instead rely of personal blogs and / or Facebook to release status updates.


Related Jobs

Petroglyph Games
Petroglyph Games — Las Vegas, Nevada, United States
[07.22.14]

Senior Unity Artist
Sony Online Entertainment, San Diego
Sony Online Entertainment, San Diego — San Diego, California, United States
[07.22.14]

Intermediate and Senior Database Tools Programmers
2K
2K — Novato, California, United States
[07.22.14]

Senior Engine Programmer
tic toc games
tic toc games — North Hollywood, California, United States
[07.22.14]

Project Manager






Comments


Michael Joseph
profile image
"I’m not willing to spend money and accept the risk of a project failing."

Well whatever floats your boat, but I think for some projects and some customers their is a risk that a project they really want made will not get funded and not get made. This makes it not just a "trust" exercise but a "hope" exercise. And there's a certain feel good factor involved with "having the vision" to back a certain project and then being one of the people who directly helped to make it happen.

All I'm saying is there's a specific type of risk and reward that you're ignoring in your investment calculus but which backers do not.

Also if you ever plan on launching a ks campaign yourself, being able to show a history of ks support may help.

Un Subject
profile image
There's an issue with relying on hope and trust - both very loaded emotional terms - when it comes to commercial enterprises. It can turn things very ugly when the project fails to deliver on what was promised in the Kickstarter pitch.

Part of the point of this analysis was to indicate that if you are backing a project from an emotional perspective, that you should be fully aware about the risks of that project not meeting that social contract with you.

I've also got issues with a financial backing system that sees projects milk emotional reactions in order to fund commercial enterprise (if you want to view Kickstarter / crowdfunding through that particular lens).

Kyle Redd
profile image
@Un Subject

Do you also have an issue with charities that milk emotional reactions to things like cancer or starving kids in order to fund their enterprises, where the backers rely on the hope that one day cancer will be cured and kids will no longer starve?

Voldemaras ZT
profile image
Last I heard charities are not commercial enterprises.

Cordero W
profile image
@Kyle: In the case of actual charities, you're giving money for a cause that may help the betterment of overall human lives. Video games we can do without, which should only make people who ask for crowdfunding to have more of an incentive to deliver since you're funding to develop a specific product, and thus have more of an ethical responsibility.

However, I have always been against Kickstarter for the simple fact that most people there have no sense of project management.

Un Subject
profile image
Depending on how those charities go about milking those emotions, yes.

But charities aren't (well, shouldn't be) looking to generate a profit off their donated money. And charities that start out broadly proclaiming what they'll do with your money only to not deliver can end up with some pretty hostile reactions.

Andy Lundell
profile image
Un Subject says "I've also got issues with a financial backing system that sees projects milk emotional reactions in order to fund commercial enterprise"

Ha! That's what I consider the best part!

For creative projects I really do think that being backed by people enthusiastic about the art is way better than being backed by people enthusiastic about return on investment.

Alexander Jhin
profile image
Agree with Michael Joseph. If NO ONE spends money on a Kickstarter game, it's pretty much GUARANTEED to fail. If people spend money on it, there's 50% chance that it will at least partially ship.

That's what's missing from this analysis: The opposite of the author, that is, the people who really want a game to ship and who are so excited they can't bear to have it fail.

Katy Smith
profile image
I find this report very interesting, thanks! I wonder if the games that are hitting the 100k-1M mark are struggling because they weren't able to cope with the extra stretch goals and feature creep?

I see backing Kickstarter projects as becoming a publisher of that game. As the publisher, do I think I'll get out of it at least what I put into it? The nice thing about kickstarter is that the risk is distributed over a large group of people, so I'm more willing to make small commitments on something that might be risky. When looking at it that way, developers that use Kickstarter need to pitch their games in a way that shows a plan, goals, and experience. So if you're making an emotional pitch, don't just give me "it will be cool", tell me why and how you plan on getting there.

Daniel Gutierrez
profile image
I've always found KS's presentation misleading. Coming from a startup/investment working background, I understand the risk with new projects pretty well. I know the hundreds I've thrown into projects was never going to give me the ROI that I'd get from just waiting to buy post-release, but I wanted to see some things have a chance, and willingly accepted that risk. But with how most of these projects are presented, coupled with the lack of research like the data presented here, the "pay $X and you get the game when it's done" tier options feel more like an Amazon pre-order than a probabilistic choice.

I would love to see KS do this type of analysis, it would be a responsible undertaking on their part. I also assume they have a lot more insight than is publicly available. Also following that up, it would be nice to see them setup a system where people can put up milestones, and they get portions of the money when they meet those milestones. This would at least let people investing cut their losses some on failed projects.

Bruno Xavier
profile image
Milestone payments are the key.

Martin Grider
profile image
I would love to see an analysis of all these game projects from the perspective of the companies / individuals behind them. 1) Do they do this full time? And 2) Is there an established company / team. if so, how many people? And how many people are working full time?

I'm surprised the idea that bigger funded projects take longer because the developer can afford to spend more time on them was not suggested. Games are never done. If you can live off your KS money for a year, why not spend that time working on it, polishing it? For projects that funded less than $20k, what are the chances the developer doesn't have a day job? That is simply not enough money to sustain a couple of people long enough to make a decently sized game. A single individual, yeah maybe. So they're going to be working slower, is my guess.

I think there are a good chunk of kickstarter games that are put out there in the hopes that the developer can quit their job, or at least go on hiatus long enough to make the project.

Thomas Happ
profile image
We need a comparison between Kickstarted games and games backed by a traditional publisher. I'd say I've worked on as many unfinished games (with a publisher) as completed ones. And even the completed ones don't always ship (publisher changes management at the last minute, etc.).

Michael Pianta
profile image
I think treating Kickstarter as some kind investment or unusual store for pre-orders is completely wrong headed. It is simply a means of artistic patronage. As with any artistic patronage, results vary. Things get delayed or they turn out poorly - sometimes they turn out so poorly as to essentially be worthless - but nothing about that is an ethical problem as far as I can tell. Kickstarter is ostensibly a means of getting things done that otherwise wouldn't get done, while also affording more authorial control to the creators. Both of those goals are good I think.

Will the projects that fail leave backers so burned that they abandon Kickstarter entirely? I don't think so, mainly because I think failure as defined in this article is too broad. True failure, meaning the failure to deliver anything at all is fairly uncommon as far as I can tell (I would be interested in the data on that). I don't think delays, or changes in scope or even substantive changes are really that big a deal to backers, because I think most backers are savvy enough about games to know to expect these things. It's not as though traditionally published games don't get delayed, or have features changed or removed, or turn out really disappointing when the trailer looked so promising. Those things suck, but they are also just part of the risk of being heavily invested in the hobby.

Cordero W
profile image
The worst thing anyone can do is hand money over to someone for free. There is no incentive to deliver other than "I promise." There is no gauge to go off to determine if a game is "good enough." It's simple psychology.

If it takes only ten thousand dollars to make a game, and someone gives the developer one hundred thousand, there are two things that will happen:

1) The developer will stick to the budget of "ten thousand" and pocket the rest. They may pay for a little extra polish, but most of the time, that money will be their living expenses, both necessary and unnecessary commodities.

2) The developer will actually use the money to make the game have more quality through its increased funding. But, unless the developer has any sense of budgeting and project planning, they will either overspend and suddenly do what Double Fine did, or they will underspend and do what Double Fine did. In both cases, they will pretty much do whatever they want with their extra income and can tell through PR whatever story they can fathom.

Moral of the story: don't give people free money.

Michael Pianta
profile image
A few thoughts:

First, define "for free". I'm not sure exactly what you mean. I assume you mean, 'without getting something specific back'. In which case I would say that that's not so bad at all. There are all sorts of reasons you might do that, one of which, historically, has been artistic patronage. The novel aspect of crowd funding lies only in the fact that artistic patronage has historically been the province of a small number of wealthy people, rather than a large number of mostly middle class people. In either case, you don't know for sure that it's going to work out, although you naturally hope so. But on the other hand, if by "for free" you simply mean without legal strings attached, then I would say that I don't know that it matters as much in a public forum like Kickstarter. Although as a Kickstarter backer you don't have any legal powers (sort of related to the idea of being a patron I think), there is still public pressure upon the developer. Unless the expect to make enough money to never work again, they still need to act in good faith or risk the ruination of their reputation and career. But at any rate, trustworthiness and competency to deliver is part of what everyone evaluates when deciding whether or not to back a project.

As for your other points, regarding 1.), I see nothing wrong with that. You asked for money to make a project, you get sufficient money, you make the thing you promised. No problem there, I think.

Regarding 2.) I don't see anything wrong with that either if you're acting in good faith. You seem to feel that Double Fine did something wrong, but I don't think so. They did what was expected of them and what we, the backers (I did back that project) wanted. I mean everyone was hoping they would expand the scope. One side effect of that is that of course it's going to take longer. The splitting of the game into two parts (in order to capitalize on further revenue from the first part) is sub optimal, but hardly unethical, nor is it harmful to backers in any way. Backers were mostly fine with it. It seemed to me that it was mainly other people and perhaps some in the press who were trying to characterize it as some kind of scandal.

Kevin Fishburne
profile image
Michael, you nailed the (at least initial) premise of Kickstarter, but as with many things in life what should be often isn't, or isn't for long. It's now a preorder system: the most popular reward tier is a copy of the game and Kickstarter campaigns take great pains to maximize revenue using that knowledge.

I'd hate to feel the rage of thousands of backers should a project fail to appear or live up to expectations, people demanding refunds when the money's gone, credit card chargebacks, and though we've yet to see it probably class action lawsuits if any sign of fraud can be found. I'm planning a campaign myself soon, and to be honest it's pretty terrifying despite knowing 100% I'll deliver. [Edit] Let's make that 99%, as theoretically I could die or be killed before finishing. :)

So basically what I'm saying is that Kickstarter is a lot of things to a lot of people, regardless of what it's "supposed" to be. My main concern about its long term success is resistance to fraud and exploitative campaigns. I'm sure it's a large target and I hope the Kickstarter team is vigilant in weeding such things out, hopefully before they're allowed to even occur.

Haran Sened
profile image
I don't understand why you're counting delayed games as failures. Your data definitely supports the notion that there are long delays, but concerning failures, you cite only 5% projects announced as cancelled\delayed. This perfectly matches my personal kickstarter experience - one of about twenty games I supported has been cancelled (Unwritten), and all of the others are in various stages of completion, almost all of them delayed by six months to a year. So, My conclusion from your data is that the success rate is a bit below 95% (probably around 90%), and that long delays are almost always to be expected, which heightens my confidence in crowdfunding games.

Un Subject
profile image
Only a small proportion of games get 'formally' cancelled. A lot of titles just stop updating or keep promising backers that the game is coming if only they wait.

But I can't count them as cancelled because there is always the chance that the developer is just terrible at communicating things to their backers.

I certainly don't count not formally closing your doors (and thus being able to better dodge people asking for refunds) as success.

JoseArias NikanoruS
profile image
I think that the beauty of Kickstarter is that the risk is "shared" (even by the developers) so you don't need to risk too much money. I'm from a third world country and I found that backing a project was a pretty small risk (I backed one of the classic first tiers that include the game and some digital goods). I feel that if you were to risk something like $1,000 you should be wary indeed (and publishers risk millions so they are quite paranoid); but risking something like $25 is actually a pretty cheap price to help someone fund their dream project and maybe get something in return. Your point is totally valid but from a pretty hard economical view, and Kickstarter is not all about getting something tangible from your money. We as a species have other values besides the material ones, and so as long as we want to help each other this method will still be viable.

Matt Allmer
profile image
I hope Gamasutra picks up where this blog leaves off. I'd love to know the percentage of undelivered projects that are likely, less likely or uncertain to be delivered. I imagine it would resemble the right half of the pie.

Michael Wenk
profile image
First of all, I think you're being harsh. Companies fail all the time. A company succeeding is a rare event. Take a look at what the WSJ has reported in the past. They say that 3 out of 4 venture capital backed startups fail. That's a 25% rate of failure.

Now the data you have indicates a 37% success rate. That's pretty good. In fact its so good, I think it may be the small amount of data you're seeing. Of course this is more like product launch, but the rule of thumb for that is 80% fail.

"I’ve seen some comments that this delivery rate is pretty good, or at least in line with expectations about software development. Maybe that’s true, but I wonder how all those software project failures are dealt with by the people who funded them. Formal investors are likely to have some kind of contractual protection against a project being a write-off and I doubt software developers often just get to shrug their shoulders, go, “Eh, we tried,” and leave a failed project without any consequence."

Of course there are consequences. Those developers are quite often made unemployed. Given the market for software developers, the developer is not likely to remain that way for long, but losing your job sucks. The founder level people may have other liabilities. They probably have personal equity in the company which if they get any of it back, it will be pennies on the dollar. Some founders have lost things like their houses on such things. There is also a lot of family lending too, and that sucks when you have to tell your Uncle Bob and Aunt Kim that they won't get their 10,000$ back when he thought, and he might never get it back.

Most people don't just shrug their shoulders and say we tried. They burn their souls out to not fail. And they still fail a lot of the time. I personally have worked some of my longest times when I worked for a startup. A "light" day was 14 hours. There were a few times that I worked continously for a few days, then slept for an hour or two, then did it over again. It is an incredibly wild ride. Of course there probably are some out there that don't try, but in my experience, that doesn't happen.

Ideas fail for any number of reasons and games are no different. And while I don't believe you can ever avoid failure, unless you are incredibly good AND incredibly lucky, you can mitigate it. The key is failing fast. The faster you fail, the less time and other resources you waste on the failure. So it is much cheaper to fail after 1 month, than to fail after 18 months.

Kickstarter is awesome for many reasons. However one of the best is it encourages you to fail fast. If you can't get funded, then you lose all of it. You have no ability to just take a partial funding and then try to make it up later. Of course it has negatives. It depends on the founder to budget adequately, and that's a very hard skill. Most tech people aren't really bean counters, and aren't good at it. I'd wonder how many of those failures are because they ran out of money because they underestimated their costs.

The biggest negative is the excitement the fact of being able to help get a game you want is kinda in the nose from the fact that you need to make sure you can tolerate the risk. If anything will kill crowdfunding will be that. And its bound to get worse once some high profile projects fail and fail hard.

Kickstarter enables a consumer to put his/her money where his/her mouth is and back a game they want to play. Its another medium for independent games. You can contrast this to the publisher model which only makes games that will at least bring back the investment in making them. This assumes that the publisher has enough room in their catalog to shoulder the risk of complete or near complete failure. That's the main reason why you see less diversity in what games are made, because the costs are getting such that its hard to shoulder risk.

So that's the main reason why Kickstarter is awesome. It opens up indie and indie like games. It promotes diversity in what games are made. Don't like the fact that you're playing GTA 10 ? Then you have the ability to kick in some money to get a "quirky" side scrolling game made. That is simply awesome. That assumes that the crowd can and does assume the risk of losing the money they put in. However, that is how crowdfunding can work. Its much easier for 10,000 people to lose 70$ each than 10 people to lose 70,000 each.

Andrew Wallace
profile image
Regardless of my (and most other commenters, it seems) difference of opinion about the spirit of KS than the author's, this is a very valuable analysis and I think we're all better off for having it. Thank you.

I suppose it supports my feelings that Kickstarter projects should not be seen as a type of investment as much as it should just be an attempt to support a creator, not unlike Patreon. I first backed Broken Age because it was exciting and I wanted to be a part of it, so I wasn't disappointed when I really didn't like the game at all- something that (rightfully) isn't taken into account by this analysis. Most of the other projects I've backed are those of people I knew either directly or indirectly, and I wanted to support them out of friendship, rather than a deep desire to own the game.

Un Subject
profile image
No problems at all and I'm glad you found the figures useful.

James Yee
profile image
I LOVE having numbers like this, thank you for doing the hard work for me! (Again, as I think I' mention this every time you post numbers. :) )

Numbers like these are why I try to educate creators and backers about the perils and rewards of Kickstarter. Thank you for the added ammunition! :)

Un Subject
profile image
No problems at all.

It might be that the delivery rate goes up over time, but I also think it worth knowing that the majority of successfully funded Kickstarters are yet to actually deliver their games to their backers.

I wonder about the continued appetite that gamers will have for crowdfunding games if enough of them don't actually deliver what they paid for.


none
 
Comment: