It seems like apps contests and apps challenges and hackathons sponsored by governments have been all the rage over the last couple of years.
They were kinda cool when they were new, in 2008, such as Washington DC’s original “Apps for Democracy“. Vivek Kundra, then CTO of the District, valued the apps developed there at $2.3 million, although how he arrived at that number is unknown.
But now apps contests are not just overdone – there are just so many of them – but actually are becoming counterproductive, turning off developers and governments. In fact, they are stupid.
Why?
First, the data sources are not standardized between different cities, counties and states. This leads to myriad problems.
One of the great breakthrough ideas in the government of the District of Columbia in 2008 was the concept of a “data catalog“. Governments have always had masses of data which they crunched to produce reports and insights and to support policy development. But they also kept such data under lock-and-key. The District’s government made some of those datasets open and available for citizens, academics, researchers and app developers to use. Kundra’s successors in D.C., Bryan Sivak and Rob Mancini, extended the District’s data catalog to 507 datasets. Kundra ported the idea to the federal government with data.gov, and a whole industry – including Seattle startup Socrata, is now built on opening up data.
But very few “normal” citizens are interested in this stuff. Many of them don’t have the skills or interest to spend a lot of time loading thousands of rows of data into a spreadsheet or some statistics program and crunching the data. Most non-policy-wonk citizens, if they have an opinion on a policy issue at all, have that opinion based on a logical or emotional basis and don’t generally want it polluted or confused by facts. As an example many people feel “crime is rampant in our city, we need more cops”, when, actually, violent crime has dropped steadily in almost every US City for the last 20 years.
Nevertheless, the open data catalog is a brilliant idea because it allows developers to build apps using that data and display it in new and interesting ways.
My favorite example of this from the Apps for Democracy days is the “stumble safely” app.
This app used the crime dataset from the DC government. It took your location, as determined from the GPS on your smartphone, along with the time of day, day of the week and other information. It mashed that data against the crime dataset. And it showed the safest path for you to “stumble home” from a night of eating or drinking.
It no longer works.
A primary reason: “stumble safely” used the DC crime dataset, but was useless outside the District of Columbia. Few other cities – even today – have put their crime datasets in an open data catalog. And the formats of those datasets vary widely from place to place. So “stumble safely”, written for the District of Columbia, won’t work anyplace else unless it is modified for each individual city or government.
This is a common problem with almost all government open data on the web.
In fact, the only commonly used data which is standardized – as far as I know – is transit data with the GTFS standard promulgated by Google. Transit data is widely standardized because transit agencies are like private companies. They have to market and sell their services – ridership on buses and trains – or they will run budget deficits and cease to exist.
Few other government organizations are in the same situation – there is little incentive for police departments or fire departments or building inspectors or water departments or electric utilities to standardize their datasets and make them open. Indeed, most of them fear (wrongly so), that exposing data will open them to criticism for problems with the data or bad comparisons with other agencies, e.g. being less efficient or more expensive at delivering service than the same agency in a nearby city. And every agency will say “we don’t have the time or money to standardize our data.” It takes strong leadership (aka “arm twisting”) by an elected official or extraordinary leadership by a CIO to push an open data initiative forward.
Yelp, Code for America and the City of San Francisco are trying to standardize restaurant inspection data with the LIVES specification, an effort I applaud. But I predict this will be a slow slog – slow adoption – for the reasons mentioned above.
A second major reason apps contests are stupid is the lack of monetization. Apps developers need to eat, too, and put a roof over their heads and a Porsche in their garage. They’re all looking to create that “killer app” which will be downloaded a million times at a $9.99 per download.
But how many apps created from hackathons or apps challenges or contests are monetized? How many of them are still active and downloaded and used?
My answer: few, very few.
Stumble safely, alas, is dead (just click on the link here). Most of the apps on Apps for Democracy and other apps contest sites are dead.
A few are living and continuing to develop because their creators are passionate about them (Living Voters Guide by Travis Kriplean) or they do have a monetized component (Seattle Emergency Radio by Brian Adams, monetized via advertising).
Another potential avenue for monetization is for a City/County/State government to take over the use, care and feeding of an app after an apps contest. That sometimes happens, but often can be a dicey proposition. Brian Ferris, a Ph.D. student at the University of Washington in Seattle, wrote One Bus Away, an extraordinarily innovative transit application to track, in real time bus arrival times without using GPS (since Seattle buses at that time had no on-board GPS). It has a 100,000 users each week. Brian now works for Google in Zurich, and the area’s four transit agencies are trying (and fighting with each other) on how to keep the app going.
Another possible solution to the monetization problem is offering prize money to encourage participation such as the $100,000 offered this summer by Battle Hack. When we did the evergreen apps challenge last year in Seattle, the three governments involved were able to offer some prize money funded by federal grants and local innovation funds. But that’s rare – taxpayers will quickly wonder why we’re offering money to hackers when potholes are still jarring their cars.
One potential exception to the hackathon/monetization problem is New York City. New York City has its “Big Apps” competitions, 8 million people, hundreds of thousands of businesses and innovative Mayoral leadership with Michael Bloomberg. It represents an “apps ecosystem” which might survive and thrive despite the other hurdles.
If most apps contests are stupid, where do we go from here?
I’m not sure how to get datasets standardized across 3000 counties and 18,000 or more cities and 50 states. Perhaps private companies such as Socrata, a premier provider of data catalog services such as data.gov, or a company which has monetized data such as CrimeReports.com can figure that out. Or – and this is a long shot – perhaps the Mayors’ Innovation Task Force of the US Conference of Mayors would take this on, but I’m not holding my breath.
Perhaps an innovative state government will take leadership and build a data catalog with its cities and counties and standardize the data on the catalog. This presents a great economic development opportunity – think of all the startup companies in that state which might develop statewide apps, or mash up the government open data against data from other sources such as the census or sales data from, say, Amazon.
The second problem – monetization of apps – will be hard to solve unless and until data is standardized so a single app – a crime reporting app, for example – can be downloaded and used across an entire state or the nation with the potential for millions of users.
In the meantime, we’ll have to rely on the kindness of friends to the government community like Travis Kriplean, innovative leaders like Jay Nath in San Francisco and Mark Headd in Philly, plus the ongoing attempts to standardize data like GTFS and LIVES to carry us forward.
But proliferating these stupid apps contests sure ain’t going to do it.
the Chief Seattle Geek blog 
Amen. Thanks for the truth-telling, Bill. Another point: while there definitely some good open data apps out there (Living Voters, One Bus Away), apps contests didn’t get them built.
Personally, I think government itself is the number one “customer” for open government data–and that the most transformative “apps” that could be built (but mostly aren’t yet) are going to be those that help government do a better job of governing by using its own data better.
That’s a great comment, Jon, and good food for thought. Governments still tend to operate in silos where one department (e.g. social services) doesn’t have access to data in other departments (e.g. police or courts). Transformative apps help break down those barriers.
But I also think there are some cool Gov’t to Citizen apps like One Bus Away, Seattle Traveler’s Information map etc. Question would be how to make them more interactive.
Food for another blog post!
Thanks for the Impressive write up on a lot of stuff that has been scathered across the twitter Sphere.
Standardisation is a key issue, however the question is whether governments can move quickly enough to make this happen.
In the Netherlands there is a strong developerscommunity which engages through topic networks such as education and culture. We have experienced that App competitions are suitable for gathering a lot of working examples of possible directorions.
Goverments in the Netherlands are still in the stage of convincing collegues internally and App competitions most definitely helps this process. At the same time it brings together data owners, developers and domain experts, which provides a sustainable basis for further work.
From those topic networks API’s – and insight – arise which actually help appmakers to make the next step into entrepeneurship. This pace goes way beyond what (at least our) government could achieve by itself.
If groups themselves can build awesome API’s, my question would be how can we help people to turn great Apps into succesfull businesses? I reckon this is the most difficult part, as the average app-team often doesn’t have startup skills and mindset (yet).
Some good comments here, Lex. it is interesting that you are still able to engage a significant developer community on these topics – sounds like it is almost “brainstorming” about apps. I think there is some fatigue here in the US in the developer communities about all the various hackathons and contests.
Thanks for responding!
I do not disagree with your statement that app contests and hackathons are often over-sold, though I do think that their greatest benefit is often excitement and community engagement, rather than the more obvious deliverables of applications. Further, I agree that standards are an important component of how to improve the ecosystem. However, I’d like to take issue with the idea that there is only one successful data standard, GTFS, and one other in the wings, LIVES. This is an area that has received and continues to receive a lot of attention with many good people doing excellent work. A few to which I’d draw your attention:
* Voting Information Project (VIP) – https://votinginfoproject.org/ – extremely successful effort led by Pew Charitable Trusts and includes almost every state in the U.S.
* Open 311 – http://open311.org/ – used by many local governments and is beginning to pay interoperability dividends
* Open Elections Project – http://blog.openelections.net/ – new effort to assemble election results data
* Open Civic Divisions (OCD) – https://github.com/opencivicdata/ocd-division-ids
* House Facts Data Standard – http://housefacts.me/
The Sunlight Foundation is also leading data standardization efforts around elected officials, legislative bills, and other government transparency topics.
There is also an alphabet soup of other standards that have helped support interoperability with geospatial data and are used broadly by many government agencies, including:
* FGDC metadata
* GeoJSON – simple vector features
* CSW – data catalog services
* WMS – web mapping service
* WFS – web feature service
* WPS – web processing service
* WCS – web coverage service
* WMTS – web map tile service
* KML
to name just a few that are in widespread use.
Governments, working on their own, rarely get standards right. The most successful standards are often led and developed with support from a) commercial firms; b) non-profit organizations; or c) consortia of commercial and/or non-profit organizations. So commercial firms play an important role here. Nevertheless, to say that we’ve only got one successful standard and so need to turn to commercial products to become de facto data standards would, in my opinion, be a mistake. I think the most successful standards have been those developed in collaboration with local government (often led by commercial or non-profit organizations) with the explicit purpose of creating a broader standard. That’s how GTFS, LIVES, Housing Facts, Open311 and others were created and it’s the template we should follow for other domains.
I disagree that the most successful standards were those developed in collaboration with local government: GTFS and Open311 are the only examples. LIVES and HousingFacts are still “wait and see” – LIVES has two adopters, and HousingFacts has one or two. Not enough to write home about!
I think it’s a good approach, but the most successful approach, in terms of the number of standards it produced, is the usual: “get specialist, motivated stakeholders together to develop a data specification by rough consensus via a public process, make participation open to all, and put the spec through rigorous review and implementation steps.”
That’s how most of the alphabet soup of standards were created and maintained.
Open311 is suffering from its lack of a governance model, with cities taking it in diverging directions. GTFS is following a “benevolent dictator” model, with many transit agencies being frustrated by Google’s refusal to adopt features that are favored by its community of adopters, but that are not of interest to Google’s products. Both Open311 and GTFS has significant issues in terms of governance. I would not recommend their approach as a template.
Thanks for the thoughts on data standards, James, although see also Robert Cheetham’s comments above, which I haven’t analyzed completely yet. Seems like you basically agree with me about the lack of data standards holding back the development of apps. And I sure agree with you that it will be hard – probably impossible – for such standards to be developed by governments themselves.
Bill, I was responding to Robert, so of course I’ve seen his comments 😉
But to respond to your blog post, I think you misunderstand the motivations of the organizers and the participants of many of these hackathons and app contests.
Some app contests and hackathons aim to produce long-living apps that provide continual value to the community. The organizers, unless they structure the contest/event carefully, are very likely to be disappointed for the reasons you gave.
Focusing on hackathons first: The vast majority of hackathons (in my experience) are about bringing people together to exchange ideas and build community around open data and civic technology. The apps created are secondary. Such events make it easier to lobby the city to open up more data, by demonstrating the demand. They help people find collaborators for longer-term projects that will plan for sustainability, and help advance community projects that need more hands and feedback. The #1 goal is community building, and most hackathons are good at that.
But why do most developers even attend hackathons? Because they’re hackers. If they weren’t hacking on city data on a weekend, they’d be hacking on some other pet project. A hackathon is an excuse to go out, meet new and old friends, in a (usually) casual environment surrounded by people who want to be there. For most, it’s about having a good time. Most of the projects will not survive the day – but that’s normal for a pet project. It was fun to work on at the time, and the developer likely picked up some knowledge or skills. Not all activity is motivated by money or long-term glory!
App contests, on the surface, seem to be about building lots of apps. For some app contests, there is nothing under the surface, and you are right to criticize those. But when I’ve spoken to organizers in my region, it is more about getting people talking about open data, working with the open data, giving feedback to the city about its open data portal or open data license, and, as above, building community. An app contest is just one small part of a larger open data strategy. Even if all the apps are dead in a year, the contest will have still fulfilled a lot of other, larger goals, like increasing awareness and use of open data.
TL;DR: If app contests and hackathons were only about the apps, your article would be on the mark. However, they are not, in most cases.
First, I think the mechanics of how a standard is actually constructed vary quite a bit, but I think “interested stakeholders” frequently does include local governments.
Second, are you suggesting that VIP is not a successful standard? What about the widely adopted OGC standards I listed?
Third, I was specifically responding to Bill’s assertion that we should simply turn to private firms as the developers of standards. While I think commercial firms have a role, I don’t think this is a good approach for the development of standards.
Fourth, the “get specialist, motivated stakeholders together to develop a data specification by rough consensus via a public process, make participation open to all, and put the spec through rigorous review and implementation steps.” has frequently resulted in bloated, over-engineered standards that aren’t adopted by anyone. In the geospatial world, this has include GML and a pile of other OGC standards (I listed half a dozen that have been successful, but OGC has adopted a couple of dozen others that have not gone anywhere. I don’t think it’s necessarily a recipe for success.
I agree that interested stakeholders frequently includes local governments. Open311 and GTFS got that part right, and that part is worth repeating. However, they didn’t get other parts right, and that is why I would not recommend them as “the template” to follow. To clarify, I’m disagreeing with these statements:
“I think the most successful standards have been those developed in collaboration with local government (often led by commercial or non-profit organizations) with the explicit purpose of creating a broader standard. That’s how GTFS, LIVES, Housing Facts, Open311 and others were created and it’s the template we should follow for other domains.”
I don’t know if VIP’s development process was comparable to those of Open311 or GTFS; you didn’t make any claims about VIP in the part I’m concerned about. But to answer your question, VIP is certainly a success!
OGC standards did not follow a process like Open311 or GTFS, and definitely not like LIVES or Housing Facts, so my comments are not about the OGC standards. FWIW, many OGC standards are a success IMO.
I’d agree that private firms should not be solely responsible for standards; however, as important stakeholders, they are heavily involved in many of the most widespread standards, like video formats, HTML, etc.
In terms of “bloated, over-engineered standards that aren’t adopted by anyone” – I think every process has that risk. Pairing a commercial or nonprofit organization with a local government does nothing to reduce that risk. The two things that reduce that risk are, in no particular order:
– a good editor/chair for the specification, who can keep the standard on track and cut bloat diplomatically
– shared values like those of the IETF of “rough consensus and running code” to guide the community
Anyway, in terms of what’s lacking in the process you propose as being the “template” is any thought towards the governance model for the spec, which is what I devoted about a third of my original comment towards.
Apps contests exist to enable self employed coders monetise their ideas? Is that really likely?
Well, that’s a good question, Michael – why do apps contests exist? I would have answered “to get apps usable by citizens to better understand what’s happening in their community and improve their quality of life” and to improve government services. I’m not sure why developers would ever code such apps unless for money or out of a sense of civic duty, and the latter implies they have a “day job” to pay the rent and put food on the table.
Let’s raise the bar to grand challenges and longer term impact. How can crowdfunding help to take seed apps to the start-up phase of small business? Here are a few ideas for apps for humanity: http://storify.com/meganesque/apps-for-humanity/slideshow The medical and defense industries are ripe for disruption, and privatized space exploration is well on its way to viability.
Crowdfunding start-ups or individuals writing apps is an idea I hadn’t considered, Megan. Also I don’t know that it has been used by governments who operate apps contests. This is an idea I’ll explore more and provides grist for a follow-up blog post.
Bill – This is prescient and long-overdue post about the open data app ecosystem. David Eaves wrote a similar post on the need for data standardization in order to scale the open data ecosystem. He used the shipping container to illustrate the same idea you just articulated: http://eaves.ca/2012/07/12/containers-facebook-baseball-the-dark-matter-around-open-data-iogdc-keynote/
Our approach at Socrata has focused on different facets of this problem like making expressive, well documented APIs, to reduce friction for developers. We also strive to deliver a user experience that allows non-technical (“normal”) folks to make sense of the data on an open data portal and relate it to their own context. For example, making it easy to create a map of crime in one’s neighborhood or filtering crime by type. This is an ongoing quest to make government data as easy to use as anything on Yelp, AirBnB or Amazon.
With respect to apps, our experience has been that app contests and hackathons are a very necessary but not sufficient. They are a productive, creative forum that brings people together to spark ideas that connect innovation to real-world problems. Cities like Chicago have done a great job of fostering that developer community in order to iterate on those ideas and produce apps that actually work (http://opencityapps.org/). The missing link -as you pointed out- is app portability, and a business model that allows these innovations to become more mainstream, and therefore sustainable. Lack of standards is part of the problem, but you brought up other key issues we see every day, like procurement hurdles and immature monetization models. That’s in addition to high support requirements and a legal vacuum in terms of how to operationalize these apps.
A few organizations and passionate individuals are working on these issues. Code for America and Socrata, for example, have collaborated on some key data standards like LIVES for restaurant inspections (http://www.socrata.com/blog/lives-standard-for-health-inspection-data/) and most recently Residential Building Inspection Data (http://housefacts.me/). More are coming.
Bill – we hear you loud and clear. We are starting to focus on this problem in a very holistic way and invite like-minded people to join us in this effort.
— Saf Rabah, VP of Products at Socrata.
This is a good reply with a number of good observations and ideas, Saf. Socrata clearly has been a leader here and is a “light at the end of the tunnel” to make apps contests more useful and productive. Thanks.
Bill I posted this on another blog where this ran but wanted to add to the conversation here. You make great points that unfortunately have little to do with whether app contests are valuable including:
1) Data needs to be standardized — This would help developers scale their projects, whether an app contest is involved or not. Why this makes apps challenges ‘dumb’ is left unexplained and is a stretch.
2) Monetization — Developers aren’t just motivated by money, but working on interesting stuff, having impact, getting exposure, and developing skills. Yes, cities and developers want businesses to come out of this. You point out that BigApps (3 of 4 years of which we powered at my company ChallengePost) is a successful example. It’s because we and the city got VCs on the panel, got entrants mentors, opened up tons of data, promoted like crazy, and most important let developers know it was a real initiative rather than a one night stand.
Rather than saying apps challenges are ‘dumb,’ how about looking at why New York’s example is successful and spend your efforts telling people how to do it right? If you truly care about this stuff, which I believe you do despite the cynicism, you’ll find it useful.
3) Sustainability — While monetization is one part of making apps sustainable over time, there are two points you miss about devs:
1) Developer projects ALWAYS are more likely to be experiments than to live on, whether it’s a business attempt or a submission to a challenge. Hackers want to work on cool stuff. If you make the portfolio big enough, some huge stars will shine through.
2) Show developers you mean it. The number one thing you as a city CIO should be showing developers is that you don’t want to waste their time. You’re in this for the long haul, you’re working on your data, you’re listening, you’re doing outreach and opportunities for them (yes including challenges which drive more usage and exposure than anything else by orders of magnitude).
Happy to chat more, hopefully for a future article in which you explain how to do it right.
Thanks, Brandon, for a respectful and informative reply. You are absolutely right – I am preparing a blog on how to “do apps contests right”, and it will be based, a lot, on this good material you’ve provided.
Pingback: The Capitals™ – Capitalists' Magazine | 資本家札記 | Why Silicon Valley can’t be copied anywhere else [Links]
Pingback: An inclusive community-based civic hackathon in Western Mass | Molly McLeod
Pingback: America…we have a problem. | thedigitaldruid
Pingback: Hacking the hackathon | shauna gordon-mckeon | blog
Shauna Gordon-McKeon has some thoughtful comments on improving hackathons here: http://www.shaunagm.net/blog/2013/10/hacking-the-hackathon/
-bill