CLICK HERE TO LISTEN TO THE AUDIO RECORDING OF THE CALL

There is a long history of using crowds to solve problems. For example, in 1707 part of the British Fleet sunk. The government offered 20,000 GBP to improve longitude navigation. Isaac Newton is a genius and an expert and he felt the only solution was astronomy, and held back non-astronomical solutions. The crowd gave ridiculous and inane solutions of course (telepathic dogs!) but also something that worked. A cabinetmaker named John Harrison invented a chronometer that worked.

A Challenge Lakhani ran for NASA – longerons on space center which collect solar energy – how to maximize the time they are in the sun without exposing them to the risk of bending. $30,000 prize. 459 people submitted 2,000 code submissions. Competitors from Italy, China, Canada, Belarus, etc. How did the code perform? Lots of people exceeded the NASA solution. The NASA solution was far, far costlier.

Have mostly done computational contests, but also engineering, and scientific hypotheses.

When we are innovating, at the boundaries of knowledge, we are faced with uncertainty. That’s a huge challenge. There is also a labor market problem — no matter who you are, most of the smartest people work for someone else (Bill Joy’s Law). And our models for organizing creative efforts are changing — we had Encyclopedia Britannica, then Microsoft Encarta, and now Wikipedia. That’s a change in the distribution model, sure, but also the creation model. It is not experts creating knowledge — anyone can participate. And it’s distributed. Some users only correct grammar or add images.

Both Encarta and Britannica are dead.

Crowds can be organized into communities or contests.

• _Why contests? When you need diversity and experimentation; and you aren’t sure what approach will work.
• _Why community? When you need cumulative knowledge building and aggregation of diverse inputs.

Why do these work? In part due to sheer volume. Innovation is about drawing from a system of values — if you draw once, you get the average. But if you draw a lot, you find an extreme value. The LILA Confidential Page 2

extreme values are what you care about for innovation. Firms say, “we have smart people, so our bell curve is shifted to the right.” That may be true, but you will still miss out on the tail.

Collaboration has also been huge in history — for example, the invention of flight. Open sources software is now the bleeding edge of collaboration. It has become the standard way to produce software for many companies. Google, Amazon, Twitter, Facebook… a lot of this innovation has been driven by open-source. Apple is seen as closed but is also highly reliant on open source.

A test at Harvard Medical School — they feel like the smartest in the world of medicine. There is more output in journals from HMS than Canada and Germany combined. Does Joy’s Law apply there too?

Did a crowd study on genomics. 122 coders and 654 submissions. Exceeded the state of the art by 10^2 – 10^5! Made the problem more abstract, just about math, rather than biology. Made it easier for everyone to participate.

Even if the average solution is worse than what traditional experts do, the crowd has much more diversity and breadth… you can draw from the tails.

Research suggests that it’s unconventional individuals who win in contests – people on the outside who can bring in their knowledge from other arenas.

Why do people participate in contests?

• _Extrinsic values – cash, prestige, job market signals.
• _Intrinsic – fun, enjoyment.
• _Prosocial – community belonging and identity.

Similar reasons for community collaboration. By definition, most people lose contests. So it’s really about these factors that drive participation.

When don’t crowds work?

• _When there is no governance (formal or informal) – e.g. Reddit trying to solve the Boston bombing.
• _When incentives mismatch – e.g. the Archon Genomics reward was really too low, because you could start a company doing the same and make far more money.
• _When there is task interdependence – well-defined and separable tasks work best.

Question to end with: How do we get our internal staff to use crowds and how do we ‘crowdify’ our internal staff?

LILA Confidential Page 3

Question to end with: How do we get our internal staff to use crowds and how do we ‘crowdify’ our internal staff?

Insights, Connections, Actions, Puzzles

Marga – LILA: what type of culture has to exist in order to support integration of ideas rather than threatening the internal organizational experts?

I have a student who studied how people respond to this. There are some who are extremely worried, identity threatened, worried they will lose their job (“the lab is my world”) and others who embrace it (“the world is my lab”). The second type can get ideas from around the world for their problem. Huge benefit. And the act of breaking down the problem into clear tasks and then evaluating the solutions, makes a huge difference. Goes back to first principals, makes them rethink all their old assumptions.

Implementing crowd-sourcing does require top management buy-in; or it can fail. At NASA and HMS, there’s enough inside the organization to make it work. The whole organization can get involved. But management needs credibility — we will look at all these solutions and implement them if they are worthwhile.

Josh – Sapient: You have some great examples; do you have examples where commercial organizations have used crowd solving internally? I think that internally would be an easier sell at most companies.

Yes, there are plenty of examples — such as Qualcomm. But an issue is that managers don’t want to let go of authority or their people. Managers think, “If I post my problem to the crowd, what does it say about my team and my team’s ability to solve the problem?” And if someone creates a solution, they wonder, “what does that say about how much free time I have? Shouldn’t I be doing my paid job?” Management needs to follow through with allowing time and effort on these initiatives.

When I was working at BCG, we were building our internal wiki and wanted to do internal knowledge generation. I wanted it to be anonymous. But they said no. All the famous partners wrote these great posts to start things up…. No one participated. No one wants to approach the famous partner and say “I disagree” or “you are wrong.” The hierarchy silenced people.

TJ for ETS: Link to cognitive entrenchment (Dane’s work) – seems like contests could be a counter to the cognitive entrenchment. Is there a link?

Well, academics are famous for not talking to each other! But, yes, solvers are most frequently outside of the home domain. And problem articulation is huge (Markus Baer’s work). People engage in local search – what I know, what my friends know, etc. Search in a very small locale in knowledge space. That’s a weakness that contests get around. A great research question is if people who use contests would be less entrenched.

David Peterson Google: the distributions and bell curves. Where do they come from?

Some of those images were just impressionistic, but I can send you the real data. Very quickly you see that the solutions come from people with zero background in the field. Why? Great work done in sociology of science literature — problem framing. These frames are useful initially but can stop further progress. Problems can be represented from multiple perspectives. e.g. The brain — is it biological, chemical, electrical — one of those frames may not be appropriate to solve the problem, but if you are stuck in one, you might not make progress.

David Peterson Google: Liz Wiseman in her book Rookie Smarts suggests you should get a non-expert to work if you have a tough problem!

Not quite. One non-expert is not enough. You need numbers (more shots) and diversity (I can’t say in advance which combo of knowledge will solve the problem, so diversity helps). Inside the firm, it’s expensive to put 100 people on the same problem. That’s why contests and community are so useful — it’s not just the payment that people are working for. It’s the other benefits.

And it’s not like the winners are non-experts. They are at the top of their games in a different field.

Marga – LILA: As we explore flexpertise, we are curious about how it is developed. With crowd sourcing it seems like we are still looking at experts rather than novices and all coming from diverse fields.

Right. It’s similar to the cross-functional teams literature in management. It’s about diverse knowledge sets attacking a problem. The challenge is exposing the problem to diverse knowledge sets and having an incentive to participate. (Marga: this seems related to problem articulation that Marcus Baer shared with us on the November Member call)

Case on Siemens and open innovation — Had great results but want to stop doing it! Why? Engineers don’t feel good when someone else has a better solution.