- For The Better
- Posts
- Startup Lessons Learned Blog RSS
Startup Lessons Learned Blog RSS
Solving the Pipeline Problem
Solving the Pipeline Problem by Eric Ries
Is this email not displaying correctly?View it in your browser.
Solving the Pipeline Problem
By sarahm on Nov 29, 2012 09:14 am
Thispost was co-written by Sarah Milstein and Eric Ries, co-hosts of The LeanStartup Conference.
It’swell-known—and we ourselves have been publicly frustrated—that white mentend to dominate the speaker rosters for tech and entrepreneurship conferences,not to mention the portfolios of many entrepreneurship programs. Conferencehosts, VC’s, and others often attribute this to a "pipeline problem," the ideathat there simply aren’t enough qualified women or people of color who wanted to or were qualified to participate. So we were proud earlier this weekto announce our program for The Lean Startup Conference, which comprisesapproximately 40% women and 25% people of color. We still have room to grow,but this is a significant improvement over last year’s conference, which hadalmost none of either. Our approach was deliberate, and we want to shareit with you, in the hopes that you can replicate it for other conferences andfor processes like hiring where equity is important.
We have argued previously that the pipeline problem may be caused by theselection process itself. If under-represented groups have a reasonable expectation ofnot being selected, it’s perfectly reasonable that they therefore don’t apply.After all, if you were thinking of submitting a proposal to present at aconference that had a record of not choosing people like you—if you weren’tsure that your proposal would be assessed on its merits or that you’d bewelcome at the event—why would you spend the time applying? The problemcompounds itself: because women and people of color are not often speakers,we’re less aware of them, and we’re less likely to think of them for our ownevents.
There’sa solution that addresses these issues: meritocratic selection. It’s not a gameof quotas; it’s quite the opposite. Indeed, we picked the speakers we thoughthad the best stories and would be the most engaging presenters. We didn’t ruleout any candidates for being white or men, and we didn’t favor women or peopleof color. Instead, we used a handful of principles to guide us: transparentprocess, blind selection, proactiveoutreach and enlisting help. Here’s how they played out.
Transparent process. Over the summer, we made a bigdeal about how we wanted to find speakers based on merit (i.e., the greatstories they had to share) rather than on their proximity to us. In August, weposted a call for speakers, asking people to apply. We explained that we werelooking for speakers based on what they knew rather than who they knew. We alsonoted that in the past, Eric had mostly drawn from a pool of people he had worked with directly, which meant that in 2010 and 2011, this conference had had almost nospeakers typically under-represented at tech events. We received nearly 200applications, more than half from women and about a quarter from people ofcolor. That included a notable number of black people, which we call outbecause we’ve received very few applications from black speakers at otherconferences we’ve run. Similarly, quite a few men and women over the age ofabout 50 applied to speak. One person talked about the importance of representingpeople with disabilities. In addition to being pleased by the demographicrange, we were stunned by the consistently high quality of the proposed talkideas across groups, as the previous open calls we’ve run have brought inrelatively few great speakers.
Here’swhat resulted from that call for speakers: from the pool of applicants whom wechose for the December 3 conference program, more than half were women. Fromthe pool of applicants whom we chose for the December 2 Ignite: Lean Startup program, more than a third were people of color. And we had to makesome heartbreaking decisions to pass on a number of speakers just because wedidn’t have enough time on stage for everyone. Taken together, this shows that when you have a broader group fromwhich to draw, there’s a good distribution of speakers; not all—not evenmost—of the good speakers are white men. The common conference organizer’sargument that we don’t know any black people in tech or that women didn’t applyto speak just doesn’t hold up.
Didemploying a transparent process make a difference? The August call forproposals described above was actually our second attempt to change the makeupof our applicant pool; the first attempt failed. In June, we posted our first call for speakers, asking people to nominate others they thought would be goodspeakers. We noted that we were particularly keen on learning about women andpeople of color who might be great speakers but weren’t on our radar yet, butwe didn’t say anything about what had happened in the past or how we weretrying to change it. We received about 35 nominations. Although some were verygood, just about 10% were women and almost none were people of color. Everypiece of data we have been able to gather on conferences says that 10% is thestandard rate at which women will apply or be nominated and that very fewpeople of color will be among these pools.
Let’sdwell on this for a moment. By using principles of meritocraticselection—i.e., being explicit about our desire to find great speakers whom wedidn’t know, and by being honest about the process we’d used in the past—wecreated an atmosphere in which a much broader range of strong speakers feltinvited to participate.
It’salso worth mentioning that we knew we’d hit on something important not justbecause we got such different results in the second round, but also becauseapplicants told us so. We got a lot of comments like this:
“ILAUGH when you say, ‘under-represented at a tech conference,’ because had younot presented such a compelling invitation, I would have never even dreamed ofapplying for a position of a speaker.”
“Themain reason I'm applying is because I have a huge amount of respect andadmiration for your efforts to reach out to a new circle of contributors.Anything I can do to support you, whether it's speaking or just behind thescenes, is personally very worthwhile to me.”
“Thanksfor opening this up to the non-famous. I think there are lots of great storiesout there to be told.”
Blind review. It’s well documented that whenpeople know the sex or race of a job applicant or the main character in a story, they generally assess the person’s qualifications and performance morefavorably if the person is male or white. Because we all have internal biases,everybody does it, including women and people of color. We wanted to eliminatethat bias as best we could. So we asked applicants to submit some writteninformation, along with a video, and we made the first cut based on thewrite-ups, which we read without checking names or other identifying info.
Didusing blind review make a difference? By reading the applications, we quicklyeliminated people who weren’t a fit for the conference because their topicclearly wasn’t on point for our audience. That was a small group of people, butthe distribution was broad. Beyond that, blind review didn’t make a bigdifference. Why? We asked for relatively little info in writing, both becausewe wanted to hear directly from speakers (not their PR people) and becausevideo much better represents the medium we’re trying to assess (often, peoplewho can write a nice description of their talk can’t deliver the presentationwell, and vice versa). So video was much more important to us than writing, butwe couldn’t assess it blind. It’s worth considering whether there’s more we cando with this tool in the future.
Proactive outreach. A very common way thatconferences build a program is to brainstorm speakers, come up with a bunch offamiliar names, and then notice late in the game that you have almost no womenor people of color (at which point, you might scramble to find some or justcomplain that you don’t know any). Our initial brainstorm wasn’t magicallydiverse; in fact, it included an overwhelming percentage of young white men.But we immediately—during the same meeting—started digging deeper to think ofpeople we’d left out, and when we came up with less-top-of-mind candidates fromunder-represented groups, we reached out to them early and often. We kept thatup throughout the whole cycle.
Didproactive outreach make a difference? As we noted earlier, Eric’s previousconferences rosters were based almost entirely on people who were top of mind,and they skewed heavily toward white and male.
Enlisting help. There were two primary ways weasked other people to help us. First, when we had a call for participation, weposted it to mailing lists with lots of women and/or people of color, we askedour friends to do the same, and we approached organizations like Women 2.0 to publishor cross-post our info on their blogs and Twitter accounts. Second, when weasked other people for speaker ideas or when we formed our few panels withmoderators, we told them about our search for under-represented speakers, andwe said that we’d appreciate their support.
Didenlisting help make a difference? It can be scary to ask for this kind of help,because you don’t want to suggest that you’re interested in diversity overquality. But we found that people were consistently, surprising open to workingwith us on this issue. And it did make a difference: When we forgot to mentionto people that we were looking for speakers we didn’t already know, we almostalways got back suggestions for more white men who’d be great presenters. Whenwe included a note about our efforts to reach farther, we almost always gotback suggestions for women and people of color who’d be great presenters. Inaddition, we heard regularly from people who’d found out about our call forspeakers through Women 2.0 and other mailing lists we worked with.
--
Our four-prongedapproached generated really encouraging, even exciting, results. We’re furtherencouraged by the fact that we didn’t execute this plan perfectly, so there’sstill room for improvement. Here’s what we we’ll work on next time out.
Coordinating our approaches. As noted above, in addition toour public call for speakers, we also reached out to people we did already knowin the Lean Startup community whom we thought would have really compellingstories. Although this group included some women and people of color, it had abig percentage of white men. Because we started this process before the publiccall, and because we didn’t know if our public call would work, we quicklyfilled a number of speaking slots with these great candidates from the group wedid know, and we didn’t leave as much room as we could have for terrificspeakers new to us. We’ll use a hybrid approach again, but next time, we’ll trustthe open-call process and be more deliberate about the distribution ofspeakers.
Getting beyond the easy wins. Related to the issue above, it’sjust plain easier to put people onstage when you know them personally or byreputation. There are a few reasons for this. First, like most conferences, wewant to put well-known people onstage, because they’re a draw. So when somebodyhas a big name and is available, we we’re more likely to consider them for ourprogram, even if their story is only tangential to our angle. For instance, wewere interested in having some well-known entrepreneurs join us, and we werewilling to let them talk about their experiences generally rather than havethem focus on Lean Startup principles. Unsurprisingly, all of the people whofit into this category were white men; women and people of color withcomparable stories but less prominence didn’t get this kind of consideration. Correspondingly,we had much stricter standards for people who were less well known, insistingthat they have a direct Lean Startup story—and if we didn’t already know them, wewere less trusting that their stories were really good examples of Lean Startupmethods. (For the record, we had so many good Lean Startup speakers, we didn’twind up taking anybody with a general entrepreneurship story. But we definitelyconsidered a bunch of them.)
Additionally,we found that when we already knew a speaker, it was easier to find good slotsfor them in the program because we were more aware of how they were using LeanStartup principles. For instance, if we were looking for somebody from ahardware startup to tell a great pivot story, we might have had sixcandidates…but it just so happened that the one guy we knew from that group hadan even better story about innovation accounting. Because we were familiar withhis company generally, we knew his other areas of expertise, and it wastherefore easier to put him in the program. It was harder to find the variousangles that would work from people we didn’t know.
In otherwords, between knowing their stories less well and bringing our biases to bearin trusting them, we had a harder time green-lighting the people we didn’tknow. Again, that’s human. But if we don’t make conscious decisions to accountfor it, we’ll never change the dynamics of these events.
Finding people in otherunder-represented groups.Our program has almost no people of our parents’ generation, and we don’t thinkwe have any speakers with disabilities. In the future, we can apply theapproach above to find more great speakers from these groups.
--
Pullingtogether a conference program that highlights deservedly well-known peoplein your field *and* reveals compelling new stories is a challenge. But it’s oneyou can take on—and that, frankly, not enough conferences do. As we’ve found, meritocratic selection isn’tjust a phrase you can mull or a fantasy for the future. These tools, usedthoughtfully and in combination, can help you achieve it: transparent process,blind selection, proactive outreach and enlisting help. And we hope this post will help sparkthis question: if you’re not using these tools, is your selection process susceptible to unconsciousbiases that could be making it work less well? Is it really as merit-based as it could be?
