Article by Greg Lindsay
Wired (UK)  |  October 2013

How to Build a Serendipity Engine

'My phone knows your CV, but not if I should know you.'


(This article was taken from the November 2013 issue of Wired [UK] magazine.)

Silicon Valley’s latest management fad is to not manage. “Some of the best decisions and insights come from cafeteria discussions, meeting new people, and impromptu team meetings,” said Yahoo! when it revoked working from home. Marissa Mayer, ex-Google, knew that hits such as Gmail and Street View were the product of engineers meeting serendipitously over lunch.

Google’s new campus is designed to encourage “casual collisions” at its rooftop cafés, and Facebook has hired Frank Gehry to build “the perfect engineering space: one giant room that fits thousands of people, all close enough to collaborate together,” Mark Zuckerberg explained.

Study after study has demonstrated the power of proximity when it comes to birthing ideas, and network theorists have shown how bridging “structural holes” between teams and disciplines consistently leads to better and more innovations. What’s ironic is how the world’s largest search companies and social network appear to have no better ideas of how to plug these holes than to toss everyone into a room with an espresso machine.

Google is obsessed with “engineering serendipity”, combining users’ search history, location and context to answer questions before they’re asked. What would it take to build a social serendipity engine—one that could identify who’s nearby, mine your hidden contexts and relationships and make the necessary introductions? After all, not everyone drinks coffee.

The first step is finding people, which location-based dating services such as OKCupid and Grindr manage well. They’ve been joined by “social discovery” apps such as Sonar and Circle, which link users’ networks to their smartphone’s GPS.

“Ten years from now, you will look at your phone or through Google Glass, and know everyone in the room,” says SocialRadar founder Michael Chasen. “I haven’t met anyone who disagrees with that.”

The next piece is harder. It’s not enough to know who’s in the room—you also need to care. Facebook “likes” aren’t accurate predictors of a match. What’s missing is context—why I want to meet this person now. Grindr works because its consideration is binary. Any serendipity engine with higher aspirations must grasp deeper motivations than social media supplies.

One solution is to build rich profiles manually. That’s the tack taken by Relationship Science, which maps potential connections between 2.3 million members of the 1 percent. Customers can trace the shortest path to their quarry via colleagues, corporate boards and alma maters, each link graded into strong, medium and weak ties—captions Glass might someday flash over their faces.

Another approach is to collect data using sensors. An MIT Media Lab spin-off, Sociometric Solutions, equips office workers with “sociometric badges” to measure movement, speech and conversational partners, using the results to plot maps of organisations. Its findings suggest Yahoo! should invest in large cafeteria tables: workers who eat at tables for 12 are more productive than those at tables for four.

The last step is the trickiest: given enough context about the strangers headed towards you, can an algorithm crunch enough data in time to spot connections? Another startup, Ayasdi, does that by rendering every big data set as a network map, revealing hidden connections that CEO Gurjeet Singh labels “digital serendipity”.

If yoked together, these approaches could help fill in the blank spaces on the org chart where work actually happens and new ideas are spawned. Although luring them into the same room with an espresso machine would probably speed things up a bit.

About Greg Lindsay

» Folllow me on Twitter.
» Email me.
» See upcoming events.

Greg Lindsay is a generalist, urbanist, futurist, and speaker. He is a 2022-2023 urban tech fellow at Cornell Tech’s Jacobs Institute, where he leads The Metaverse Metropolis — a new initiative exploring the implications of augmented reality at urban scale. He is also a senior fellow of MIT’s Future Urban Collectives Lab, a senior advisor to Climate Alpha, and a non-resident senior fellow of the Atlantic Council’s Scowcroft Strategy Initiative.

» More about Greg Lindsay

Articles by Greg Lindsay

-----  |  August 3, 2023

Microtargeting Unmasked

-----  |  July 1, 2023

2023 Speaking Topics

CityLab  |  June 12, 2023

Augmented Reality Is Coming for Cities

CityLab  |  April 25, 2023

The Line Is Blurring Between Remote Workers and Tourists

CityLab  |  December 7, 2021

The Dark Side of 15-Minute Grocery Delivery

Fast Company  |  June 2021

Why the Great Lakes need to be the center of our climate strategy

Fast Company  |  March 2020

How to design a smart city that’s built on empowerment–not corporate surveillance

URBAN-X  |  December 2019


CityLab  |  December 10, 2018

The State of Play: Connected Mobility in San Francisco, Boston, and Detroit

Harvard Business Review  |  September 24, 2018

Why Companies Are Creating Their Own Coworking Spaces

CityLab  |  July 2018

The State of Play: Connected Mobility + U.S. Cities

Medium  |  May 1, 2017

The Engine Room

Fast Company  |  January 19, 2017

The Collaboration Software That’s Rejuvenating The Young Global Leaders Of Davos

The Guardian  |  January 13, 2017

What If Uber Kills Public Transport Instead of Cars

Backchannel  |  January 4, 2017

The Office of the Future Is… an Office

New Cities Foundation  |  October 2016

Now Arriving: A Connected Mobility Roadmap for Public Transport

Inc.  |  October 2016

Why Every Business Should Start in a Co-Working Space

Popular Mechanics  |  May 11, 2016

Can the World’s Worst Traffic Problem Be Solved?

The New Republic  |  January/February 2016

Hacking The City

Fast Company  |  September 22, 2015

We Spent Two Weeks Wearing Employee Trackers: Here’s What We Learned

» See all articles


August 05, 2023

FT: Self-Created Communities for the Digital Age

August 03, 2023

Microtargeting Unmasked: A Threatcasting Report

August 01, 2023

At What Point Managed Retreat?

July 18, 2023

Prepare for Descent: The Relative Decline of the US Passport

» More blog posts