Designing for Consistency


AppNexus has changed a lot during the year that I’ve worked here. We’re expanding quickly, which can sometimes cause growing pains for products. One constant challenge is keeping multiple products consistent with each other. This challenge led our UX team (myself included) to develop several processes to help prevent inconsistencies.

What causes inconsistencies?

At AppNexus, we are organized around a few products which are targeted to different users. Each product is run by an independent team. Within the product teams are smaller teams devoted to specific product features. As you can imagine, this division could make it difficult to communicate consistently across products and features. Because the UX team is separate from the product teams, we have a unique perspective across all product and feature teams.

The users of each of our products have very different goals, so it might seem challenging to design an interface that works for all users. But the UX team has proven that it is possible to solve smaller independent challenges consistently, while building designs that suit each group’s goals. For example, one user may be trying to minimize cost, while another is trying to maximize revenue, but they could both be selecting objects from a list as part of their work flow. On the UX team, our processes have evolved over time to help support our existing work flows. We have a few different ways to keep things consistent:

How to Make New Products Consistent

Templates: When I started using Sketch for UX design, I built a file for all of our elements. Some were basic, like form elements and buttons; others were large, like our dialogues. Building the file allowed me to understand some of the consistency issues—by exploring the product and rebuilding elements, I found similar objects that were inconsistent, and recorded them so they could be fixed in the future. Having a template file means that when you’re building a design, it’s easier to use pattern elements since they’re already constructed.

Pattern Library: The UX team created a pattern library in order to formalize how some of our design elements work. In the pattern library, we document our usage guidelines for designers, engineers, PMs, etc.

Review Process: Designers for each product are in constant communication with each other. They’re even trying Pair Design. In order to get an outside perspective, we often request feedback from team members who are not working on our product. Sometimes a reviewer will respond, “I’m working on something that this design would be perfect for!” Or a designer might say, “This isn’t consistent with how we do this elsewhere,” In this way, the review process enforces consistency.

Design Council: If a designer is changing an existing pattern or creating a new pattern that could be useful for other teams, he or she presents at a meeting we call “design council.” At design council, we discuss how each change impacts other pages. If it’s a new idea, we discuss whether it could be abstracted out in order to be useful elsewhere.

When you’re making a product that you want users to easily understand, consistency is important. If you want to be more consistent, the best way to get started is two-fold: first, build your own template files because they provide a good base for building a pattern library later. Second, start a conversation at your company about the importance of consistency.

The AppNexus UX Team is hiring! Join our team.

Tagged , | comments (0)

AppNexus welcomed over 93 new employees in Q1!



AppNexus hit a big milestone in Q1 by hiring over 93 new employees! Crain’s New York published a feature on the record-breaking quarter: $1B ad-tech giant AppNexus hits hiring record

Here are some awesome first quarter highlights:

  • 93 new hires
  • 40% of the new hires are women
  • 38% were tech hires
  • 82% acceptance rate for job offers
  • 920 AppNexians to-date!

As Chief People Officer Brandon Atkinson said, “Building a diverse workforce is very important to AppNexus…We’re making strides.”

In addition to the record-setting quarter, AppNexus also acquired Yieldex, welcoming even more employees to AppNexus.

To read the full article, visit Crain’s New York.

Visit our careers website to learn more about Life at AppNexus.



Tagged | Comments Off

Crafting Q2: How To Get High Performance on Low Priced Inventory

The latest AppNexus whitepaper provides insights specific to Mother’s Day and Father’s Day. These two days not only represent occasions to think of Mom and Dad, but also present clear buying opportunities! Download the complimentary whitepaper today.

Below is a sample of what you will learn from our whitepaper to help you maximize your success and #WinInProgrammatic.

In addition to low CPC and high CTR values, the CPM (cost per thousand) of Arts and Entertainment impressions is generally low during the early part of each month in Q2. The below graph provides an example of the general trend that CPM values follow during the course of a quarter, and provides the increase in CPM values for each month in Q2 in 2014. CPM values tend to go up toward the end of each month and especially increase at the very end of the quarter when demand increases.


CPM Values Q2 2014

Download the complimentary whitepaper today and #WinInProgrammatic

Tagged , , | Comments Off

10 Years of the Ad Exchange



I spent the first three months of 2005 working on an ad serving technology called “Yield Manager” that packaged the proprietary predictive optimization we developed at Right Media into a software product that other companies could use to run their online advertising businesses. The fundamental idea was that we could treat every ad impression as a real-time auction where thousands of advertisers could bid based on a predictive model. Almost by accident, I built a feature that allowed companies to bid into each other’s auctions. We called it “linked networks.”

On April 1, 2005 we migrated the Right Media ad network onto the new platform along with two other networks. As we obsessively watched the reporting data to see how it performed, I began to yell. Something incredible happened: advertiser CPAs were improving, publisher revenue was increasing, and the networks were making more money. In hindsight, it seems obvious what happened: by combining three separate pools of supply and demand, every campaign could bid on more inventory, giving it more chances to find the right user. Every publisher was getting three times the demand. And the networks tripled the effective size of their networks.

Over the next few months, tens of other ad networks joined the platform to take advantage of this revolutionary new concept. As we watched this phenomenon take shape, we realized that we had created an entirely new business model: the ad exchange.

Today is the tenth anniversary of the launch of the ad exchange. Since 2005, it has spread to mobile, video, TV, print, and radio. The programmatic industry represents $10 billion of advertising spend, and has spawned hundreds of companies and billions of dollars of IPOs and acquisitions.

As we enter the second decade of programmatic advertising, I see huge opportunities ahead for marketers, publishers, and consumers. As the CEO of the leading independent ad tech company whose mission is to create a better Internet, I’m constantly asking, “What can we do today that will significantly impact the online ecosystem for years to come? How can we improve the system for marketers, publishers, and consumers?” Here are five ideas that I hope we see the industry embrace over the next decade:

  • Privacy by Design. Many Internet businesses start by building large consumer audiences, then tack on an advertising model for monetization. Consumers expect advertising experiences that protect their privacy and respect their personal information, and the only way to deliver on that expectation is to start with privacy and build around it.
  • The End of the Black Box. While the ad technology landscape is complex, the basics of marketing haven’t really changed: deliver the right message to the right audience on the right medium. We need a combination of human intelligence and machine learning to achieve this at scale. Let’s open up the black box and allow marketers and publishers to leverage and create proprietary insights to improve the efficacy, monetization, and impact of digital advertising.
  • Unbundling of Ad Tech and Media. For the past 20 years, every major Internet portal has had the realization that its revenue and success depend on a handful of ad technology vendors that track and optimize digital marketing…they’ve then gone on a build/buy/partner mission to try to “own the stack” and allocate a disproportional (they would say proportional) share of spend to their own properties and related ad networks. This has been a boon for entrepreneurs and VCs who start and fund ad tech companies, but leaves marketers, agencies, and publishers without an ecosystem of scaled independent technology companies that they can trust. It’s time to unbundle tech from media. Agencies and marketers will spend money where it has the most impact, and won’t trust media companies that try to use ad tech to tip the scales.
  • Single Digit Take Rates. Executing a programmatic campaign is dramatically more expensive on a cost-per-dollar-spent basis than the equivalent campaign executed through the traditional RFP method, which itself is far more expensive than traditional TV buying. Technology is supposed to make things cheaper and more efficient. I believe the introduction of programmatic direct technologies like twixt will have a huge impact on agency efficiency. Targeting and bidding are commodity technologies now. The total take rate from ad tech will drop from the current 40-50% to below 10% as volumes increase, creating significant leverage for marketers and publishers.
  • Disruptive Innovation. Most of the successful companies are still DMP, SSP, DSP, ad network, retargeting, or exchanges…all of which were invented by or around the Right Media platform. It’s time for entirely new ideas to disrupt the industry. My bet is that by offering an open, independent, low-cost platform, one of our clients or partners will be able to invent the next disruptive idea that will change digital marketing. I can’t wait.



Tagged | Comments Off

The Alchemy of Org Building: Keeping the Magic Alive as Organizations Scale

When I started at AppNexus, we had 28 employees. Over the last five years, we’ve grown to nearly 800. As we’ve grown, a question I’m frequently asked is, “how do we maintain the incredible culture we have?” This challenge isn’t unique to AppNexus—all start-ups go through it. So how do we keep the magic alive?  How do we foster a culture where employees authentically care about each other’s success; where everyone acts as “owners”; where everyone has the opportunity to impact important things; where a strong sense of shared purpose exists?

First, let’s ask what culture is. For me, it’s how employees behave and feel in the workplace. By that definition, at some level culture must change as companies grow. At AppNexus, our CEO cannot release production code on Friday afternoons anymore. Our Services team cannot have direct access to production databases anymore. We won’t know everyone’s names or faces anymore. Culture always changes as companies mature—it’s unavoidable.

But it doesn’t mean that what gave our organization that magical feeling five years ago must be lost.  The spirit of a company isn’t found or lost in its operational processes or company size—it’s more complex than that. Keeping the magic alive requires us to recognize a few truths:

#1 Clarity about our intention will create the conditions for growth.

In a previous post, I wrote about defining your organization’s intention. The idea is that creating a great organization requires defining our intention along three dimensions: company purpose, team mandate, and employee experience. We should clearly answer:

  • At the company level, what is our purpose? Why do we exist?
  • At the team level, what is our mandate? What are we trying to achieve?
  • At the individual level, what is the employee experience required to achieve the above?

If we have clear answers to those questions, and if those answers have emotional resonance with employees, something like gravity is created which pulls employees together and propels them forward in unison.

#2 It has always been, and always will be, about the people.

At the end of the day, when things get really hard, when we’re feeling fried, or when we doubt the path forward, it isn’t the purpose, the work, or the money that keeps us going – it’s the person next to us. It’s that person we respect and admire. That person we don’t want to let down. That person who inspires us to be better. It’s the people that make a place special and keep it special. It was that way when we were a five-person org, and it will be that way when we are a 5,000-person org.

With that in mind, here are two things to consider:

  • DNA Hiring. “Special” isn’t found in the companies where people have worked, or in the skills they have developed. Special is found some place deeper – in their passions, values, and capabilities. Hire for those things. At AppNexus, we look for people who are driven: to learn and teach, to make greatness happen despite obstacles and failure, to see and improve the whole system, and to empower the success of others.
  • The Law of Attraction. The Law of Attraction states that like attracts like.  Great people beget great people.  Crappy people beget crappy people.   Find one A-Player and inevitably many will follow because of that person’s influence—perhaps through a referral, or because of the great impression an A-Player makes when he or she is interviewing candidates. Remember that A-Players never follow B-Players, and B-Players only attract Bs (or worse!). So hold out for the As and avoid the Bs – no matter how desperate you are to hire.

Here is a case-in-point: Justin was a DNA hire. He had just finished teaching in Hawaii for Teach for America. Before that, he graduated with honors in Philosophy, and was an accomplished runner. He had countless examples of overcoming adversity. And he had zero ad tech and zero technical experience. After joining AppNexus, he quickly grew from being a Product Specialist, to a Technical Account Manager, to the manager of the team, to an influential Product Manager. Justin is also a great example of the Law of Attraction, because at least twelve amazing AppNexians have joined due to their direct or indirect connection to him. Hire Justins.

#3 All things atrophy; we must keep reinforcing what matters.

After we define our organizational intentions and hire great people, what happens next? Atrophy. It’s inevitable. The real work of leadership is in the maintenance. On AppNexus’s Global Services team, for example, we continually reinforce the things that create the conditions for great outcomes:

  • Hiring: We never stop talking about how important hiring is, and we continually improve our process. For example, we recently improved our job descriptions to ensure that they are written as inclusively as possible in order to attract a diverse talent pool.
  • Employee Onboarding: We designed our onboarding process specifically to reinforce the culture we want to maintain. I also give every new employee an orientation where I discuss the cultural touchstones that are important to us.
  • Employee Recognition: We created an employee recognition program called The Merlin Awards where employees recognize other employees for living our values. We do this in a fun way that resonates with our culture.
  • Knowledge Sharing: We maintain a tradition called “Friday Bullets” in which employees send around an email mentioning something they are excited about, and something they are concerned about. We maintain a program called Know@ where employees help each other answer technical questions and share best practices.

To keep the magic alive, focus on what created the magic to begin with (being clear about your organizational intentions, hiring for DNA, and creating the conditions for great outcomes by reinforcing what matters). This is hard work. It’s easy to focus our minds exclusively on the outcomes we need, like revenue growth, innovative products, or thrilled customers. However, focusing our minds on process (cause) over outcome (effect) is the surest path to achieving a sustainably powerful culture that scales with your organization.

Comments Off

AppNexus Acquires Yieldex

Forecasting anything is hard. If you think March Madness is tough to pick, try forecasting Internet traffic patterns. Every major publisher, including mobile app developers and cross screen content producers, on the Internet has this problem every month: how do you predict how many people in a particular audience will visit your site so you can package this and sell it to advertisers? What happens if two audiences (women and 25-34 year-olds) overlap, and different advertisers want a guarantee that you’ll sell them a certain number of impressions?

The traditional solution to this problem was to throw everything you’ve sold into Excel, use various rules of thumb to guess at overlap and traffic patterns, and then be very conservative about what you allow your salespeople to sell. This inevitably causes a large part of the inventory to go unsold.

In 2007, Tom Shields and Doug Cosman set out to solve this problem by founding Yieldex. As the architect of NetGravity, one of the first ad servers, Tom knew the pain first-hand, and had a unique vision for how to solve it. Instead of looking at aggregate data and trying to parse apart the increasingly complex audiences after the fact, why not run a simulation of how an actual ad server would deliver campaigns impression by impression? As with all brilliant ideas, there were a lot of reasons why not: it requires writing an ad server, it requires disproportionate processing power, it requires deep integration into the publisher, and … you still have to forecast Internet traffic patterns to make it work. Not surprisingly, Tom was undeterred, and not only overcame these challenges, built an incredible roster of clients like The New York Times, The Weather Channel, and Pandora that use these tools to significantly increase revenue and sell-through.

With the ability to accurately forecast publisher inventory availability, Tom realized that he had the technology to solve an even more important problem: how to allow agencies and publishers to use technology to improve the media buying process itself. Instead of having agencies pick up the phone to ask about availability – in other words, to have someone dig through the spreadsheet and try to guess – why not expose this data programmatically? The long-discussed idea of programmatic guaranteed could suddenly become technically possible.

While Tom and the Yieldex team were approaching programmatic guaranteed from the sell-side, AppNexus was making significant investments of its own. In April 2014, AppNexus released Twixt, an elegant buying automation tool that was intended to help media buyers bring their RFP process online. In theory, this could be the buy-side for a programmatic guaranteed marketplace… but how could we help publishers get their inventory availability online? We began to realize that publishers needed a new set of tools that would allow them to operate programmatic-first, not as a bolt-on to the traditional ad serving tools that they had been using since the late 1990s. In November 2014, we acquired Open AdStream, a powerful publisher ad serving platform, and began to integrate our programmatic technology and vision for the future.

Today, we take the next logical step: AppNexus is acquiring Yieldex. The two companies couldn’t be more similar. Both were founded by tall (Tom is taller) ad tech veterans, funded by First Round Capital, are pure-play technology companies with no media business, and are dedicated to solving the hardest problems to make the Internet better. Our approaches to programmatic guaranteed fit together like puzzle pieces (Twixt for buyers, YieldexDirect for sellers), as do our publisher solutions (Yieldex Analytics for forecasting and analytics, AppNexus for ad serving and RTB). Both companies have built world-class teams of professionals who are passionate about empowering our customers.

Welcome Tom, Andy, and the Yieldex team!

To learn more, check out the video below:

Tagged , | Comments Off

With a Little Razzle Dazzle, We’ve Made Learning and Teaching a Core Company Value

Join AppNexus to get Razzle Dazzled by some of New York’s top technology leaders on March 18! Read on for more details and to RSVP:

As companies grow, some of the biggest challenges they face include communicating effectively across teams, staying true to their culture and values, and building a strong sense of community.

At AppNexus, we’re growing quickly and realize there’s no one-size-fits-all solution to addressing these issues. However, one way we have experienced success is through our ‘Razzle Dazzle’ program.

Razzle Dazzle began in 2009 when one of our Sr. Systems Analysts emailed AppNexus CEO Brian O’Kelley, asking that he gather the company so he could showcase what he was working on. “This would be no dog and pony show” he wrote, “this would be a Razzle Dazzle!”

On that day, Razzle Dazzle was born and quickly became a cornerstone of our culture – a regular company-wide gathering that ensures the learning never stops.

Whether showcasing a new product feature, highlighting an upcoming company initiative or celebrating the achievements of cross-functional teams, Razzle Dazzles give AppNexians of all levels, and from offices around the world, the opportunity to share knowledge with and gain visibility across the entire company, while developing presentation skills in an informal and fun setting.

But, Razzle Dazzle is more than just a company initiative – it’s also a way for us to learn and teach with our community. Join us on March 18 to experience Razzle Dazzle for yourself at our next event: True Stories from the Trenches: How to Build and Lead a Successful Technology Organization. Hear from some of New York’s top technology leaders as they share their experiences, struggles, and strategies for success. Learn how they scale complex technology systems, stay ahead of industry trends, and balance technical decision-making with effective team management.


  • Minerva Tantoco, CTO, New York City
  • Lucille Mayer, Managing Director and CIO, Bank of New York Mellon
  • Camille Fournier, CTO, Rent the Runway
  • Catherine Williams, Chief Data Scientist, AppNexus

When: Wednesday, March 18, 2015. 6pm – 8pm
Where: AppNexus, 19 West 22nd Street, 4th Floor, New York

Learn more about Razzle Dazzle at

Tagged | Comments Off

Germany is a Premium Market for Programmatic

At last summer’s European AppNexus summit, my presentation focused on the different marketplaces across Europe, and how each of these is uniquely defined by consumer behavior. I shared some numbers from emarketer at the time on predictions of the penetration of programmatic into the wider display market in each country across Europe. Germany is the second largest market for digital display in Europe, yet it’s estimated that programmatic will make up just 33% of that market by 2017, compared to 59% in the UK and 56% in France. [Source: emarketer]

These numbers alone don’t paint the full picture. Globally, the AppNexus platform delivered an average of over 34 billion ad impressions a day in 2014, meaning that we see in-depth market dynamics in every country in which we operate. In Germany, this data gives meaningful insight on how the market is developing and what makes it unique. For the first time, we are now able to share this insight and today launch the first in an ad-hoc series of reports, which you can access here. As you’d expect, the detailed analysis is in German, but I thought it useful to share some of the high level findings from the whitepaper with a wider audience:


  1. Firstly, despite the predictions referenced earlier, we’re seeing significant growth in RTA in Germany, with a 54% increase in programmatic impressions in 2014 and an 85% increase in revenues traded over the platform
  2. Between 8% – 10% of all of the inventory traded through the AppNexus platform in Germany is done so within the Deals marketplace
  3. There are more impressions on large formats, such as billboards and half page ads, in Germany than in any other market worldwide
  4. Linking the previous two points, 70-90% of the large formats traded through the AppNexus platform in Germany are also done so on Deals

These numbers tell an interesting story. Of course, the growth in the market is positive, but it’s what’s driving that growth that’s most interesting. What’s unique about the data is the relatively high percentage of both Deals transactions and large format sizes. This provides support for our theory that the German market is unique and dynamic, and really focused on developing strong relationships between buyers and sellers. The findings show that it is premium inventory that’s driving the growth that we’re seeing in Germany, which is why you see revenues are growing faster than impressions. We’ve used these insights to really drive our strategic investments and positioning in Germany, so that we can help our customers be better positioned to win. So while penetration of the German display market may be lower than in other countries for programmatic, its application is tightly focused and offers massive opportunities for publishers and advertisers alike.

To download the full whitepaper please visit

Tagged | Comments Off

RTA Made in Germany

Click here to read this post in English

In den vergangenen Jahren hat sich das programmatische Geschäft global von einem separaten Kanal mit „Performance Stigma“ hin zu einem inhärenten Teil des Mediageschäfts entwickelt. RTA ist erwachsen geworden und Teil der Advertising- bzw. Salesstrategie beinahe aller Marktteilnehmer auf Einkaufs- und Verkaufsseite. In Europa sehen wir in den ersten Ländern, wie beispielsweise in den Niederlanden, dass RTA  mehr als 50 Prozent des Displaygeschäfts ausmacht.

Doch die verschiedenen europäischen Märkte sind sehr unterschiedlich und in Deutschland ist der Anteil des programmatischen Geschäfts noch deutlich geringer. Das der deutsche Markt in Europa „besonders“ oder „speziell“ ist, ist sicherlich nicht neu. Auch in der Vergangenheit hat sich oftmals gezeigt, dass sich in anderen Ländern erfolgreiche Geschäftsmodelle im Online Advertising nicht zwangsläufig auch in Deutschland durchsetzen müssen oder dies zumindest zeitversetzt tun. So haben es beispielsweise klassische Adnetworks, ein international sehr erfolgreiches Geschäftsmodell, hierzulande immer sehr schwer gehabt .

Doch wie genau unterscheidet sich der deutsche RTA Markt im internationalen Vergleich? Wir haben uns, auf der Grundlange der über die AppNexus Plattform getätigten Transaktionen, den deutschen Markt einmal etwas genauer angesehen.

Dabei haben wir einige interessante Beobachtungen gemacht:


  1. Der deutsche Markt wächst massiv. Im Jahresverlauf 2014 stellen wir ein Wachstum der programmatisch gehandelten Impressions um 54% fest, gleichzeitig wuchs der gehandelte Mediaumsatz um 85%
  1. Agenturen holen, im Vergleich zu unabhängigen Trading Desks und Re-Targetern, sehr stark auf. Ihr Anteil am programmatischen Mediaumsatz liegt mittlerweile bei 46%.
  1. Zwischen 8 und 10% der Media wird bereits im Deals Marktplatz gehandelt.
  1. Deutschland ist bei Großformaten im internationalen Vergleich Spitzenreiter. In keinem anderen Markt sehen wir einen ähnlichen Anteil an Formaten wie etwa Billboards und Halfpage Ads.
  1. Der Schlüssel zu Großformaten sind Deals. Der Handel dieser Formate findet nahezu ausschließlich (>90%) im Deals Marktplatz statt.

Insgesamt zeichnen die Daten ein sehr interessantes Bild vom deutschen Markt. Dieser ist geprägt von einem starken Premiumverständnis und den engen Beziehungen zwischen Einkaufs- und Verkaufsseite. Marktteilnehmer in Deutschland suchen nach Wegen, diese Beziehungen im programmatischen Geschäft abzubilden und sind so dazu bereit in großem Maße auch Premium Media über RTA zu handeln. Effizienz und die zielgruppengenaue Steuerung und Optimierung der Online Marketing Aktivitäten stehen im Vordergrund.

So mag Deutschland im internationalen Vergleich bei Volumen und Umsatz noch keinen der vorderen Plätze belegen, die Art und Weise wie die Marktteilnehmer die Möglichkeiten des programmatischen Mediahandels nutzen jedoch ist im globalen Vergleich sehr fortschrittlich und innovativ. Bei AppNexus freuen wir uns darauf auch in 2015 mit technologischen Innovationen weiter zu dieser Entwicklung beizutragen und sind sehr gespannt darauf, wie sich unsere Partner diese für ihr Geschäft zu Eigen machen.

Das vollständige Whitepaper lässt sich hier kostenlos herunterladen:

Comments Off

The Evolution of Data Science at AppNexus (or: A Big Bag of Dirty Words)

Until I joined AppNexus three years ago, my work involved no data and no science—I was a pure mathematician. My PhD was in the area of differential geometry and general relativity, and I spent graduate school and a couple of postdoctoral fellowships thinking about things like geometric models for black holes and how to measure mass relativistically. The most real-world thing connected to my work was that the models for surfaces of black holes I worked with are geometric cousins, twice removed, of those for soap bubbles. My technology was a notebook, a pen, occasional books or papers, and logic. And despite heroic effort (or so it seemed to me), in the end maybe a dozen people in the whole world cared about the theorems I proved – if I was lucky.

My job has changed a lot since then.

In 2012, I joined AppNexus as a Quant. As part of the Optimization team, affiliated with our Product team, my job was to help design and build tools for buy-side clients to enable them to achieve their campaign delivery and performance goals. This was enormously exciting for me. Suddenly I was using real technology: Excel, Python & Pandas, SQL, and exploring flavors of math (probability, statistics) that I had previously ignored. And there was data – plentiful data! But it was rigid: it consisted entirely of aggregations created for our customers to use in their own reporting to their clients.

Over time, it became clear that data could be a powerful tool for AppNexus if we really focused on it. We shifted the Quant team from Product to Engineering, broadened its scope and mission, and renamed it Data Science. We got our own little Hadoop cluster and started playing with raw logs, using Hive to create a few aggregations for our own research and experimentation. We started looking at ways that we could use the data creatively, well beyond the narrow buy-side optimization we had been focusing on.

Ironically, our most creative use of data during that time didn’t use our own data. And it was motivated         by . . . porn.

Turns out, our clients do not want to serve on porn—it’s bad for their clients and thus bad for them. But at that time, our approach for detecting whether or not a domain was porn or not was summed up by Supreme Court Justice Potter Stewart when he said of hardcore pornography, “I know it when I see it.” AppNexus created a team who did just that: human auditors would look at websites and determine whether the sites were or were not porn.

Time to put a Data Scientist on the job. Let’s call him Ryan (since that’s his name).

As you may know, most porn domain names are not subtle—if you look at the name of a site, you’ll usually know whether or not it’s porn. So Ryan found a “big bag of dirty words” (fun, that) and cooked them into an algorithm that would search for these particular words within domain names. Think of some more dirty words to add to the “big bag”? Catch more porn. Add in their creative misspellings too? Yet more. But there was an ongoing issue of false positives, domains that his algorithm identified as porn but which were in fact entirely wholesome.

For instance:

Now, you and I can read that domain name and easily parse it as “st/pauls/lutheran/church” rather than, say, “st/paul/slut/her/an/church” or other variations.  Ryan’s work was cut out for him: create an algorithm that could tell, too.  But how?

Enter the Google trillion word corpus, an amazing data set that includes the counts of how often every single possible word appears among 1 trillion words of running text, misspelled or no, provided that word appears at least 200 times.  Armed with this data, Ryan could directly compute the probability of each word in each distinct way of parsing the domain name, choose the parsing with the highest probability, and then search for dirty words. And bingo – no more church websites classified as pornography.

While this is a fun problem to describe and its solution is elegant, in the grand scheme, this particular problem was small potatoes compared to the others we face, and our end-to-end porn domain detection process is not fully automated today; our (human) auditors still play an important role. But this is one example of how the Data Science team’s work and our creative use of data furthers AppNexus’s mission to Create a Better Internet: by helping to ensure that advertisers can buy in a brand-safe way at scale.

These days, Data Science at AppNexus is no longer the awkward step-child of Product or Engineering – it’s a first class citizen alongside both. We research, experiment, and design innovative algorithms to achieve product goals across the company, far beyond our buy-side roots. On the sell-side, we help Publishers monetize their content; in the performance marketplace, we focus on nuances of auction logic to help create a virtuous feedback loop for buyers and sellers both to extract value; and across the platform, we help root out unwanted or fraudulent elements such as click-bots, porn, and non-human traffic.

It’s a new era, both for the team and for me. The amount of data we see every day continues to grow at a monumental rate, and our team now has the right tools to dig in and solve problems the right way—our iron pickaxes of 2012 have been replaced by plasma torches going into 2015. Data-wise, we can access and analyze impression-level data in both static and streaming form, create our own feature variables, and design machine learning algorithms that operate at scale—and we’re investing in more hardware so that we can do even more. People-wise, we’ve assembled a team from diverse backgrounds ranging from physics to biomedical engineering to political science, every one of whom has rigorous depth that he or she successfully balances with business reality.

As for me, I love all of it: the people, the data, the technology, and the mandate to bring them together to solve problems that more than a dozen people in the world care about.  And these problems? In some ways they’re harder than the ones I studied in academia: here we can’t cut off one little piece of a problem and study it in isolation, we must see and comprehend the whole system in order to improve it.

Tagged , , | Comments Off