Leading Search Engine Provider Enhances Ad Relevance

We provided in-market evaluators for multiple international markets to ensure accurate ad performance

The Company

A leading multilingual search engine provider looking to serve the best possible ads to its visitors.


The Challenge

A leading multilingual search engine provider relied on algorithms supported by an in-house team to serve the best possible ads to its visitors, and the company’s bottom line depended on clicks. Knowing whether ads were performing optimally was critical to the provider’s success. This company needed in-market evaluators for multiple international markets to ensure accurate ad performance.


The Solution

The search engine provider turned to us for this effort, based on strong recommendations from other internal teams. The project required multiple in-market evaluators who reviewed, verified, cleaned and labeled data to ensure the client’s ads services were performing accurately. The ads were reviewed, along with the content that the ads linked to, and ratings were assigned based on a five-point scale. Our project managers ensured that the evaluation data flowed back to the client in a timely manner according to the client’s specifications.


The Result

The search engine provider successfully measured ad relevance for specific queries as a result of the labeling provided by our team. This not only gave it a trend over time to understand which experiments were driving up relevance, but also allowed it to analyze root causes on irrelevant ads. Once the root cause was fixed, the provider could perform additional experiments to ensure other queries weren’t negatively impacted. Our high-quality labeling data has become a crucial piece of the machine learning process, training algorithms to provide more relevant ads, which results in higher revenue for the client.

As the project ramped globally, the search engine provider expanded search advertising into several new markets with the confidence of knowing that ads had been evaluated, sorted, and ranked appropriately by in-market experts, increasing the opportunity for revenue. This confidence was achieved due to the high-quality data that was provided. The client has repeatedly commended our team for its strong quality and performance.

Access to in-market evaluators for all of the client’s requested countries resulted in ads that were much more highly relevant to users in those locations.

Our highly skilled project management team ensured that rating information was delivered on time and according to the client’s quality standards. The team was also able to assist the client in identifying the root cause for ads that were underperforming so that they could optimize their resources to make the right improvements. Overall, our ability to consistently deliver quality data cemented this successful partnership.

Website for deploying AI with world class training data


Andrew Ettinger | Chief Revenue Officer

Andrew Ettinger joined Appen as Chief Revenue Officer in May 2023 overseeing the company's revenue strategies and driving growth in the field of AI. He joined Appen with more than 25 years of sales experience in sales and services in the technology industry. 

Andrew's expertise extends to harnessing the power of data to drive insights and optimize processes. As the Chief Revenue Officer at Astronomer, he successfully grew the adoption of their open-source data solution, leading to a remarkable increase in monthly downloads and revenue. His strategic initiatives resulted in a 600% growth in customer count and a 75% win rate. 

Prior to joining Astronomer, he served as the VP of Sales at Pivotal Software, where he helped grow the business from zero to $100 million in annual recurring revenue in a single year leading up to Pivotal’s initial public offering, and up to $500 million thereafter. Under his leadership, the company achieved three consecutive years of 50% revenue growth, fueling digital transformations for Fortune 500 companies in various sectors. 

Andrew holds a Bachelor of Science in Business Marketing from The Ohio State University.