Mobile Device Manufacturer Achieves Greater Accuracy for Map Software with In-market Raters

The firm partnered with us to develop and execute a testing plan to meet its launch objectives.

The Company

A leading software and hardware company partnered with us to test accuracy of its new, confidential mapping software for an upcoming major release of its mobile phone operating system.

The Challenge

A large software and hardware company needed to test out map software for the next major release of its mobile phone operating system. Specifically, it needed users in multiple major cities to determine whether walking/transit directions provided by the app were accurate. The app had not yet been released and needed to be kept top-secret, so surveying actual users was not an option. Meanwhile, sending its own employees would be costly and would not provide the local expertise needed to thoroughly test the map software.

The Solution

The firm partnered with us to develop and execute a testing plan to meet its launch objectives. After consulting with the firm on the desired outcome, we developed a customized approach for the testing project. Teams of 10-15 testers per city were recruited and qualified by preparing the on-site testers with virtual onboarding materials before they showed up for day one. A temporary office space was reserved in each city where a client representative could meet with the testers to provide additional training and distribute devices. Using dedicated devices with the testing app which was preloaded with maps and directions, testers followed predetermined routes, taking notes of where the directions led them to the wrong location or where information was inaccurate. This data was then used by the client to refine its software before the release to the general public.

Since this was a top-secret project for a high-profile client, confidentiality was a priority. A code name was used for the project and the client’s identity was never revealed to the testers. During the recruitment process per our team’s recommendation, testers’ employment history was reviewed to disqualify anyone who had previously worked for the client or any of its competitors. Furthermore, testers were required by non-disclosure agreements not to share details of the project with anyone. Even within our team, knowledge of the project details and the client’s identity were kept on a need-to-know basis.

The Result

The firm successfully launched its new map application into multiple markets by partnering with us. Not only was the firm able to improve the accuracy of its software in a wide variety of locations before releasing the product, it saved money by avoiding the expense of sending its employees into the field. Significant efficiencies were gained through the local expertise of in-market testers; their knowledge of the local areas meant that testing could be completed more quickly and with more thorough feedback. In addition, the confidentiality of the project remained intact, which was of significant importance to the client.

The success of this project relied heavily on the development of a customized testing plan that was tailored to meet the client’s needs. Timelines were tight, and since client representatives traveled to different locations to meet the testers, the schedule needed to be managed very closely. Our skilled project managers stayed in close communication with the client to inform them of any schedule changes so that adjustments could be made quickly in order to meet the testing deadlines. Our close management of any attrition was also key to achieving success; if there was a cancellation from any tester, our team was able to replace them in 24 hours.

Website for deploying AI with world class training data


Andrew Ettinger | Chief Revenue Officer

Andrew Ettinger joined Appen as Chief Revenue Officer in May 2023 overseeing the company's revenue strategies and driving growth in the field of AI. He joined Appen with more than 25 years of sales experience in sales and services in the technology industry. 

Andrew's expertise extends to harnessing the power of data to drive insights and optimize processes. As the Chief Revenue Officer at Astronomer, he successfully grew the adoption of their open-source data solution, leading to a remarkable increase in monthly downloads and revenue. His strategic initiatives resulted in a 600% growth in customer count and a 75% win rate. 

Prior to joining Astronomer, he served as the VP of Sales at Pivotal Software, where he helped grow the business from zero to $100 million in annual recurring revenue in a single year leading up to Pivotal’s initial public offering, and up to $500 million thereafter. Under his leadership, the company achieved three consecutive years of 50% revenue growth, fueling digital transformations for Fortune 500 companies in various sectors. 

Andrew holds a Bachelor of Science in Business Marketing from The Ohio State University.