AI Image Recognition Software for Amazon AWS

Services

UX Audit, Qualitative Research, Usability Testing, Product Design, Protoyping

Outcomes

Research Insights, Product Design, UX Standards & Guidelines

Date

2019

Core Team

Background
Amazon Web Services (AWS) had a tight 6-week timeline to launch their Custom Labels product at the annual Re:Invent conference. As part of the team, I contributed to design strategy, rapid prototyping, and user testing to bring the product from concept to market quickly.
In addition to delivering a successful product launch, we also helped define a long-term design vision to guide future product development, laying the foundation for continued innovation.

How might we translate complex machine learning concepts into a user friendly tool for beginners?

Project goals

Custom Labels was designed for users with limited knowledge of machine learning and data science who need to create and train their own models. The goal was to simplify the complex process of labeling images and training models, providing an easy-to-use interface that enables non-technical users to get started quickly.
While the primary audience was non-technical business users, there was also a secondary group of technical users—data scientists and engineers—who needed to review the model results and deploy them to production. The product needed to provide tools that both persona types could use effectively. It needed to offer clear, actionable insights for technical users and a simple workflow for beginners.
Ultimately, the goal was to democratize machine learning, making it accessible to users with varying levels of expertise. The product had to balance usability for non-experts with depth for technical users, enabling everyone to build, refine, and deploy AI models with ease.

Design Approach & Process

Research and UX Audit

We kicked off with a UX audit of existing wireframes and began immersing ourselves in machine learning concepts. The goal was to ensure the product’s usability for users with varying levels of technical knowledge.
Critique of the current state of the low-fidelity wireframes to audit the UX.

Designing New AI Patterns and Workflows

One of the key challenges was designing the core experience for creating and labeling training and test datasets, which are essential for building a machine learning model. With no existing pattern to follow, we had to create an intuitive interface that enabled users to easily upload images, draw bounding boxes around objects, and label them. This task needed to be straightforward for non-technical users while still providing the necessary precision for machine learning accuracy.
The effectiveness of the model depends heavily on the quality of these labels, so getting this process right was critical. We had to ensure that users could label objects quickly and accurately without overwhelming them. Balancing the simplicity required for accessibility with the technical precision needed for training an AI model was a significant design challenge, but one that was essential for the product’s success.

Amazon AWS Polaris Design System

The product needed to adhere to Amazon's design system, Polaris, which was established to achieve a consistent experience across all of AWS products and equip developers with the tools to ship product, faster. Within a week, we needed to become experts in using and advocating for Polaris and later on, be able to defend the design and any intentional deviations to the Polaris design team.
As with any standardized design approach, it had pros and cons. While it allowed us to move quickly, which was useful in the short timeframe, it restricted design exploration to interaction models that already existed, at times to the expense of the ideal user experience. For those scenarios, we quickly explored two options, one that would achieve sign-off and another that we suggested as a net new component or iteration to Polaris.

Rapid Iterations and Approval Cycles

In 4 weeks, the design had 16 iterations, and 3 formal design sign-offs (UXSO) by Amazon's Polaris & executive design team, which we successfully signed off each time with few changes.
Cycles of feedback from the product team were on a daily cadence, requiring us to implement changes and iterate on the design rapidly, so development could continue working alongside us in parallel. Because of the fast-paced nature of the project, myself and the other designer were the key contact for the client, taking on all communication and project management in addition to the tactical UX & UI design work.
One of the views for creating a dataset, using Polaris, the AWS design system.

Usability Testing and Findings

We designed and led remote usability testing sessions with customers to gather early feedback on key concepts and identify potential usability issues. By analyzing and prioritizing feedback and bugs, we were able to recommend actionable solutions to enhance the user experience. Overall, the feedback was overwhelmingly positive, and no significant blockers emerged during testing.
One NFL customer shared, "This product simplifies and breaks down the process of training a model step by step." They added, "I’ve used other Amazon tools for model training, and they had a steep learning curve. This product makes machine learning more approachable and less overwhelming. It’s something I could—and would—definitely use."
Performing usability studies with customers like NHL & VidMob.

Setting a North Star Vision for Amazon: A Future Experience for Custom Labels

As the Amazon team took over the final stretch to launch at Re:Invent, we took a step back to take a fresh look at the product with the user feedback we had gathered. With the constraints and pressure to launch behind us, we asked ourselves:
We spent two additional weeks exploring a version of the product that resolved user feedback while pushing Polaris to incorporate modern, consumer-facing interaction models.
Our work here set the North Star vision for the product—a strategic direction that would guide future iterations, ensuring continuous improvement while staying aligned with evolving user needs. This vision would serve as a roadmap for future design enhancements, keeping the product adaptable and scalable over time.
A series of views from the north star design where the user is creating a new project and importing a dataset of non-annotated images.
One of the views where users are creating a bounding box around an object in an image to train the model.
A view of the dataset, where users can see their images, annotation status, and model results if it's been trained.
A view of the results when a model has been trained. It needed to be easily comprehendible to a non-technical user but still relevant for data scientists.

Impact and Outcomes

Amazon’s product team challenged us with what seemed like an impossible task: to design, develop, and launch a product in just 4 weeks. Through focused collaboration and leveraging our expertise in product design, we not only met this ambitious deadline but delivered a product that exceeded expectations. Our work earned the trust of internal design leads, leading to additional project referrals from Amazon. The product was well-received by customers, with positive feedback highlighting its ease of use and ability to simplify complex machine learning tasks.
"
Allison was wonderful to work with during our 6-week engagement, in which she provided us with guidance as we transitioned our AWS Console to a new design system. She quickly got up to speed with our constraints and requirements. She was candid and very easy to communicate with, as she relied on her breadth of design experience to justify her recommendations.

She delivered results quickly, made necessary tradeoffs given ambiguous requirements, and provided effective design options for our UX planning. Allison was a joy to work with, and I would look forward to working with her again!
Alexander King
Senior Software Engineer at Amazon Web Services
This case study reflects my contributions to research and design as part of a team at Teague. It does not represent the depth and scope of services provided as a fractional consultant.

Need to design and launch a new product?

Starting at
$2,000/week
or $1,500/week for female founders.
I’ll help you gain deep, human-centered insights through research. Together, we can turn those insights into actionable personas or build an insight repository to create a user-centered foundation for your business.

Services

  • Plan and conduct user research to uncover needs and behaviors
  • Synthesize insights into an artifact, such as detailed personas or journey maps
  • Build a centralized insight repository for team-wide knowledge sharing