Hasty.ai UX case study - assignment feature for allocating annotation within teams
Overview
Hasty uses AI to train AI that allow companies with unique data to build and deploy vision AI applications faster and more reliably across any infrastructure. Their product uses the annotated data to train their AI models that make it faster to create more annotations. This continuously improving approach ensures that the users build data asset up to 12x.
​
In their current model, annotators can upload a data set and then decide which images to work on based on the “image status”.
Problem
This process of selecting an image can be a bit too simplistic for bigger teams as the work distribution is not organised. Furthermore, Annotator lead would want to finish a part of the project or delegate and distribute the work to the team from within the platform - especially in case of issues or reannotation
Task
The objective of this case study is to include annotation assignment features to optimise and allocate work while factoring in the different needs and wants from our 3 different users
Outcome
-
Introducing image assignment capabilities within the team along with setting the priority
-
Creating a dedicated view for the issues that are to be solved by annotators
​
The solution is based on the current freemium subscription of Hasty. The focus is on the process that leads to the proposed solution and how would that embed with the rest of the application.
​
For the solution design I modified some components of Carbon Design System from IBM to resemble the layout on the platform.
Role and Responsibilities
Competitive Analysis
Ideation
Sketching
Solution Design
Prototyping
Tools
Figma
Figjam
Time
14 hours
Design Process
Problem Discovery
-
Competitor review
-
3 use cases (provided by the stakeholder)
Problem Definition
-
Affinity Diagram
-
HMWs (Challenges and Opportunity)
Solution Development
-
User flow of the solution
-
Mood Boarding
-
Low fidelity sketching
Solution Delivery
-
High Fidelity Prototype
-
Setting up assets and components
Step 1: Problem Discovery
Understanding the product and the competition
In order to understand the product and its user space, I wanted to briefly review the user flow of the platform itself and the three competitors of Hasty.ai. Based on the problem statement provided in the brief and in the use cases, I chose competitors that had annotator statistics, data import and export and version control to explore the possibility of role management and allocation.
​
The competitor list was shortlisted from this website
-
Image can have rework status
-
Contributors can manage their own performance and quality
-
Annotation leads can review the progress in real time
-
List view for quick search and reassignment
-
Separate page for issues
-
A status ‘with issues’ is shown as well
-
Annotation progress available under analytics
-
role management is defined
-
Separate page for issues
-
Statistics to review the progress
Extracting the pain points from the use cases
Based on the use cases provided by Hasty.ai, I extracted the user pain points and the recommendations and needs they expressed in the excerpt. The use cases are based on real users using Hasty but are composites created from many different interviews and discussions Hasty's team had with them.
User 1
Annotation provider -
Project manager
-
User wants to assign images in batches or any other way to the annotators
-
Annotators should be prompted over assignment and priority
-
User would like to move away from managing prioritisation and assignments in spreadsheets.
User 2
Annotation provider -Annotation Team lead
-
Manage workload amidst varying experience
-
Reorganises work based on member absences
-
Needs to check for quality
-
Issues are comunicated Teams, and logged in spreadsheets
User 3
Annotator
-
Work needs to be effiient and high quality
-
Emails and messages for changes disrupt concentration
-
It’s also important that there’s always another image to work on - saves time to find another work
-
User wants to reassign an image from time to time for edge cases
Step 2: Problem Definition
Assembling the data
Affinity Mapping
The insights from the competitor overview as well as the use cases were clustered to identify recurring themes and gaps. Initially I used the pain points from the use cases but then noticed that the competitors had ideas that could address or map into those emerging themes. Because of the time constraint I only did one iteration of mapping.
Emerging themes and challenges from the Affinity Mapping
Prioritisation Management
-
Annotation providers need to prioritise images
​
Quality/ relevance check
-
Annotators have the pressure to yield high quality work fast
-
Annotation Providers have to check for quality and relevance at the end of the day
Monitoring progress
-
Can we.. monitor the progress of the images annotated in datasets?
Prior image/task assignment
-
Annotator providers and anontators would like to reassign image/s
-
Annotation providers need to prioritize images
External communication
-
We could be slowing down process with manual entry and prioritisation in spreadsheets
-
Anontators have to be informed externally for urgent task assignment
-
Stream of messages distract the Annontators from staying focused
Let's reframe these challenges as opportunities.
Step 3: Solution Development
My first step is to define the end to end user flow that addresses the challenges
​
Scope
For this assessment I am focusing on two use cases;
Annotation Provider and Annotator.
Current User Flow
The current user flow of Hasty.ai from the workspace dashboard after creating a project is as below.
Annotator
Annotation Provider
Based on the emerging themes from the affinity mapping and for the scope of this assessment I focused on the opportunity area of assigning images to other annotators in the team along with setting the priority on the assessment.
Proposed solution in an existing user flow
Annotation Provider
Annotator
Inspiration for the proposed user flow
I reviewed several task management and tracking softwares such as Jira, Notion and Asana as well as LabelBox's list view for solution design. I wanted to understand the visualisation of task managers.
​
Notion was especially interesting to explore as users can custom set the list view, thumbnail and the Kanban view of a database. Also by recognition (especially in Mac and Windows), when when users change to list view under file managers, they see more details about a said file.
Task Management
Issues tab
Step 4: Solution Delivery
the solution design focuses on two areas:
-
Issues tab in the workspace
-
Thumbnail view changeable to list view for image assignment
​
The use case of performance management and image assignment from the page Manual review has not yet been handled.
​
Current design
In the current layout there is no tab to review issues and the file manager has no image assignment feature. On selecting an image, the user is redirected to annotate the image
Proposed design
​
1. Issues tab in the workspace
in the proposed design annotator and annotation providers are also provided with a tab where they can review the rework issues and filter according to the priority, issues assigned to and by them.
2. Thumbnail view changeable to list view for image assignment ​
Under file manage the thumbnail view can be changed in a list view. List view has more details about an image, its status as well as the assignee. The image can be annotated, assigned and prioritised.
For the scope of this project, if all the images with the status done is selected, an option to modify appears. The user can change the status to rework and assign the reworked images