Hasty.ai UX case study - assignment feature for allocating annotation within teams
Background
Hasty uses AI to train AI and allow companies with unique data to build and deploy vision AI applications faster and more reliably across any infrastructure. Their product uses the annotated data to train their AI models that make it faster to create more annotations.
​
This continuously improving approach ensures that the users build data asset up to 12x.
Problem statement as provided in the brief
In Hasty today, annotators decide which images to work on based on “image status”. Annotators select an image, decide if they should annotate or skip it. When done, they can either set the status of the image to “done” or “to review” depending on if they have another team member double-checking the quality of the annotations
​
Although this works very well for smaller teams, for larger teams our system can be a bit too simplistic. They might want to concentrate on finishing one part of the project first so that it can be delivered to the customer, or they might want to spread out the work equally throughout their team. Depending on the use cases many users have requested an assignment feature that would allow them to assign an image to a user for annotation
​
​
The objective of this case study was to include annotation assignment feature to optimise and allocate work within a team of annotators
Roles and Responsibilities
Competitive Analysis
Ideation
Sketching
Solution Design
Prototyping
Tools
Figma
Figjam
Time
14 hours
Deliverable
A low-fidelity overview of your the UX (wireframes or sketches are fine)
A high-fidelity mockup for one widget or interaction
​
The solution is based on the current freemium subscription of Hasty. The focus is on the process that leads to the proposed solution and how would that embed with the rest of the application.
Design Process
Problem Discovery
-
Competitor review
-
3 use cases (provided by the stakeholder)
Problem Definition
-
Affinity Diagram
-
HMWs (Challenges and Opportunity)
Solution Development
-
User flow of the solution
-
Mood Boarding
-
Low fidelity sketching
Solution Delivery
-
High Fidelity Prototype
-
Setting up assets and components
Problem Discovery
Quick competitor review
In order to understand the product and its user space, I wanted to briefly review the user flow of the platform itself and the three competitors of Hasty.ai. Based on the problem statement provided in the brief and in the use cases, I chose competitors that had annotator statistics, data import and export and version control to explore the possibility of role management and allocation.
​
The competitor list was shortlisted from this website
-
Overview page shows the status rework, as well
-
Contributors can manage their own performance and quality in a dedicated view
-
Annotation leads can review the progress of the annotation in real time
-
List view for quick search and reassignment
-
Dedicated tab where issues are communicated
-
A status ‘with issues’ is shown as well
-
Annontation progress available under analytics
-
Invited contributors have a default view of assignments - role management is defined
-
Statistics to review the progress
Use cases
The users below are based on real users using Hasty today but are composites created from many different
interviews and discussions Hasty's team had with them.
​
The use cases were provided by the Hasty's team
User 1
Annotation provider -
Project manager
Most of my problems could be solved with some type of assignment-feature, where I can assign work to different team members in some smart way.
​
I need to manage prioritization in external spreedsheets in case of annotations that need to be redone
​
Any way of allowing me to assign images in batches would be very
helpful.
​
Also, it would be great if annotators could be prompted when work is assigned so I don’t have to
communicate it to them via Microsoft Teams.
​
Finally, having some way of letting annotators know that some images need urgent attention
would be great so we can move away from our spreadsheets.
User 2
Annotation provider -Annotation team lead
As my team have varying levels of experience, I need to manage workloads
​
there are also people being away on holidays or out sick -we need to reorganize the work
​
I review 5-10 images per annotator
after every working shift for quality control
​
If I find any errors, I’ll have to communicate what image has an issue and what the issue is in Microsoft Teams, via a spreadsheet I maintain.
User 3
Annotator
Work needs to be effiient and high quality
​
Emails and messages for changes arrive in a steady stream that disrupt my concerntration
​
It’s also important that there’s always another image to work on so I don’t have to spend precious
time looking for work together with our team lead.
​
I would also like to reassign an image from time to time - sometimes we have an
annotator that becomes good at specific annotation edge cases
Problem Definition
Affinity Mapping
The insights from the competitor overview as well as the use cases were clustered to identify recurring themes and gaps
​
The idea of the affinity mapping is to explore add ons to the core functionality
Emerging themes and challenges from the Affinity Mapping
Prioritisation Management
Annotation providers need to prioritise images
​
Quality/ relevance check
Annotators have the pressure to yield high quality work fast
Annotation Providers have to check for quality and relevance at the end of the day
Monitoring progress
Can we.. monitor the progress of the images annotated in datasets?
Prior image/task assignment
Annotator providers and anontators would like to reassign image/s
Annotation providers need to prioritize images
External communication
We could be slowing down process with manual entry and prioritisation in spreadsheets
Anontators have to be informed externally for urgent task assignment
Stream of messages distract the Annontators from staying focused
Let's reframe these challenges as opportunities.
Solution Development
My first step is to define the end to end user flow that addresses the challenges
​
Scope
For this assessment I am focusing on two use cases;
Annotation Provider and Annotator.
Current User Flow
The current user flow of Hasty.ai from the workspace dashboard after creating a project is as below.
Annotation Provider
Annotator
Based on the emerging themes from the affinity mapping and for the scope of this assessment I focused on the opportunity area of assigning images to other annotators in the team along with setting the priority on the assessment.
User flow of Annotation Provider
User flow of Annotator
Combining the proposed user flow into one single flow
Annotation Provider
Annotator
Inspiration and the Mood boarding for the proposed user flow
I reviewed several task management and tracking softwares such as Jira, Notion and Asana as well as LabelBox's list view for solution design
Task Management
Issues tab
Solution Delivery
the solution design focuses on two areas:
​
-
Issues tab on the workspace
-
List view and image assignment from the file management
​
The use case of performance management and image assignment from the page Manual review has not yet been handled.
​
Current design
In the current layout there is no tab to review issues and the file manager has no image assignment feature. On selecting an image, the user is redirected to annotate the image
Proposed design
​
1. Issues tab
in the proposed design annotator and annotation providers are also provided with a tab where they can review the rework issues and filter according to the priority, issues assigned to and by them.
2. List view and image assignment from the file management
Under file manage the thumbnail view can be changed in a list view. List view has more details about an image, its status as well as the assignee. The image can be annotated, assigned and prioritised.
For the scope of this project, if all the images with the status done is selected, an option to modify appears. The user can change the status to rework and assign the reworked images