top of page

Hasty.ai UX case study - assignment feature for allocating annotation within teams

d291ddb6.png
MacBook Pro 14_ - 3.png

Background

Hasty uses AI to train AI and allow companies with unique data to build and deploy vision AI applications faster and more reliably across any infrastructure. Their product uses the annotated data to train their AI models that make it faster to create more annotations.

This continuously improving approach ensures that the users build data asset up to 12x.

Problem statement as provided in the brief

In Hasty today, annotators decide which images to work on based on “image status”. Annotators select an image, decide if they should annotate or skip it. When done, they can either set the status of the image to “done” or “to review” depending on if they have another team member double-checking the quality of the annotations

Although this works very well for smaller teams, for larger teams our system can be a bit too simplistic. They might want to concentrate on finishing one part of the project first so that it can be delivered to the customer, or they might want to spread out the work equally throughout their team. Depending on the use cases many users have requested an assignment feature that would allow them to assign an image to a user for annotation

The objective of this case study was to include annotation assignment feature to optimise and allocate work within a team of annotators

Roles and Responsibilities

Competitive Analysis

Ideation

Sketching

Solution Design

Prototyping

Tools

Figma

Figjam

Time

14 hours

Deliverable

A low-fidelity overview of your the UX (wireframes or sketches are fine)

A high-fidelity mockup for one widget or interaction

The solution is based on the current freemium subscription of Hasty. The focus is on the process that leads to the proposed solution and how would that embed with the rest of the application.

ezgif.com-gif-maker (11).gif

Design Process

Problem Discovery

  • Competitor review

  • 3 use cases (provided by the stakeholder)

Problem Definition

  • Affinity Diagram

  • HMWs (Challenges and Opportunity)

Solution Development

  • User flow of the solution

  • Mood Boarding

  • Low fidelity sketching

Solution Delivery

  • High Fidelity Prototype

  • Setting up assets and components

Problem Discovery

Quick competitor review

In order to understand the product and its user space, I wanted to briefly review the user flow of the platform itself and the three competitors of Hasty.ai. Based on the problem statement provided in the brief and in the use cases, I chose competitors that had annotator statistics, data import and export and version control to explore the possibility of role management and allocation.

The competitor list was shortlisted from this website

Labelbox_Logo.jpeg
  • Overview page shows the status rework, as well

  • Contributors can manage their own performance and quality in a dedicated view

  • Annotation leads can review the progress of the annotation in real time 

  • List view for quick search and reassignment

  • Dedicated tab where issues are communicated

dtaloop.png
  • A status ‘with issues’ is shown as well

  • Annontation progress available under analytics

  • Invited contributors have a default view of assignments - role management is defined

68747470733a2f2f692e696d6775722e636f6d2f42323736654d532e706e67.png
  • Statistics to review the progress

competito selection.png

Use cases

The users below are based on real users using Hasty today but are composites created from many different
interviews and discussions Hasty's team had with them.

The use cases were provided by the Hasty's team

User 1

Annotation provider -
Project manager

Most of my problems could be solved with some type of assignment-feature, where I can assign work to different team members in some smart way.

I need to manage prioritization in external spreedsheets in case of annotations that need to be redone

Any way of allowing me to assign images in batches would be very
helpful.

Also, it would be great if annotators could be prompted when work is assigned so I don’t have to
communicate it to them via Microsoft Teams.

Finally, having some way of letting annotators know that some images need urgent attention
would be great so we can move away from our spreadsheets.

User 2

Annotation provider -Annotation team lead

As my team have varying levels of experience, I need to manage workloads

there are also people being away on holidays or out sick -we need to reorganize the work

I review 5-10 images per annotator
after every working shift for quality control

If I find any errors, I’ll have to communicate what image has an issue and what the issue is in Microsoft Teams, via a spreadsheet I maintain.

User 3

Annotator 
 

Work needs to be effiient and high quality

Emails and messages for changes arrive in a steady stream that disrupt my concerntration

It’s also important that there’s always another image to work on so I don’t have to spend precious
time looking for work together with our team lead.

I would also like to reassign an image from time to time - sometimes we have an
annotator that becomes good at specific annotation edge cases

Problem Definition

Affinity Mapping

The insights from the competitor overview as well as the use cases were clustered to identify recurring themes and gaps

The idea of the affinity mapping is to explore add ons to the core functionality

affinity map.png

Emerging themes and challenges from the Affinity Mapping

Prioritisation Management 

Annotation providers need to prioritise images

Quality/ relevance check

Annotators have the pressure to yield high quality work fast

Annotation Providers have to check for quality and relevance at the end of the day

 

Monitoring progress

Can we.. monitor the progress of the images annotated in datasets?

Prior image/task assignment 

Annotator providers and anontators would like to reassign image/s

Annotation providers need to prioritize images

 

External communication

We could be slowing down process with manual entry and prioritisation  in spreadsheets

Anontators have to be informed externally for urgent task assignment

Stream of messages distract the Annontators from staying focused

Let's reframe these challenges as opportunities.

How might we.png

Solution Development

My first step is to define the end to end user flow that addresses the challenges

Scope

For this assessment I am focusing on two use cases;

Annotation Provider and Annotator.

Current User Flow

The current user flow of Hasty.ai from the workspace dashboard after creating a project is as below. 

Annotation Provider
current flow lead.png
Annotator
current flow annotator.png

Based on the emerging themes from the affinity mapping and for the scope of this assessment I focused on the opportunity area of assigning images to other annotators in the team along with setting the priority on the assessment.

User flow of Annotation Provider
Group 2273.png
opportunity addressed 1.png
User flow of Annotator
Group 2272.png
Group 2271.png

Combining the proposed user flow into one single flow

proposed flow lead.png
Annotation Provider
Annotator
proposed flow annontator.png

Inspiration and the Mood boarding for the proposed user flow

I reviewed several task management and tracking softwares such as Jira, Notion and Asana as well as LabelBox's list view for solution design

mood board 1.png
mood board 2.png

Task Management

sketch 1.png

Issues tab

sketch 2.png

Solution Delivery

the solution design focuses on two areas:

  1. Issues tab on the workspace

  2.  List view and image assignment from the file management

The use case of performance management and image assignment from the page Manual review has not yet been handled.

Current design

In the current layout there is no tab to review issues and the file manager has no image assignment feature. On selecting an image, the user is redirected to annotate the image

Group 1.png

Proposed design

1. Issues tab

in the proposed design annotator and annotation providers are also provided with a tab where they can review the rework issues and filter according to the priority, issues assigned to and by them.

MacBook Pro 14_ - 3rr.jpg

 2. List view and image assignment from the file management

Under file manage the thumbnail view can be changed in a list view. List view has more details about an image, its status as well as the assignee. The image can be annotated, assigned and prioritised.

MacBook Pro 14_ - 9.jpg

For the scope of this project, if all the images with the status done is selected, an option to modify appears. The user can change the status to rework and assign the reworked images

MacBook Pro 14_ - s76.jpg
MacBook Pro 14_ - 8.jpg
bottom of page