top of page

Hasty.ai UX case study - assignment feature for allocating annotation within teams

d291ddb6.png

Overview

Hasty uses AI to train AI that allow companies with unique data to build and deploy vision AI applications faster and more reliably across any infrastructure. Their product uses the annotated data to train their AI models that make it faster to create more annotations. This continuously improving approach ensures that the users build data asset up to 12x.

In their current model, annotators can upload a data set and then decide which images to work on based on the “image status”. 

Problem

This process of selecting an image can be a bit too simplistic for bigger teams as the work distribution is not organised. Furthermore, Annotator lead would want to finish a part of the project or delegate and distribute the work to the team from within the platform - especially in case of issues or reannotation

Task

The objective of this case study is to include annotation assignment features to optimise and allocate work while factoring in the different needs and wants from our 3 different users

Outcome

  1.  Introducing image assignment capabilities within the team along with setting the priority

  2.  Creating a dedicated view for the issues that are to be solved by annotators

The solution is based on the current freemium subscription of Hasty. The focus is on the process that leads to the proposed solution and how would that embed with the rest of the application.

For the solution design I modified some components of Carbon Design System from IBM to resemble the layout on the platform.

Role and Responsibilities

Competitive Analysis

Ideation

Sketching

Solution Design

Prototyping

Tools

Figma

Figjam

Time

14 hours

Design Process

Problem Discovery

  • Competitor review

  • 3 use cases (provided by the stakeholder)

Problem Definition

  • Affinity Diagram

  • HMWs (Challenges and Opportunity)

Solution Development

  • User flow of the solution

  • Mood Boarding

  • Low fidelity sketching

Solution Delivery

  • High Fidelity Prototype

  • Setting up assets and components

Step 1: Problem Discovery

Understanding the product and the competition

In order to understand the product and its user space, I wanted to briefly review the user flow of the platform itself and the three competitors of Hasty.ai. Based on the problem statement provided in the brief and in the use cases, I chose competitors that had annotator statistics, data import and export and version control to explore the possibility of role management and allocation.

The competitor list was shortlisted from this website

Labelbox_Logo.jpeg
  •  Image can have rework status

  • Contributors can manage their own performance and quality

  • Annotation leads can review the progress in real time 

  • List view for quick search and reassignment

  • Separate page for issues

dtaloop.png
  • A status ‘with issues’ is shown as well

  • Annotation progress available under analytics

  • role management is defined

  • Separate page for issues

68747470733a2f2f692e696d6775722e636f6d2f42323736654d532e706e67.png
  • Statistics to review the progress

Screenshot 2023-02-01 at 15.38.23.png

Extracting the pain points from the use cases

Based on the use cases provided by Hasty.ai, I extracted the user pain points and the recommendations and needs they expressed in the excerpt. The use cases are based on real users using Hasty but are composites created from many different interviews and discussions Hasty's team had with them.

User 1

Annotation provider -
Project manager

  • User wants to assign images in batches or any other way to the annotators

  • Annotators should be prompted over assignment and priority

  • User would like to move away from managing prioritisation and assignments in spreadsheets.

User 2

Annotation provider -Annotation Team lead

  • Manage workload amidst varying experience

  • Reorganises work based on member absences

  • Needs to check for quality

  • Issues are comunicated Teams, and logged in  spreadsheets

User 3

Annotator 
 

  • Work needs to be effiient and high quality

  • Emails and messages for changes  disrupt concentration

  • It’s also important that there’s always another image to work on - saves time to find another work

  • User wants to reassign an image from time to time  for edge cases

Step 2: Problem Definition

Assembling the data

Affinity Mapping

The insights from the competitor overview as well as the use cases were clustered to identify recurring themes and gaps. Initially I used the pain points from the use cases but then noticed that the competitors had ideas that could address or map into those emerging themes. Because of the time constraint I only did one iteration of mapping.

affinity map.png

Emerging themes and challenges from the Affinity Mapping

Prioritisation Management 
  • Annotation providers need to prioritise images

Quality/ relevance check
  • Annotators have the pressure to yield high quality work fast

  • Annotation Providers have to check for quality and relevance at the end of the day

 

Monitoring progress
  • Can we.. monitor the progress of the images annotated in datasets?

Prior image/task assignment 
  • Annotator providers and anontators would like to reassign image/s

  • Annotation providers need to prioritize images

 

External communication
  • We could be slowing down process with manual entry and prioritisation  in spreadsheets

  • Anontators have to be informed externally for urgent task assignment

  • Stream of messages distract the Annontators from staying focused

Let's reframe these challenges as opportunities.

How might we.png

Step 3: Solution Development

My first step is to define the end to end user flow that addresses the challenges

Scope

For this assessment I am focusing on two use cases;

Annotation Provider and Annotator.

Current User Flow

The current user flow of Hasty.ai from the workspace dashboard after creating a project is as below. 

Annotator
current flow lead.png
Annotation Provider
current flow annotator.png

Based on the emerging themes from the affinity mapping and for the scope of this assessment I focused on the opportunity area of assigning images to other annotators in the team along with setting the priority on the assessment.

Proposed solution in an existing user flow

Annotation Provider
proposed flow lead.png
Annotator
proposed flow annontator.png

Inspiration for the proposed user flow

I reviewed several task management and tracking softwares such as Jira, Notion and Asana as well as LabelBox's list view for solution design. I wanted to understand the visualisation of task managers. 

Notion was especially interesting to explore as users can custom set the list view, thumbnail and the Kanban view of a database. Also by recognition (especially in Mac and Windows), when when users change to list view under file managers, they see more details about a said file.

mood board 1.png
mood board 2.png

Task Management

sketch 1.png

Issues tab

sketch 2.png

Step 4: Solution Delivery

the solution design focuses on two areas:

  1. Issues tab in the workspace

  2.  Thumbnail view changeable to list view for image assignment 

The use case of performance management and image assignment from the page Manual review has not yet been handled.

Current design

In the current layout there is no tab to review issues and the file manager has no image assignment feature. On selecting an image, the user is redirected to annotate the image

Group 1.png

Proposed design

1. Issues tab in the workspace

in the proposed design annotator and annotation providers are also provided with a tab where they can review the rework issues and filter according to the priority, issues assigned to and by them.

MacBook Pro 14_ - 3rr.jpg

 2.  Thumbnail view changeable to list view for image assignment ​

Under file manage the thumbnail view can be changed in a list view. List view has more details about an image, its status as well as the assignee. The image can be annotated, assigned and prioritised.

MacBook Pro 14_ - 9.jpg

For the scope of this project, if all the images with the status done is selected, an option to modify appears. The user can change the status to rework and assign the reworked images

MacBook Pro 14_ - s76.jpg
MacBook Pro 14_ - 8.jpg
bottom of page