Slide 1: ASX Announcement: Investor Technology Day Presentation
Appen Limited Level 6, 9 Help Street Chatswood NSW 2067 Tel: 02 9468 6300 www.appen.com ACN 138 878 298 For personal use only ASX ANNOUNCEMENT 20 May 2021 INVESTOR TECHNOLOGY DAY PRESENTATION Appen Limited (ASX: APX) is holding an online Investor Technology Day event commencing at 11.00am AEST today
Slide 2: Appen Limited Investor Technology Day
seeeeee eee eeeeee seeeeee eee eceees seeeeee se eees see ecceee Pe eeereey eee reeces seeveeces PPePErErey ee eeeees ereeee ny se eeeeees eeee eeeee sO eeeseeees seeesees seeee Oe ee eee sees. seereeeeee = sees eee eeeeeee eee eceeeee eT eee eS ee TEST ere eer errr eres PETSVTSESES ELSES ELEC SCES
Slide 3: Agenda
Agenda 11:00am CEO’s opening remarks and introductions Mark Brayan 11:15am Al market update Ryan Kolln, VP Corporate Development 12:00pm Break 12:20pm How our technology creates value for customers Wilson Pang, CTO 1:20pm Q&A 1:55pm CEO’s closing remarks Mark Brayan 2:00pm End appen
Slide 4: Introductions
Introductions Mark Brayan CEO <j Wilson Pang CTO Ryan Kolln VP Corporate Development appen
Slide 5: Appen is accelerating its transformation into an AI powered provider of AI data and solutions
~Appen is accelerating 90 hi: + sits transformation into EES : ee 8 an Al powered provider < of Al data and solutions @o @6-% a FOr pers “ appen
Slide 6: Appen continues to evolve
Appen continues to evolve Data type Delivery model Revenue Customer Org structure Reporting Language data Service led Project based Concentration Functional alignment Data modality, AUD Al data Product led Committed Diversification Customer alignment Strategy led, USD “ appen
Slide 7: Appen's evolution
Appen’s evolution FPhase 1: Dominate the Al data Phase 2: Build leading data . . ; . : : Phase 3: Product-led expansion )annotation services market annotation products Acquired Butler Hill 2010 Acquired Figure Eight 2019 Increased sales and marketing Acquired Leapforce 2017 Ongoing product investmen
Slide 8: Our areas of focus
Our areas of focus yee Customer Centric Our future is product led, enabling us to deliver high-quality training data, faster, at larger scale, with improved unit economics, and is a foundation for future capabilities We are aligning our operations to better support the needs of our target customer c
Slide 9: Appen plays an important role in the AI application lifecycle
»Appen plays an s important role in the Al E ® application lifecycle For personal a 2 Cc @o @¢-% rn “ appen
Slide 10: AI lifecycle
sn Al lifecycle Data acquisition Monitor Data preparation An ongoing, iterative Deploy Model build approach Human-in- the-loop Test
Slide 11: AI lifecycle: iterative approach with multiple loops
Human-in- the-loop Business need Monitor Deploy Data acquisition Test Data preparation Model build Al lifecycle An ongoing, iterative approach ... with multiple iteration loops
Slide 12: AI model, model instructions and training data
For personal use only Al model Model instructions + Training Data a ° + @0e oe . ° e . ° ° Ox rr oC Ss @-ore a on rn . e . + @> o- o see om . ee @eo- rare ae . a or << appen
Slide 13: Model instructions and training data explained
For personal use only Al model Model instructions \ Architecture for the model to learn - can be as little as 10 lines of code + Training Data — Examples used in the model training process, typically more the better 12 << appen
Slide 14: In AI, labelling data is the most important step
In Al, labelling data is the most important step Traditional software development Al model development Write code Label data | Test code | Provide instructions (i.e., model writes the code) | y Test model —— Deploy and monitor y Deploy and monitor appen
Slide 15: AI models are a series of nodes, weights and biases
Al models are a series of nodes, weights and biases appen
Slide 16: Training data are the examples that teach the model
Training data are the examples that “teach” the model Attributes of the file (annotation / label) File Attributes of the labels (metadata) 15 < appen
Slide 17: LIDAR training data example
5 LIDAR training data example vaaed a] I ‘paseUr'': “https://sampl frames” [ items ry ‘number position": ( "x": 66.4975 2" 1581 dimensior x": 186, y" 4.43, 2°: 1.86 0.42: n’ locked’: null, interpola labels”: ted”: tru ipointCount’: 120 isVolidl : true, x3 y": 184. Tab isMar imag 2317472 73.46 size’
Slide 18: Speech training data example
Speech training data example 1 —_ vidhs "5" rDEe “layerId"? "88620127-c109-4c6c-85a1-292e39F0Ff98", 00:00-15:80 / 00:04:20:84 nStareTine” 5 , © Speaker B “ontologyNane": "Speaker 8", "metadata": { 17209 Noise "transcription": { Tellme howyou'e gonna beat, "text": "Tell me how you're gonna <11.575/>
Slide 19: Training data quality is important
_ Training data quality is important (e) vA _ Low quality training data leads to = ~ poor performing models Poor quality data is not always obvious For persona 18 appen
Slide 20: Three types of problems we see with training data
Three types of problems we see with training data 3. Bias in labelling process 1. 2. Labelling Unbalanced errors icelialiaremeleice)
Slide 21: Task instructions: Draw a box around the cows
Task instructions: Draw a box around the cows
Slide 22: Task instructions: intended output
Task instructions: Draw a box around the cows Intended output — a tight bounding box around each of the visible portions of the cows
Slide 23: Labelling errors: missing labels
1. Labelling errors — missing labels Cow on the right was not labelled
Slide 24: Labelling errors: incorrect fit
1. Labelling errors - incorrect fit Bounding box is not “tight” enough around the cows
Slide 25: Labelling errors: misinterpreting instructions
1. Labelling errors — misinterpreting instructions One large box instead of three
Slide 26: Labelling errors: handling occlusion
1. Labelling errors — handling occlusion Box is placed around the expected size of the occluded cow
Slide 27: Unbalanced training data: class imbalance
2. Unbalanced training data - class imbalance
Slide 28: Corona search results comparison
For personal use only “Corona” search results – May 2021 “Corona” search results – April 2019
Slide 29: Bias in labelling process
3. Bias in labelling process Black Pudding Hagelslag Vegemite (Sprinkles on toast) Some datasets require specific knowledge / context for accurate labelling
Slide 30: AI model, instructions and training data
For personal use only Al model Model instructions + Training Data a ° + @0e oe . ° e . ° ° Ox rr oC Ss @-ore a on rn . e . + @> o- o see om . ee @eo- rare ae . a or << appen
Slide 31: Good AI model needs high quality training data
> a 5 a @ High =. —~GoodAl_ Model , Quality 3: — model instructions Training cele. iD) 3 Data or S LL “ appen
Slide 32: Our role in the AI lifecycle
usiness Deploy Appen focus areas Model build Our role in the Al ©* lifecycle is to deliver high- quality training data
Slide 33: The AI training data market is rapidly evolving
The Al training data market is rapidly evolving < appen
Slide 34: Five market trends
Five market trends 1 Data remains a major roadblock for Al 2 Aluse cases becoming narrower 3. Shift from model-centric to data-centric Al 4 Emerging need for training data operations 5 Al assisted annotations increasing “ appen
Slide 35: Data remains a major obstacle for AI
@ Data remains a major obstacle for Al In a recent survey, 18% of respondents see lack of Many Al practitioners spend most of their time data or data quality as the major bottleneck to Al collecting and preparing data 9 2% ena Be om oo ‘ «.. Airbnb have discovered that nearly Dae isccatess*"“” ——-”7
Slide 36: AI use cases becoming narrower
For personal use only • • • • • • • • •
Slide 37: Improve code or training data?
Is it better to improve code High or training GoodAl_ Model , Quality data? model instructions Training Data
Slide 38: Steel defect detection model
Steel defect detection model 93.1% 76.2% 76.2% Baseline Improve the Improve the code data Source: Landing.ai
Slide 39: ImageNet performance has capped without extra training data
@ ImageNet performance has capped without extra training da IMAGENET CHALLENGE: TOP-1 ACCURACY Source: Papers with Code, 2020; Al Index, 2021 | Chart: 2021 Al Index Report 100% 90.2%, With extra training data 90% —_ — - = , , s 86.5%, Without extra training data, 5 8 2 30% a & 70% 60% 01/2013 01/201
Slide 40: Shift from model-centric to data-centric AI
@ There is a shift from “model-centric” to “data-centric’” Al Improve volume and/or quality of the data Use available data Develop a model that compensates for noise in the data Try different models to achieve the best performance Hold the data fixed and interactively Hold the model fixed and improv
Slide 41: Need for training data management emerging
@ Need for training data management emerging Business need Monitor Deploy Human-in- the-loop Data collection Data labelling and preparation Model build Test Data collection Data ingestion Data exploration Data labelling Data validation Data preparation Version control Training data security Access c
Slide 42: AI assisted annotations increasing
For personal use only • • • • • • • • •
Slide 43: AI models rely on a variety of training data
,.Al models rely on a variety of training data + O O > 2} Q Cc D op) For personal use a 2 Cc @o @¢-% rn “ appen
Slide 44: AI enabled applications involve many different models
Al enabled applications typically involve many different models Voice interface architecture Contextual Awareness Language Processing Intent Handling User Interface Response Generation Exception Handling/Escalation Analytics/Supervision uone169u} Detailed steps - many models and interfaces User inte
Slide 45: Real-world models combine data sources and techniques
Real-world models typically use a combination of data-sources and techniques Contribution from different model/data approaches (Illustrative) US English chatbot for a bank French chatbot for a bank Qatari Arabic chatbot for a marine insurance company Appen Al training data solutions
Slide 46: Many research teams exploring new areas
For personal use only ~30k AI related publications on ARXIV in 2019 Many research teams exploring new areas
Slide 47: Popular AI techniques rely on human involvement
Popular Al techniques rely on human involvement Al technologies used in mature practices Techniques that typically require Unspenisedleaming some level of human annotation and/or preparation Simulation platform and digital twins Planning and reasoning {e} 20% 40% 60% 80% 100% Source: O'Reilly: Al Ad
Slide 48: Appen is accelerating its transformation
~Appenis accelerating 9 9° hi+ Sits transformation into a 3an Al powered provider oe < of Al data and solutions @o @¢-% rn FOr pers “ appen
Slide 49: Appen continues to evolve
Appen continues to evolve Data type Delivery model Revenue Customer Org structure Reporting Language data Service led Project based Concentration Functional alignment Data modality, AUD Al data Product led Committed Diversification Customer alignment Strategy led, USD “ appen
Slide 50: Appen's evolution
Appen’s evolution FPhase 1: Dominate the Al data Phase 2: Build leading data . . ; . : : Phase 3: Product-led expansion )annotation services market annotation products Acquired Butler Hill 2010 Acquired Figure Eight 2019 Increased sales and marketing Acquired Leapforce 2017 Ongoing product investmen
Slide 51: Technology combined with crowd of over 1m and internal expertise
We combine our industry leading technology with a crowd of over 1m and deep internal expertise 55). 238 fs oS! yw appen
Slide 52: The Appen Product Suite
The Appen Product Suite ° New products ° cx) “=0 f F_. io . : 5 Z @ grt SD) Appen Appen Data Appen Appen Appen Connect Annotation Platform Intelligence In-Platform Audit Mobile Match our global Collect and annotate Powers Appen products Organize and analyze Engages, enables and crowd workforce to tr
Slide 53: Value created from our products
Value created from our products New products $e Wee Improves speed, J Increases internal Delivers high Creates Reduces risk for customers quality, scale and | productivity and quality unit economics | crowd experience annotation competitive differentiation
Slide 54: For personal use only
For personal use only
Slide 55: Appen Data Annotation Platform
For personal use only Enables the creation of high-quality training data, faster and at scale, using innovative technology
Slide 56: For personal use only
For personal use only • • • • • • • • • • • • • •
Slide 57: Appen Connect
Appen Connect Appen Connect Match our global crowd workforce to annotation tasks eee . . e@rcree sete . e e@re e e-ee “ appen - ee @e ee eres
Slide 58: Appen Connect: intelligent marketplace to match crowd with tasks
Appen Connect - an intelligent marketplace to match our crowd with customer tasks “ eal am © A © AF B Crowd Registration Skill Project Annotation Payments Workers Preparation Qualification Project Worker Smart Match ESF ee — aL, = — all Sie << [ AuTo | [Auto | Productivity/ Project Fraud Quality Sup
Slide 59: Highly automated process to match contributors to projects
Highly automated process to match contributors to projects Process to match crowd contributors to projects Automatically obtain contributor requirements = Identify suitable contributors and send personalized notification = @ Automated Contributors ae Contributor initial confirm interest in project t
Slide 60: Automatic identification of fraudulent contributors
Automatic identification of fraudulent contributors User Info Suspicious Model Human and Signals Detection Review Behaviors User | Account Country IP Active onboarding Time 10029xx Brazil 10.21.368.xx on 1s} Project = application 40483 US -10.21368xx 90 5am Ongoing fraud * Processes 1 million+ profi
Slide 61: Appen Connect: customer benefits and future direction
Appen Connect Customer benefits and future direction Benefits for our customers Future direction * Connects customers to our global crowd * Continued optimisation of user of contributors experience * Reduces project overhead cost by * Further automation of project automating internal effort manageme
Slide 62: Appen Data Annotation Platform
Appen Data Annotation Platform cm om ow Appen Data Annotation Platform Collect and annotate training data “ appen
Slide 63: Data annotation platform capabilities
For personal use only Data Content Audio & Text & Image & Video 3D Point Collection Relevance Speech Language Cloud Job Designer / CML (Custom Markup Language)
Slide 64: Appen Workflows enables complicated AI data use cases
Appen Workflows enables complicated Al data use cases Simple user interface to build and automate multi-step data annotation projects Human & Machine Flexible settings Simple user interface Intelligence appen
Slide 65: For personal use only
For personal use only
Slide 66: Built-in quality controls
Built-in quality controls Test Questions Ground truth that tests contributors’ understanding of a job both before and while they work Conroe Tet Questont Quality Assurance Highly qualified contributors review and correct annotations within the workflow Dynamic Judgments Leverage majority vote, achie
Slide 67: Leading security solutions
y leading security solutions Flexible Deployment Options Customer Cloud an (9) Secure Crowd ‘ \ I . ape ae rao f ee ee ' sea Compliance & Certification = |> ' @o- = ' iP) Comply with legal obligations related to data —fa— [ (=>) —ia— security and privacy to protect the health, safety lad wilt j J '
Slide 68: Data Annotation Platform: customer benefits and future direction
For personal use only • • • • • • • •
Slide 69: Appen Intelligence
Appen Intelligence @ Appen Intelligence Powers Appen products with proprietary ML models eee . . e@rcree sete . e e@re e e-ee “ appen - ee @e ee eres
Slide 70: Automated pre-labelling and quality control
Appen Intelligence - automated pre-labelling and quality control Audio & Speech ++ Speaker Language diarization Detection sa t! i Speech Audio Recognition Segmentation 46 Gender Punctuation Detection Restoration Text & Language Text Gibberish Tokenization Detection Text Name Entity Classification Ex
Slide 71: OCR transcription
+$Q29Q5 600 © m- =- fo} Optical Character Recognition (OCR) Transcription For personal use only
Slide 72: For personal use only
For personal use only
Slide 73: For personal use only
For personal use only
Slide 74: LiDAR lane line segmentation
oa @ 2 @ & 2 A LiDAR Lane Line Segmentation
Slide 75: ML assisted annotation improves annotation time
ML assisted annotation improves annotation time by up to (00% ML assisted annotation time improvement vs human annotation Audio Transcription Up to 90% Audio & Speech Translation Up to 160% 2D Image Bounding Box Up to 30% 2D Image Semantic Segmentation Up to 90% Image & Video Video Annotation Up to
Slide 76: Appen Intelligence: customer benefits and future direction
Appen Intelligence Customer benefits and future direction Benefits for our customers * Improved speed of annotation enables customers to obtain more data at lower unit cost * Quality is typically improved compared to standalone human annotation * Faster time to obtain training data shortens model it
Slide 77: Appen In-Platform Audit
Appen In-Platform Audit whet Appen In-Platform Audit Organize and analyze training data to identify quality, distribution & - 2 eee. . . e@rcree sete . e e@re e e-ee “ appen - ee @e ee eres
Slide 78: In-Platform Audit helps to analyze and debug data
In-Platform Audit helps to analyze and debug data I winors a Raw data before labelling . need | 4 » Class imbalance ‘ ( peta \ * Number of examples in the raw dataset _ \ setanion pint for each class is not balanced vont cf \\ Data after fonitor ( ponien } labelling Appen Accuracy wy, > In-Platform
Slide 79: Example benefit of training data insights
Example benefit of training data insights - Attempt 1 9 million Tweets from males r- Attempt 2 9m Tweets labelled correctly - Attempt 3 5 million Tweets from males 5 million Tweets from females Class imbalance problem Accuracy problem Label imbalance problem
Slide 80: In-Platform Audit: customer benefits and future direction
Appen In-Platform Audit Customer benefits and future direction Benefits for our customers Future direction * Ability to visually explore data to identify * Evaluate model performance labelling errors * Support more training data ¢ Enables identification of training data management needs composition
Slide 81: Appen Mobile
Appen Mobile Appen Mobile Engages, enables and expands crowd sete e@re ° e-ee < appen - ee @e ee eres
Slide 82: Appen Mobile provides enhanced crowd experience
Appen Mobile provides an enhanced crowd experience and enables new use cases * Simple and intuitive user experience * Supports collecting many data modalities including audio, video, text, conversation, eye- tracking, and hand-written text * Enables location-based data collection, heightened during
Slide 83: For personal use only
For personal use only
Slide 84: Appen Mobile: customer benefits and future direction
Appen Mobile Customer benefits and future direction Benefits for our customers * Improved sign-on and task experience for contributors leads to a broader crowd base * Access to mobile-only workforce * Mobile form factor provides enhanced data collection capabilities Future direction * Collaborate wi
Slide 85: Recap and closing
For personal use only Recap and closing 84 “ appen
Slide 86: The Appen Product Suite
The Appen Product Suite ° New products ° cx) “=0 f F_. io . : 5 Z @ grt SD) Appen Appen Data Appen Appen Appen Connect Annotation Platform Intelligence In-Platform Audit Mobile Match our global Collect and annotate Empower Appen products Organize and analyze Engages, enables and crowd workforce to t
Slide 87: Value created from our products
Value created from our products New products $e Wee Improves speed, J Increases internal Delivers high Creates Reduces risk for customers quality, scale and | productivity and quality unit economics | crowd experience annotation competitive differentiation
Slide 88: Unique combination of technology, crowd and expertise
Appen has the unique combination of industry leading technology, crowd of over 1m and deep internal expertise appen
Slide 89: Q&A
For personal use only Q&A - 2 eee... ere e eor-ee 88 “ appen - 22 @e ee eres
Slide 90: For personal use only
For personal use only
Slide 91: For personal use only
For personal use only