BioNTech SE · NASDAQ:BNTX

BioNTech — Investor Day (October 2024)

135 slides from BioNTech's investor day presentation, Oct 1, 2024

Slide 1: ies, B
Exhibit 99.2 fd . ies, 2 B * a 2 : *“BIONT=CH | [> InstaDeep™ : / ly F 4 / : 6) fe . z aye : e . . ‘
Slide 2: Forward-looking statements
2 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 This presentation contains forward-looking statements within the meaning of the Private Securities Litigation Reform Act of 1995, as amended. In some cases, forward-looking statements can be identified by terminology such as “will,” “may,
Slide 3: Introduction and Welcome
Introduction and Welcome Ryan Richardson Chief Strategy Officer BioNTech Ugur Sahin Founder & CEO BioNTech Karim Beguir CEO InstaDeep 3 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 4: Agenda
Introduction and Vision 14:00 Welcome & Introductory Remarks 14:05 Our Vision for AI Part I. Scaling AI Capabilities 14:10 Computing Infrastructure 14:25 Innovation: Bayesian Flow Networks 14:45 DeepChain: One Platform, Multiple Tools Part II. Deploying AI across the pipeline 15:00 Applying AI end-t
Slide 5: Root Cause of Cancer Treatment Failure
DNA Mutations Healthy Cells Mutations Mutations Pre-Cancer Cell Root Cause of Cancer Treatment Failure Interindividual Variability & Intratumoral Heterogeneity Individual Patients Cancer Evolution 5-20 Years – up to 10.000 Mutations Cancer Cells Genetically Diverse & Adaptable Mutations 5 I AI Day ©
Slide 6: Towards a Potentially Curative Approach to Cancer
Towards a Potentially Curative Approach to Cancer Based on Multiple Modalities and Differentiated Novel/Novel Therapeutic Combinations Space for potentially curative approaches Immunomodulators Novel checkpoint inhibitors, cytokines, immune agonists mRNA vaccines Targeted therapy ADCs, CAR-T, TCR-T,
Slide 7: BioNTech: A pioneer in ML from day one
7 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 BioNTech: A pioneer in ML from day one
Slide 8: BioNTech and InstaDeep – The Road to Partnership
BioNTech and InstaDeep create joint AI Lab with committed budget and dedicated infrastructure 2020 BioNTech and InstaDeep begin project work 2019 2023 BioNTech invests in InstaDeep Series B round alongside Google and syndicate of investors BioNTech acquires InstaDeep to operate as a wholly owned AI
Slide 9: Two Companies: One Mission
Two Companies: One Mission BioNTech: >6,8001 Employees InstaDeep: >3701 Employees Developing medicines to fight cancer, infectious diseases and other serious diseases. Focused on productizing disruptive AI innovation HQ: Mainz, Germany HQ: London, UK Our Goal: Building a leading AI-first, personaliz
Slide 10: Charting the Course for Tomorrow's Personalized Medicine
Charting the Course for Tomorrow’s Personalized Medicine Deep genomics & immunology expertise to analyse patient data Individualized treatment platforms to address inter-individual variability In-house manufacturing AI & digitally-integrated target & drug discovery and development Drug classes Inter
Slide 11: Deploying AI end-to-end in our immunotherapy pipeline
11 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Our Goal: Deploying AI end-to-end in our immunotherapy pipeline 1. Immunohistochemistry 2. DNA/RNA sequencing 3. Proteomics 4. Protein Design 5. Lab Functional Validation AI Vision DNA LLMs AI AgentProtein LLMs Protein LLMs DNA Sequence
Slide 12: Our Approach
12 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Our Approach: 1. Scaling AI capabilities 2. Deploying across the pipeline
Slide 13: Part I. Scaling AI Capabilities
Part I. Scaling AI Capabilities 13 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 14: WHMMHAT TA
WHMMHAT TA
Slide 15: Computing Infrastructure
15 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Computing Infrastructure
Slide 16: Cluster Specifications
16 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Cluster Specifications 224 Nvidia H100 GPUs 86,000 CPU Cores 1.7 PetaBytes persistent storage 400 Gbps RoCE network InstaDeep's Supercomputing Cluster Source: Internal Data
Slide 17: Supercomputing Cluster Nearing Exascale Levels
17 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Our Supercomputing Cluster Is Nearing Exascale Levels: InstaDeep on-premise Cluster totals to ~0.5 ExaFLOPS Top 100 worldwide [1] Top 20 H100 GPU clusters worldwide [2] [1] “Top 500, The List”, June 2023 [2] “State of AI Report Compute I
Slide 18: Advanced In-House Rack Design
18 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Advanced In-House Rack Design Easy to expand with modular nodes Consistent performance, cost, power, cooling Optimized for large-scale AI workloads Simplified management with consistent architecture Minimize expenses with standard design
Slide 19: Software Stack Supporting our Cluster
Software Stack Supporting our Cluster: 19 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Cluster Management Compute Networking Storage Projects Users Frameworks & Tools Cost Management SecurityFully tailored AI stack from hardware to experiments Open standards Cutting-edge tooling
Slide 20: Strategic Benefits from our SuperComputing Cluster
20 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Strategic Benefits from our SuperComputing Cluster • Availability when most needed • Flexibility on Sw/Hw Integration • No Vendor Locking • Repeatable design • Predictable costs • Cost efficient (50% savings on cloud equivalent at 60% us
Slide 22: Scaling Laws
Scaling Laws Why How What Scaling Intelligence Engineering Expertise Accelerating Scientific Discovery 22 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 23: Scaling Laws
Scaling Laws Why How What Scaling Intelligence Engineering Expertise Accelerating Scientific Discovery 23 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 24: Scaling Laws
Scaling Laws Performance of Large Language Models (LLMs) is a smooth, well-behaved and predictable function of the number of parameters of your model, the amount of data used to train it, and computing resources. Source: Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E
Slide 25: Scaling Laws
Source: Achiam, Josh, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida et al. "Gpt-4 technical report." (2023) We can expect “more intelligence” by scaling existing algorithms. Scaling Laws 25 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 26: Scaling Laws
Scaling Laws Why How What Scaling Intelligence Engineering Expertise Accelerating Scientific Discovery 26 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 27: How to Scale Next-Generation Foundation Models
How to Scale Next-Generation Foundation Models 27 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Scaling next-generation AI systems demands advanced engineering solutions, tightly with the hardware to balance training and deployment constraints. Memory Model Sharding Rematerialization Q
Slide 28: Scaling Laws
Scaling Laws Why How What Scaling Intelligence Engineering Expertise Accelerating Scientific Discovery 28 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 29: Accelerating Reinforcement Learning
29 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 #1 Accelerating Reinforcement Learning Reinforcement Learning is the science of learning from trials and errors. A simulation engine turns computation into data. The Sebulba Architecture on 8x hardware accelerators1 Scaling Reinforcement
Slide 30: Accelerating Reinforcement Learning results
Better: 50% improvement in performance as we scale the hardware and simulated data. Cheaper: 13x cost reduction due to the more efficient use of the hardware. Faster: 240x faster to train an RL agent up to convergence. [1] “InstaDeep’s scalable reinforcement learning on Cloud TPU”, October 19, 2023,
Slide 31: Scaling the Next-Generation of our Generative AI Models
31 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Scaling the Next-Generation of our Generative AI Models • in-house JAX-based software library • best-in-class engineering for scaling LLMs e.g. Hybrid Parallelism, mixed-precision, rematerialization, etc. Results • Train multi-billion pa
Slide 32: Summary
Summary InstaDeep’s Supercomputing Cluster and advanced software stack could facilitate new scientific breakthroughs, services and products that were previously out of reach. Scaling Laws Engineering Expertise Accelerating Scientific Discoveries 32 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I Octo
Slide 33: AI Innovation: Bayesian Flow Networks
33 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 AI Innovation Bayesian Flow Networks
Slide 34: Generative AI
Generative AI A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage… Sora: generative AI model for videos OpenAI 2024 34 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Source: SORA blog post by OpenAI: https://openai.com/index/sora/
Slide 35: Joint Probability Distributions
Joint Probability Distributions For a generative model of face images, creating a new face means picking a sample from the joint probability distribution of all the pixels. Q: Why is this so hard? A: Because all the pixels are interrelated Generating Diverse High-Fidelity Images with VQ-VAE-2 (Razav
Slide 36: Steerable Generation
We want to control what we generate. With a multimodal model (e.g. images and text) we can do this with conditional sampling: fix one modality, generate another. Given an image, generate a caption (CLIP OpenAI 2021) Given an prompt, generate an image (DALL·E OpenAI 2021) Steerable Generation 36 I AI
Slide 37: One Model, Many Tasks
One Model, Many Tasks Learning a joint distribution over many variables then choosing which to fix and which to generate gives us one model for many tasks 37 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 38: But Which Model?
38 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 boss cat report Sorry ____ the ___ ate my ______ Autoregression (GPT) Masked prediction (BERT)Diffusion Pros: Continuous data (especially images), inpainting, fast gradient- based sampling Cons: Discrete data Pros: Sequence data (especia
Slide 39: Bayesian Flow Networks
Graves, Srivastava, Atkinson, Gomez 2023 Bayesian Flow Networks Unlike diffusion models, they generate discrete data in a continuous way, allowing for gradient-based sampling This makes BFNs well-suited for controllable generation across diverse data modalities. BFNs are a new class of generative mo
Slide 40: Proteomics: Joint Modelling
Generative Modelling p( ) Binders Binding assaysStructureTaxonomyGO Terms EC numberSequence AGL… PE score Structural domains pTMs AGL… , , , , , , , , , ,... Proteomics: Joint Modelling A unifying framework to learn useful functions from data (1) Learn to model the joint distribution of all your dat
Slide 41: Proteomics: Protein Folding
p( ) Binders Binding assaysStructureTaxonomyGO Terms EC numberSequence AGL… PE score Structural domains pTMs Proteomics: Protein Folding A unifying framework to learn useful functions from data (1) Learn to model the joint distribution of all your data, and (2) conditionally sample for tasks of inte
Slide 42: Proteomics: Function Prediction
Binders Binding assaysStructureTaxonomyGO Terms EC numberSequence AGL… PE score Structural domains pTMs Proteomics: Function Prediction A unifying framework to learn useful functions from data (1) Learn to model the joint distribution of all your data, and (2) conditionally sample for tasks of inter
Slide 43: Proteomics: Antibody Design
Binders Binding assaysStructureTaxonomyEC numberSequence AGL… PE score Structural domains pTMs Proteomics: Antibody Design A unifying framework to learn useful functions from data (1) Learn to model the joint distribution of all your data, and (2) conditionally sample for tasks of interest. p( )AGL…
Slide 44: Proteomics: Sequence Generation
Sequence AGL… Proteomics: Sequence Generation A unifying framework to learn useful functions from data (1) Learn to model the joint distribution of all your data, and (2) conditionally sample for tasks of interest. Generative Modelling 44 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 D
Slide 45: Natural, Diverse & Novel Protein Sequences
Natural, Diverse & Novel Protein Sequences ProtBFN learns statistical and biochemical properties of natural proteins with high-fidelity. 1. 10,000 generated sequences from each model are matched to clusterings from UniRef50. A hit is determined as a match with >50% sequence identity. Coverage score
Slide 46: Globular Structural Motifs With Novel Sequences
Globular Structural Motifs With Novel Sequences Predicted structures of generated sequences show natural, globally coherent and functionally diverse folds. Structure largely determines function in nature. Sequence AGL… Structure Function Single and multi-domain proteins. Globally coherent generation
Slide 47: BFN for Protein Sequences
BFN for Protein Sequences Outperforms or matches task-specific autoregressive, diffusion and BERT models. Improved naturalness, diversity and novelty. Uses zero-shot conditioning of model. Released only a few days ago!1 Patent application filed. 1. Available at https://www.biorxiv.org/content/10.110
Slide 48: Going Beyond Sequence-Only Models
Going Beyond Sequence-Only Models Our goal is to model everything: building foundational models of the joint distribution of heterogeneous scientific data. Performance across multiple data types and sources. Flexibility in the hands of scientists with task-specific inference. 48 I AI Day © 2024 Inst
Slide 49: Going Beyond Sequence-Only Models
Task #1 Task #2 AGL… Task Data Model Scientist A?L… AGL… AGL… AGL… AGL… Data Conventional ML AI AI Going Beyond Sequence-Only Models Our goal is to model everything: building foundational models of the joint distribution of heterogeneous scientific data. Performance across multiple data types and so
Slide 50: Going Beyond Sequence-Only Models
ModelData AGL… Our vision AI (BFN-X) A?L… AGL… AGL… Tasks determined flexibly at inference with conditional generation Scientist Going Beyond Sequence-Only Models Our goal is to model everything: building foundational models of the joint distribution of heterogeneous scientific data. Performance acr
Slide 51: Introducing AbBFN-X
Introducing AbBFN-X First look at our multimodal model for antibodies. 36 different attributes jointly modelled: sequence, genetic, biophysical Empowers scientists with tunable generation, being highly flexible across many tasks. Today’s use cases go beyond standard AI-enhanced antibody design workf
Slide 52: AbBFN-X antibody attributes
CDR-L1 CDR-L2 CDR-L3 CDR-H1 CDR-H2 CDR-H3 VH: EVQLLESGGGLVQPGGSLRLSCAASGFTFSSYAMSWVRQAPGKGLEWVSAISWNSGSIYADSVKGRFTISRDNSKNTLYLQMNSLRAEDTAVYYCARGWSQVDTAMDLDYGQGTLVTVSS D gene AbBFN-X VL: DIQMTQSPSSVSASVGDRVTITCRASQSVSSNLAWYQQKPGKAPKLLIYGASSLQSGVPSRFSGSGSGTDFTLTISSLQPEDFATYYCQQYNNWLTFGQGTRLEIK L2 L1 H
Slide 53: AbBFN-X antibody attributes
Length Attributes CDR-H1 length CDR-H2 length CDR-H3 length CDR-L1 length CDR-L2 length CDR-L3 length VH length VL length HV gene HD gene HJ gene HV seq. identity HD seq. identity HJ seq. identity LV gene LD gene LV seq. identity LJ seq. identity LC locus Species Genetic Attributes % % % % % Amino a
Slide 54: AbBFN-X antibody attributes
CDR-L1 CDR-L2 CDR-L3 CDR-H1 CDR-H2 CDR-H3 VH: EVQLLESGGGLVQPGGSLRLSCAASGFTFSSYAMSWVRQAPGKGLEWVSAISWNSGSIYADSVKGRFTISRDNSKNTLYLQMNSLRAEDTAVYYCAKDLLGSFPYDASGYYDYFDYWGQGTLVTVSS VL: DIQMTQSPSSVSASVGDRVTITCRASQSVSSNLAWYQQKPGKAPKLLIYGASSLQSGVPSRFSGSGSGTDFTLTISSLQPEDFATYYCQQANSFPPTFGQGTRLEIK L2 L1 H3 H2 H1
Slide 55: AbBFN-X antibody attributes
CDR-L1 AbBFN-X Length Attributes CDR-H1 length CDR-H2 length CDR-H3 length CDR-L1 length CDR-L2 length CDR-L3 length VH length VL length HV gene HD gene HJ gene HV seq. identity HD seq. identity HJ seq. identity LV gene LD gene LV seq. identity LJ seq. identity LC locus Species Genetic Attributes %
Slide 56: AbBFN-X antibody attributes
CDR-L1 CDR-L2 CDR-L3 CDR-H1 CDR-H2 VH: EVQLLESGGGLVQPGGSLRLSCAASGFTFSSYAMSWVRQAPGKGLEWVSAISWNSGSIYADSVKGRFTISRDNSKNTLYLQMNSLRAEDTAVYYCAKDRGGNWAILDYWGQGTLVTVSS VL: DIQMTQSPSSVSASVGDRVTITCRASQSVSSNLAWYQQKPGKAPKLLIYGASSLQSGVPSRFSGSGSGTDFTLTISSLQPEDFATYYCQQANSFPPTFGQGTRLEIK L2 L1 H3 H2 H1 L3 AbBFN-X Len
Slide 57: Example Task 1: Anti-HIV Antibody Library Design
Heavy chain synthesis. CDR-H3 design and grafting. Light chain library synthesis with desired CDR length. Paired library generation. Mutagenesis. Discard poor candidates. Generating a library of rare antibodies against HIV: Example Task 1: Anti-HIV Antibody Library Design 57 I AI Day © 2024 InstaDee
Slide 58: Example Task 1: Anti-HIV Antibody Library Design
FWR-H1 CDR-H1 FWR-H2 CDR-H2 FWR-H3 CDR-H3 FWR-H4 FWR-L1 CDR-L1 FWR-L2 CDR-L2 FWR-L3 CDR-L3 FWR-L4 AGL… AGL… AGL… AGL… AGL… AGL… AGL… AGL… AGL… AGL… AGL… AGL… AGL… AGL… Generating a library of rare antibodies against HIV: (1) identify target attributes. (2) conditionally sample for rare, desired anti
Slide 59: Example Task 1: Anti-HIV Antibody Library Design
AbBFN-X antibodies are 5600x more likely2 to have all desired characteristics. Generating a library of rare antibodies against HIV: (1) identify target attributes. (2) conditionally sample for rare, desired antibodies. Example Task 1: Anti-HIV Antibody Library Design 59 I AI Day © 2024 InstaDeep Ltd
Slide 60: Example Task 1: Anti-HIV Antibody Library Design
ARDEIYFLEWLISY AKVRLGELPYEAFDI ARGVRVQ SYNWFDP ASGEYFFDTSSYPN ARSSFVYPKSGYDFYFDY ARDIAVDPESTAYFDY AKGFSYGDGWADY VRLRVGVLPGAFDI ARDGGHYSH ASGSGDSRYAQPLWFTTAFDI ATSLNYGVIISD ASGKMAVAYYFDY AREGMDASMYYFDY ARDMGYHDGALVFDN… L2 L1 H3 H2 H1 L3 100%Unique CDRs1 Unique CDR-H3s 99%Unique CDR-L3s 100% 52%Unique
Slide 61: Example Task 2: Heavy-Light Pairing
Generating a set of developable light chains that will pair with a heavy chain: (1) Condition on desired properties and heavy sequence. (2) Sample for stable, diverse sequences. Example Task 2: Heavy-Light Pairing 61 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 62: Example Task 2: Heavy-Light Pairing
Generating a set of developable light chains that will pair with a heavy chain: (1) Condition on desired properties and heavy sequence. (2) Sample for stable, diverse sequences. FWR-H1 CDR-H1 FWR-H2 CDR-H2 FWR-H3 CDR-H3 FWR-H4 Length Attributes CDR-H1 length CDR-H2 length CDR-H3 length CDR-L1 length
Slide 63: Example Task 2: Heavy-Light Pairing
Generating a set of developable light chains that will pair with a heavy chain: (1) Condition on desired properties and heavy sequence. (2) Sample for stable, diverse sequences. Conditioning on the required heavy chain Example Task 2: Heavy-Light Pairing 63 I AI Day © 2024 InstaDeep Ltd. & BioNTech
Slide 64: Example Task 2: Heavy-Light Pairing
Generating a set of developable light chains that will pair with a heavy chain: (1) Condition on desired properties and heavy sequence. (2) Sample for stable, diverse sequences. Conditioning on the required heavy chain results in diverse light chains in length and structure Example Task 2: Heavy-Lig
Slide 65: Example Task 2: Heavy-Light Pairing
Generating a set of developable light chains that will pair with a heavy chain: (1) Condition on desired properties and heavy sequence. (2) Sample for stable, diverse sequences. Conditioning on the required heavy chain results in diverse light chains in length and structure while respecting the requ
Slide 66: Generative AI for Proteomics
Generative AI for Proteomics InstaDeep is developing next-generation GenAI models across the whole stack, from fundamental ML research, to modelling scientific data and enabling new capabilities for scientists. Bayesian Flow Networks Unified modelling of multi-modal data Task-specific conditional ge
Slide 67: DeepChain: One platform, Multiple tools
67 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 DeepChain One platform, Multiple tools
Slide 68: DeepChain tools accelerate the R&D pipeline
68 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Functional validation Lab Integration Development Stability enhancement Optimisation Binding improvement Target identification Genome annotation By combining State-of-the-art Science with Engineering, Our AI Tools aim to Accelerate the R
Slide 69: I AI , 2024 Source: Internal Data
69 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October, 2024 Source: Internal Data
Slide 70: We Are Releasing our Flagship Models on DeepChain
We Are Releasing our Flagship Models on DeepChain ProtBFN & AbBFN1 Nucleotide Transformer2 & SegmentNT3 State-of-the-art generative protein models Generates natural-like, diverse, structurally coherent, and novel protein sequences Outperforms leading autoregressive and discrete diffusion models Enab
Slide 71: We Are Releasing our Flagship Models on DeepChain
We Are Releasing our Flagship Models on DeepChain ProtBFN & AbBFN1 Nucleotide Transformer2 & SegmentNT3 State-of-the-art generative protein models Generates natural-like, diverse, structurally coherent, and novel protein sequences Outperforms leading autoregressive and discrete diffusion models Enab
Slide 72: Our Foundation Models for Genomics are State-of-the-Art
Our Foundation Models for Genomics are State-of-the-Art [1] The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics, Dalla-Torre et al., Nat. methods (press) Nucleotide transformer models are state-of-the-art in the space1 72 I AI Day © 2024 InstaDeep Ltd. & B
Slide 73: One of the Most Downloaded Genomics AI Models
One of the Most Downloaded Genomics AI Models on Hugging Face2 73 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 [1] Cumulative Downloads for Nucleotide Transformer models, 50-2.5B parameter sizes, September 2024. Hugging Face Statistics. Models Release Date: April 2023. [2] Count by fa
Slide 74: Capabilities to build and scale with our AI models
We are releasing capabilities to build and scale with our AI models 74 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 76: Optimized Setup: hardware-accelerated workflows
01 02 03 Get access to our hardware- accelerated workflows to run models with a few lines of code Send a request to the Inference API and receive a fast response containing the model’s output 76 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Optimized Setup
Slide 77: DeepChain inference is 7X faster and 2X cheaper
Running inference with DeepChain is 7X faster and 2X cheaper for in silico design of regulatory sequences* * reference methodology: Jores, T., Tonnies, J., Wrightsman, T. et al. Synthetic promoter designs enabled by a comprehensive analysis of plant core promoters. Nat. Plants 7, 842–855 (2021). * T
Slide 78: Customization
Optimized Setup 01 Customization 02 03 Customize models for your needs Customize a model for a specific task via supervised fine-tuning with our proprietary parameter efficient fine-tuning methods 78 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 79: Fine-tuning increased performance for splicing prediction
Fine-tuning a model on a specialised data set increased performance ≈1.5X for a splicing prediction use-case Relative improvement of fine-tuning for a splicing task * AUCPR: metric that measures the overall performance of a binary classification model by plotting precision against recall at differen
Slide 80: Assistants
Optimized Setup 01 Customization 02 Assistants 03 Build using natural language with Laila Laila can form hypotheses, design experiments, and call tools to analyse results collaboratively with a human scientist 80 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 82: Laila AI Agents built with Meta Llama 3.1
82 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 The Laila Series of AI Agents is built with Meta Llama 3.1 70B 405B Llama-3.1-70B-Laila-fine-tuned Llama-3.1-405B-Laila-fine-tuned Model Sizes in billions of parameters Model Versions internally fine-tuned by InstaDeep 8B Llama-3.1-8B-La
Slide 83: Laila an AI Agent for biology
83 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Capable of reasoning and making decisions Laila an AI Agent for biology Learning through constant feedback Expert knowledge of biology Integrated with powerful tools Laila is more than just a chatbot...
Slide 84: Go to deepchain.bio to gain early access
84 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Go to deepchain.bio to gain early access
Slide 85: Part II. Deploying AI Across The Pipeline
85 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Part II. Deploying AI Across The Pipeline
Slide 86: Deploying AI end-to-end in our immunotherapy pipeline
86 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Our Goal: Deploying AI end-to-end in our immunotherapy pipeline 1. Immunohistochemistry 2. DNA/RNA sequencing 3. Proteomics 4. Protein Design 5. Lab Functional Validation AI Vision DNA LLMs AI AgentProtein LLMs Protein LLMs DNA Sequence
Slide 88: The Challenge
88 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 The Challenge Pathologists face increasing workloads as demand grows for precise tumor and tissue analysis.
Slide 90: Our Approach
90 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Our Approach Develop tools using state-of-the-art AI to enhance pathologists' workflows by producing faster and higher-quality annotations.
Slide 91: AI-Assisted Tissue Annotation Tool
91 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 AI-Assisted Tissue Annotation Tool Aiming to enhance Pathologists' Precision And Efficiency Through Human-AI Collaboration
Slide 93: 5X Speed-Up
5X Speed-Up By increasing efficiency fivefold compared to manual annotation, our AI tool allows pathologists to complete annotations in a fraction of the time, optimizing resource utilization and accelerating research and development efforts. 93 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October
Slide 94: Superior Annotation Quality
Superior Annotation Quality Enabling pathologists to refine annotations at different magnification levels in Whole Slide Images, resulting in higher-quality annotations. 94 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 95: Whole Slide Image Segmentation Tool
95 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Whole Slide Image Segmentation Tool
Slide 96: From Classification To Segmentation
96 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Source: Own Data From Classification To Segmentation Use a state-of-the-art vision foundation model that we train specifically on pathology images. Decompose the image into patches. Transform the process from image segmentation to patch
Slide 98: Scalable and Fast
Scalable and Fast By processing thousands of patches in parallel, We deliver more than a 100 times speed-up compared to manual annotation. 98 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 99: Step 2: DNA Foundation Models at Nucleotide Resolution
Step 2: DNA Foundation Models at Nucleotide Resolution 99 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 100: Nucleotide Transformer: Self-Supervised Learning on Genomes
Nucleotide Transformer: Self-Supervised Learning on Genomes DNA database of genomes ATTCGACTATCCCGTAG … CCGTAG ATTCGA CTATCC CGTAG CCGTAG ATTCGA CTATCC CGTAG [MASK] random sample 12,000 bp sequence Tokenization (6-mers) random masking … … Probability Token 0.01 AAAAAA 0.01 AAAAAC 0.14 CCGTAG … 0.01
Slide 101: NT Acquires Genomics Knowledge During Pre-Training
NT Acquires Genomics Knowledge During Pre-Training 101 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Layer 1 Layer 5 Layer 21 T S N E 2. 5 B M ul ti -s pe ci es Even without supervision, information about genomic sequence features is learned in the “sequence representation” Intergenic
Slide 102: SegmentNT: Inspiration from Computer Vision
SegmentNT: Inspiration from Computer Vision Segmentation Models C om pu te r V is io n G en om ic s 102 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Source: https://techxplore.com/news/2020-05-deep-image-recognition-ability-self-driving.html
Slide 103: SegmentNT: Annotating the Genome at Nucleotide Resolution
SegmentNT: Annotating the Genome at Nucleotide Resolution 14 annotations per nucleotide (e.g. 700,000 predictions at 50kbp)enhancer promoter 5'UTR exon intron polyA 3'UTR insulators We fine-tuned the Nucleotide Transformers on 2.5M high-quality gene and regulatory elements annotations . 103 I AI Day
Slide 104: SegmentNT: Annotating the Genome at Nucleotide Resolution
The resulting SegmentNT model segments sequences up to 50kbp with state-of-the-art performance for splicing, gene finding and regulatory element detection. SegmentNT: Annotating the Genome at Nucleotide Resolution 104 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Source: adapted from d
Slide 105: SegmentNT: 700,000 Accurate Predictions over 50kbp
M od el P re di ct io ns G ro un d Tr ut h 105 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 SegmentNT: 700,000 Accurate Predictions over 50kbp in less than a second Source: de Almeida et al., 2024 (in revision)
Slide 106: SegmentNT State-of-the-Art for Splicing Detection
SegmentNT is State-of-the-Art for Canonical Splicing Detection Splicing is a biological process that removes non-coding sequences (introns) from a primary messenger RNA (mRNA) transcript and joins the coding sequences (exons) together to create a mature mRNA. Dysregulated splicing can be a vulnerabi
Slide 107: Alternative Splicing Events Detection with SegmentNT
Alternative Splicing Events Detection with SegmentNT Alternative splicing events can disrupt protein production and cancer pathways and is associated with cancer development. We finetuned segmentNT to identify tumor antigen candidates from alternative splicing events, which represent potential targe
Slide 108: Step 3: AI-Enhanced Proteomics for Target Discovery
Step 3: AI-Enhanced Proteomics for Target Discovery 108 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 109: Proteins Processed and Presented on MHC Complexes
Intracellular Proteins are Processed and Presented on MHC Complexes 109 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Created with BioRender.com
Slide 110: MHC-presented Epitopes
MHC-presented Epitopes are Immune System’s Window into the Intracellular Proteome 110 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Created with BioRender.com
Slide 111: Mass Spectrometry for MHC-presented Epitopes
Mass Spectrometry is the Current State-of-the-art for Detecting, Identifying, and Quantifying MHC-presented Epitopes 111 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Created with BioRender.com
Slide 112: Massive Database of MS Validated MHC-bound Peptides
BioNTech has a Massive Database of MS Validated MHC-bound Epitope Peptides from Studies Performed Internally and Externally ~200 million spectra 1.8 million peptides 112 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Created with BioRender.com
Slide 113: Dig Deeper into our Data to Identify Novel Targets
With the Power of AI, we can Dig Deeper into our Data to Identify Novel Therapeutic Targets 113 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 ~200 million spectra >>1.8 million peptides Created with BioRender.com
Slide 114: AI Maximizes our Ability to Discover Novel Cancer Targets
AI Maximizes our Ability to Discover Novel Cancer Targets Chromatographic Retention Time Observed Pr ed ic te d Signal intensities of peptide fragments 0 200 400 600 800 1000 1200 1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00 Observed Predicted Up to 200% increase in recovered peptide IDs Identificat
Slide 115: Targets are Validated with High-precision Mass Spectrometry
Targets are Validated with High-precision Mass Spectrometry Lung Squamous Cell Carcinoma: Example peptides validated using synthetic controls Validated targets are candidates for TCR-based therapies Underway: Novel in silico approaches to discover and enhance TCRs 115 I AI Day © 2024 InstaDeep Ltd.
Slide 116: The Challenge of HLA Mass Spectrometry
Unassigned • LncRNAs • Circular RNAs • Non-canonical open reading frames Endogenous retroviruses Post-translational modifications ?? • Canonical human protein sequences The Challenge of HLA Mass Spectrometry 55-75% of data cannot be mapped to a known human peptide Assigned 116 I AI Day © 2024 InstaD
Slide 117: Traditional Mass Spectrometry Target Decoy Search
Target Database WQIPLCTVR NRRRYTSSC YVFGGLASA FTASKTTW … ASLMPTYY Decoy Database RVTCLPIQW CSSTYRRRN ASALGGFVY WTTKSATF … YYTPMLSA 117 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 MS2 Spectrum #1 #2 #3 #4 #5 #6 #7 #8 #9 Traditional Mass Spectrometry Target Decoy Search MS2 Spectrum #1
Slide 118: De Novo Peptide Sequencing
De Novo Peptide Sequencing 118 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 ASLMPTYY ASALGGFVY RVTCLPIQW YVFGGLASA CSSTYRRRN YVFGGLASA WTTKSATF WQIPLCTVR NRRRYTSSC PeptideMS2 Spectrum 1. 2. 3. 4. 5. 6. 7. 8. 9. Sequence to Sequence AI Model
Slide 119: InstaNovo - De Novo Peptide Sequencing with Deep Learning
InstaNovo - De Novo Peptide Sequencing with Deep Learning De novo peptide sequencing using deep learning. No database needed. The approach Model trained on 28 million labeled spectra matched to 742 thousand human peptides from ProteomeTools project. The dataset The models Source: De novo peptide seq
Slide 120: InstaNovo - De Novo Peptide Sequencing with Deep Learning
InstaNovo - De Novo Peptide Sequencing with Deep Learning 120 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Preprint is available on BioRxiv https://www.biorxiv.org/content/10.1101/2023.08.30.555055v3 Code is available on GitHub https://github.com/instadeepai/instanovo • InstaNovo has
Slide 121: Step 4: Protein Design: RiboMab Platform
Step 4: Protein Design: RiboMabTM Platform 121 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024
Slide 122: BioNTech and InstaDeep strategic collaboration
122 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Source: https://investors.biontech.de/news-releases/news-release-details/biontech-and-instadeep-announce-strategic-collaboration-and-form/
Slide 123: Enabling Antibody Co-Formulation / Co-Expression
123 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Our goal: engineer the H-L interface to prevent mispairing. Correct Pairing Antibody A 12.5% Antibody B 12.5% Mispairing: 75% H-L Mispairing H-H Mispairing Co-expressed and bi-specific antibodies hold significant therapeutic interest. H
Slide 124: Our Protein Engineering Approach
124 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 We set out to introduce mutations to enforce orthogonality between neoCH1—neoCL v.s. wild- typeCH1—wild-typeCL. VH VL CH1 CL VH CH1 VL CL CH2 CH3 CH2 CH3 Antibody A neoCH1 & neoCL Antibody B w.-t.CH1 & w.-t.CL Interface mutations Abroga
Slide 125: Finding Optimal Orthogonal Mutations
125 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 A combinatorial multi-objective optimisation problem we could solve thanks to our DeepChainTM platform and an efficient in silico – in vitro feedback loop. Binding energy estimations For all correctly paired and mispaired complexes. Mit
Slide 126: >90% correct pairing
126 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 >90% correct pairing Matching the best patented designs on the market. Functional activity of antibodies confirmed. 0 20 40 60 80 100 Correct Pairing (%) Competition BioNTech/InstaDeepWild Type RiboMab Results obtained with DeepChain Co
Slide 128: Lab Automation Could Transform Research & Development
128 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 In-lab runs In-silico runs Accelerate scientific discovery Lab Automation Could Transform Research & Development Challenges Change R&D is changing constantly Complexity High complexity of science and automation Transparency Scientists n
Slide 129: Opportunities to Unlock Full Lab Automation with AI
129 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Supporting Change Management Information Discovery and Transparency Fast Protocol Development and Change Cross-Team Interaction Machine Error Diagnosis Opportunities to Unlock Full Lab Automation with AI With the assistance of Artificia
Slide 131: Case Study: liquid handler error diagnosis
Case Study: liquid handler error diagnosis 131 I AI Day © 2024 InstaDeep Ltd. & BioNTech SE I October 2024 Increased Efficiency Across Laboratory Research Activities *Stated times are for an unexpected error
Slide 132: Lab Automation Demo – Future Outlook
AI based Assay optimization Hiding Complexity of devices Carve out different group requirements Connect to BioNTech's Digital R&D Backbone Scale up to other devices Vendor Agnostic Solution Interactive and natural language based DNA/ RNA Template optimization AI-led Assay Information 132 I AI Day ©
Slide 134: Executive Summary
AI Day Executive Summary Ryan Richardson Chief Strategy Officer BioNTech Ugur Sahin Founder & CEO BioNTech Karim Beguir CEO InstaDeep
Slide 135: Thank You!
BIONTECH | [> InstaDeep™ Thank You! THE END

BioNTech — Investor Day (October 2024) on 6ix

BioNTech — Investor Day (October 2024)

Slide 1: ies, B

Slide 2: Forward-looking statements

Slide 3: Introduction and Welcome

Slide 4: Agenda

Slide 5: Root Cause of Cancer Treatment Failure

Slide 6: Towards a Potentially Curative Approach to Cancer

Slide 7: BioNTech: A pioneer in ML from day one

Slide 8: BioNTech and InstaDeep – The Road to Partnership

Slide 9: Two Companies: One Mission

Slide 10: Charting the Course for Tomorrow's Personalized Medicine

Slide 11: Deploying AI end-to-end in our immunotherapy pipeline

Slide 12: Our Approach

Slide 13: Part I. Scaling AI Capabilities

Slide 14: WHMMHAT TA

Slide 15: Computing Infrastructure

Slide 16: Cluster Specifications

Slide 17: Supercomputing Cluster Nearing Exascale Levels

Slide 18: Advanced In-House Rack Design

Slide 19: Software Stack Supporting our Cluster

Slide 20: Strategic Benefits from our SuperComputing Cluster

Slide 21: Scaling Intelligence

Slide 22: Scaling Laws

Slide 23: Scaling Laws

Slide 24: Scaling Laws

Slide 25: Scaling Laws

Slide 26: Scaling Laws

Slide 27: How to Scale Next-Generation Foundation Models

Slide 28: Scaling Laws

Slide 29: Accelerating Reinforcement Learning

Slide 30: Accelerating Reinforcement Learning results

Slide 31: Scaling the Next-Generation of our Generative AI Models

Slide 32: Summary

Slide 33: AI Innovation: Bayesian Flow Networks

Slide 34: Generative AI

Slide 35: Joint Probability Distributions

Slide 36: Steerable Generation

Slide 37: One Model, Many Tasks

Slide 38: But Which Model?

Slide 39: Bayesian Flow Networks

Slide 40: Proteomics: Joint Modelling

Slide 41: Proteomics: Protein Folding

Slide 42: Proteomics: Function Prediction

Slide 43: Proteomics: Antibody Design

Slide 44: Proteomics: Sequence Generation

Slide 45: Natural, Diverse & Novel Protein Sequences

Slide 46: Globular Structural Motifs With Novel Sequences

Slide 47: BFN for Protein Sequences

Slide 48: Going Beyond Sequence-Only Models

Slide 49: Going Beyond Sequence-Only Models

Slide 50: Going Beyond Sequence-Only Models

Slide 51: Introducing AbBFN-X

Slide 52: AbBFN-X antibody attributes

Slide 53: AbBFN-X antibody attributes

Slide 54: AbBFN-X antibody attributes

Slide 55: AbBFN-X antibody attributes

Slide 56: AbBFN-X antibody attributes

Slide 57: Example Task 1: Anti-HIV Antibody Library Design

Slide 58: Example Task 1: Anti-HIV Antibody Library Design

Slide 59: Example Task 1: Anti-HIV Antibody Library Design

Slide 60: Example Task 1: Anti-HIV Antibody Library Design

Slide 61: Example Task 2: Heavy-Light Pairing

Slide 62: Example Task 2: Heavy-Light Pairing

Slide 63: Example Task 2: Heavy-Light Pairing

Slide 64: Example Task 2: Heavy-Light Pairing

Slide 65: Example Task 2: Heavy-Light Pairing

Slide 66: Generative AI for Proteomics

Slide 67: DeepChain: One platform, Multiple tools

Slide 68: DeepChain tools accelerate the R&D pipeline

Slide 69: I AI , 2024 Source: Internal Data

Slide 70: We Are Releasing our Flagship Models on DeepChain

Slide 71: We Are Releasing our Flagship Models on DeepChain

Slide 72: Our Foundation Models for Genomics are State-of-the-Art

Slide 73: One of the Most Downloaded Genomics AI Models

Slide 74: Capabilities to build and scale with our AI models

Slide 75: Optimized Setup

Slide 76: Optimized Setup: hardware-accelerated workflows

Slide 77: DeepChain inference is 7X faster and 2X cheaper

Slide 78: Customization

Slide 79: Fine-tuning increased performance for splicing prediction