Slide 1: Optimizing emissions for machine learning training
Optimizing emissions for machine learning training Sachini Ekanayake^, Tapan Shah*, Scott Evans* ^: University of Albany, *: GE Vernova Research
Slide 2: AI and Emissions
AI and Emissions Model Total Training Energy Gross tCO2e model Consumption (MWh) training Evolved Transformer 7.5 3.2 T5 85.7 46.7 Meena 232 96.4 Gshard -600B 24.1 4.8 Switch Transformer 179 72.2 BERT 1.5 1.4 GPT-3 1287 552 As AI models become ubiquitous, the numbers are going to scale. Need for alg
Slide 3: Motivation
Motivation Similarly, while training AI models, cognizance of appropriate compute resourcing (location and time is essential)
Slide 4: Dependence on location
Dependence on location • Same deep learning models trained on differently located compute infrastructure. • At certain locations, the carbon emission can be ~30 times lower compared to average. API and data provided by CodeCarbon — CodeCarbon 0.0.1 documentation (mlco2.github.io) Optimal Location of
Slide 5: Dependence on time
Dependence on time 1) Same deep learning models trained in Ohio at different timestamps. 2) At certain timestamps, the carbon emission can be half the average. API and data provided by CodeCarbon — CodeCarbon 0.0.1 documentation (mlco2.github.io) Optimal Time of Model Training can half the carbon em
Slide 6: Problem Formulation
Problem Formulation • We formulate a constrained optimization problem Portfolio of ML jobs ML jobs scheduling • The objective is to minimize the carbon Portfolio of HPC sites Scheduler emissions. Grid Energy Mix ML jobs priority queue/site • Input: ML Task characterized by model used, quality expect
Slide 7: Empirical Results on Simulate Data
Empirical Results on Simulate Data Example Setting ML Task j 0 1 2 Expected Energy 20 32 15 Optimal Scheduling Requirement Arrival Time 2 2 3 Computation time 2 8 1 Start time Margin 2 4 6 Greedy Scheduling Key Observations • For optimal scheduling, the training starts much later, using margin for m
Slide 8: Next Steps
Next Steps • Add uncertainty into expected computation and energy required as well as energy mix forecasts. • Conduct simulations for a large real world set of ML tasks. • Incorporate communication costs into the optimization. • Interface the scheduler as a wrapper to a corporate/cloud HPC schedulin