Slide 1: Intel AI DevCon 2018
INTEL Al DEVCON 2018
Slide 2: Winning solution to Kaggle's ship vs Iceberg competition
Winning solution to Kaggle’s ship vs Iceberg competition David Austin & Weimin Wang 5/23/2018
Slide 3: Ships vs icebergs competition
Ships vs icebergs competition • AI Competitions • General workflow • Specific solution workflow • Tips for competitions
Slide 4: AI Competitions
AI Competitions Real World Competitions Establish business problem Yes No Collect data Yes No Deployment Considerations Yes No Complexity vs Accuracy Yes No Algorithm constraints Yes Maybe Problem target value Yes Yes Adapted from Coursera’s “How to Win a Data Science Competition” 4
Slide 5: Competition Sponsors
Competition Sponsors • Platform for data science competitions. • Norwegian based energy company, presence • Hosts competitions by providing in 20 countries. community, data, and prize. • Focused on energy exploration, development, • Data science community collaborates and production of oil, gas, and
Slide 6: The Problem
The Problem Develop an algorithm to classify ships vs icebergs using SAR satellite images 6
Slide 7: Dataset
Dataset icebergs Dataset predict 3425 images ships (75x75x2) 1604 images Model [(75,75,2), inc_angle] Input Evaluation metric: Log loss 7
Slide 8: Solution Pipeline
Solution Pipeline Problem Understanding Modeling • EDA • Algo Eval • Model selection • Diversity, diversity, diversity • Ensembling • Post processing 8
Slide 9: Image Exploration
ImageExploration Data exploration Asymmetrical, larger volume, amorphous Side lobes, diffraction Indistinguishable spikes Possible background bias Conclusion from visual analysis: No clear consistent pattern that would guide specific design considerations. 9
Slide 10: Image visualizations: Grad-cam
Image visualizations: Grad-cam Example intermediate convolution visualization 10 10
Slide 11: inc_angle
Key finding Clear inc_angle alternating patterns of dense clusters of ships vs icebergs Tips: • Visualize each input feature vs target value. • Cluster each input feature and compare to target value Jitter 11
Slide 12: Solution Architecture
Solution Architecture Group1 Predict CNN1 Final Prediction Stacking VGG Group2 Ensemble Predict Stacking All images CNN2 Stacking VGG Post Ensemble Processing CNN Group2 only 12
Slide 13: Custom CNN Architecture
Custom CNN Architecture Low High Conv Layers 2 4 Conv size 16 256 image preprocessing FC Layers 1 3 xder = np.array([[-1,0,1],[-2,0,2],[-1,0,1]]) FC size 16 512 yder = np.array([[1,2,1],[0,0,0],[-1,-2,-1]]) Dropout 0.2 0.3 smooth = np.array([[1,1,1],[1,5,1],[1,1,1]]) Filter size 2x2 3x3 xder2 = np.a
Slide 14: VGG16 architecture
VGG16 architecture Low High FC layers 2 3 FC size 256 1024 Pooling GlobalMax Max ● VGG image preprocessing - synthesize third channel with random weighting 14
Slide 15: Stacking overview
Stacking overview Train Set Test Set Model 1 to n for training Model 1 to n for test CV1 We will have 4 CV2 sets of predict ... ... CV3 ions on test set as well CV4 Stacking uses hold-out predictions from a model as features for a different model 15
Slide 16: Ensembling
Ensembling Ensemble Ensemble CV Greedy Stacking Blending ○ Sort each prediction according to CV score ○ Using CV predictions from CNN, VGG, and ○ Scan from best to worst, add prediction in only if Xgboost of image stats as L1 features median cv score improves ○ Create new 2 layer CNN as level 2 stac
Slide 17: Solution Architecture
Solution Architecture Group1 0.18 Predict CNN1 Final Prediction Stacking VGG 0.14 Group2 Ensemble Predict 0.082 Stacking All images CNN2 Stacking VGG 0.10 Post Ensemble Processing CNN Group2 only 17
Slide 18: Post processing
Post processing • For incorporating learning that base models don’t handle – Group identical inc_angles – Unsupervised inc_angle clustering – inc_angle trending – log_loss clipping 18
Slide 19: Solution differentiation
Solution differentiation 1. Identified inc_angle signal 2. Very large ensemble of weak base learners 3. Separating group1 and group2 into different models 4. Careful management of log_loss 19
Slide 20: Key learnings
Key learnings ICEBERG COMPETITION COMPETITIONS IN GENERAL • EDA was key • Difference from real world • Diversity and ensembling • Great for learning • Great to team up, • Outstanding for collaboration especially cross geo 20
Slide 21: Slide 21