Snowflake Inc. · NYSE:SNOW
Snowflake — Data for Breakfast (D4B) Las Vegas (November 2019)
13 slides from Snowflake's conference presentation, November 2019
Slide 1: David Withers, Director - Data Engineering & Analytics
David Withers, Director - Data Engineering & Analytics
Slide 2: Snowflake definition
Snowflake definition: (n) a flake of snow, especially a feathery ice crystal, typically displaying delicate sixfold symmetry.
Slide 3: Snowflake definition (according to Tyler Durden of Fight Club)
Snowflake definition (according to Tyler Durden of Fight Club): (adj) A term to describe someone that thinks they are unique and special, but really are not. Gained popularity after the movie "Fight Club" from the quote “You are not special. You're not a beautiful and unique snowflake. You're the sa
Slide 4: Snowflake evolved definition
Snowflake evolved definition (courtesy Snowflake Data Warehouse): (n) A data warehouse that delivers the performance, simplicity, concurrency and affordability not possible with other data warehouses making users supremely unique and special in their rockstar capabilities!
Slide 5: Spireon: How Spireon Can Help Your Business
Spireon ding Business Intelligence How Spireon Can Help Your Business y<Goldstar sikahu *MFleetLocate for for Fleet for BHPH Dealer & Lender Trailer & Asset Operations
Slide 6: Unified IoT Platform Underpins All of Our Businesses
UNIFIED IOT PLATFORM UNDERPINS ALL OF OUR BUSINESSES ~4.5M 15B+ ACTIVE SUBSCRIBERS DATA EVENTS Integrated, MODULAR SUPPORTED PER YEAR Scalable, IoT Hardware and PLATFORM Software DATA 99.99% 24/7/365 SYSTEM TIER 3 AND 4 AVAILABILITY DATA CENTERS Vertically Tailored MOBILE Apps & UI Scalable Cloud Mu
Slide 7: Challenges and Results
CHALLENGES RESULTS Faster time to insight at reduced cost • Reduced costly MongoDB footprint Database Migration • Broader and real-time data sets • Insights in < 30 sec vs. 48 hours Real-time asset tracking for clients • Enhanced analytics reporting Data Sharing • Enabled by data sharing • Different
Slide 8: Snowflake @ Spireon - Evolution
Snowflake @ Spireon - Evolution ● Analytics evolution: ○ Reporting from MongoDB for both transactional + analytics ○ Separation of transactional/ODS vs analytics concerns with Snowflake ● Redshift lessons learned ○ Semi-structured data format (e.g. JSON) unfriendly ○ Administrative overhead ■ “Analy
Slide 9: Snowflake @ Spireon - Today
Snowflake @ Spireon - Today ● Building out Data Warehouse ● Batch ○ Airflow ○ Groovy Scripts ● CDC ○ Alooma ● Schema versions with release process ● Streaming ○ Kafka Connect ○ Snowpipes ● Looker
Slide 10: Kafka Connect & Snowpipe
Kafka Connect & Snowpipe ● Raw Data Payload => Event Archiver ○ Storing payloads sent from devices in the field ○ Used for analytics and debugging for hardware team ○ Used to rely on ElasticSearch with 7 days of data ○ Launched in May 2019 ● Performance Stats ○ ~ 14 billion total records in Snowflak
Slide 11: Kafka Connect & Snowpipe
Kafka Connect & Snowpipe x Kafka Topic -— Kafka Connect Data Lake Data Warehouse O
Slide 12: Snowflake @ Spireon - The Future
Snowflake @ Spireon - The Future ● More Streaming ● Use known architectural patterns (like previous slide) across the platform ● Schema Registry ○ Allow for planned and thoughtful schema evolution ○ Keep data lake and warehouse clean ○ Prevent pipelines from breaking ○ Data as a first class citizen
Slide 13: Future Opportunity: Snowflake As ODS
Future Opportunity: Snowflake As ODS ● New project: HFTP ● Events API ○ Ingesting 3800 messages / second ○ Reading 30 query / second ○ Append only data ○ Known query pattern ○ Original plan: <6 months in Atlas, >6 months in Snowflake as reports ● Why not just use Snowflake? ○ Is it performant enough
Snowflake — Data for Breakfast (D4B) Las Vegas (November 2019) on 6ix