Internet Initiative Japan Inc. · TSE:3774
Internet Initiative Japan — Internet Systems Consortium — Technical Presentation (May 2012)
42 slides from Internet Initiative Japan's technical presentation, May 8, 2012
Slide 1: Anycast Overview and Operational Experience
Anycast Overview and Operational Experience Presented by Leo Bicknell, Senior Network Architect
Slide 2: Logistics
Logistics • Webinar is 1 hour long • A recording will available by May 12 – http://www.isc.org/webinars • Participants are muted • Questions should be entered into the WebEx Q&A tab for the presenter – The presenter may defer some questions until the end of the presentation
Slide 3: Agenda
Agenda • Define Anycast • Examine use cases • Explore the impact on Internet protocols • Explore Anycast and DNS • Share ISC’s operational experience • Answer questions
Slide 4: Define Anycast
Define ANYCAST
Slide 5: What is Anycast?
What is Anycast? • Anycast describes a method of using the same IP address on multiple servers • Fundamentally, Anycast is a routing scheme • Anycast is more about the configuration of routers and routing than servers – Server admins have to understand what’s going on in order to properly operate th
Slide 6: Routing Schemes Compared
Routing Schemes Compared Baseline Unicast Broadcast Multicast Anycast Diagrams from http://en.wikipedia.org/wiki/Anycast, and are public domain.
Slide 7: Properties of Anycast
Properties of Anycast • Each packet sent to an Anycasted IP address may reach a different server • Packets are routed to the IP address with the best network metric – This is often the nearest server, but not always. Metrics could be set based on other factors, such as bandwidth, cost, load or relia
Slide 8: Examine Use Cases
Examine USE CASES
Slide 9: Use Cases
Use Cases • Local Anycast – Distributes load across multiple servers on same subnet – Eliminates need for load balancer by making the network (router) distribute traffic A➔1 Flow based A➔2 ECMP routing Server 1 ONE ROUTE! A➔3 Reduces routing issues A➔4 Server 2 Routes may originate via any supported
Slide 10: Global Anycast
Global Anycast • Distributes load across multiple locations • Provides redundancy ASN ASN ASN ASN ASN ASN ASN ASN
Slide 11: Use Cases
Use Cases • Most popular things to Anycast: 1. DNS, recursive servers • Configured by IP address on clients • Latency is important • Distribute load across multiple devices 2. DNS, authoritative • Limited number of authority IP’s can be listed in a single reply packet • Latency to the server is impo
Slide 12: Explore Impact on Protocols
Explore IMPACT ON PROTOCOLS
Slide 13: Impact on Protocols: ICMP
Impact on Protocols: ICMP • Global, stateless options work fine – Ping request/reply – ICMP Traceroute • Network instability can produce some odd results with traceroute • Avoid LAN options – Router Advertisement/Solicitation – Address Mask Request/Reply – Redirect – A unicast address on the server
Slide 14: Impact on Protocols: ICMP
Impact on Protocols: ICMP • Transmission failure messages are a problem – Destination {network,host,protocol,port} {unreachable,unknown} – Fragmentation required – Source route failed – Source host isolated – Network administratively prohibited – {Network,Host} unreachable for TOS – Communications a
Slide 15: Impact on Protocols: ICMP
Impact on Protocols: ICMP ICMP Source Quench STOP Congestion ICMP packet discarded here; other server never Client receives Net notification Query ISP B Response
Slide 16: Impact on Protocols: ICMP
Impact on Protocols: ICMP • Operationally, what really matters? • Losing “packet too big” breaks PMTU – Packets from an Anycast host should never be sent with the DF bit set • Options are to accept packets being fragmented mid-stream, or to send with the minimum MTU – IPv6 does not allow for interme
Slide 17: Impact on Protocols: UDP
Impact on Protocols: UDP • Stateless, which is good for Anycast • Works well when the query is one packet, and the response is 1-n packets, and there is no state between queries – Sounds like the majority of DNS queries! • If the query is more than one packet, or there is state between queries, the
Slide 18: Impact on Protocols: TCP
Impact on Protocols: TCP • Only works when the network path is stable. – This is never true in the long term, but is often true for short periods of time • The Unicast sender has to reach the same Anycast destination for the duration of the connection – One packet to the wrong device causes it to ge
Slide 19: Impact on Protocols: TCP
Impact on Protocols: TCP Network TCP SYN TCP SYN/ACK TCP ACK/Data TCP Reset
Slide 20: Path Instability: Sources
Path Instability: Sources 1. Load Balancing – Per-packet load balancing directs each packet to a different link and possibly server – Per-flow load balancing typically hashes on a 5-tuple, which creates a stable path for many topologies, but there are topologies where even this sort of hash won’t be
Slide 21: Impact on Protocols: TCP
Impact on Protocols: TCP • Operationally, what does it mean? – The location of the Anycast servers is important, and depends on the network topology and configuration – When properly deployed, there is a high success rate for short duration connections – The longer the connection, the greater the ri
Slide 22: Explore DNS & Anycast
Explore DNS & ANYCAST
Slide 23: DNS & Anycast
DNS & Anycast • Most common queries are a single UDP packet, with 1-3 UDP packets of response • TCP queries are extremely short lived – User->Server: SYN, ACK w/query, ACK/FIN – Server->User: SYN/ACK, ACK w/Data, ACK/FIN • Maybe an additional data packet – The FIN can be lost in some implementations
Slide 24: End User Resolvers
End User Resolvers Backbone User queries Failure reroutes Stay Regional No local user outage Regional Hub Hub Pop #1 Pop #2 Pop #3 Pop #4 Users Users Users Users
Slide 25: Anycast & DNS
Anycast & DNS • Authority servers across an ISP/Enterprise provide redundancy, load distribution and hitless maintenance ORD LGA SFO DFW ATL LAX PopQueries Failure, stay service local still up
Slide 26: Anycast & DNS
Anycast & DNS • Authority servers across multiple networks Asia centric US centric ISP ISP Europe Centric ISP - ISP redundancy Authority - Lower latency, owner’s net Keep traffic local Queries stay local
Slide 27: Anycast & DNS: Advanced
Anycast & DNS: Advanced • Inconsistent content – Part of the secret sauce in some CDN’s • Each Anycast server is loaded with a slightly different data set, and returns answers that direct users to specific servers or to names or IP’s that provide some information about the name server the user queri
Slide 28: Share ISC's Operational Experience
Share ISC’S OPERATIONAL EXPERIENCE
Slide 29: SNS@ISC
SNS@ISC • ISC’s authoritative hosting product • Present on 3 different ISP networks – Cogent, Hurricane Electric, Tata Communications • Anycast inside of each ISP – IP address space is used from each ISP inside their own network – A minimum of 3 locations on each ISP’s network • By including 3 NS re
Slide 30: SNS@ISC
SNS@ISC sa | v4 #
Slide 31: F-Root
F-Root • Three levels of Anycast – Local LAN • Each deployment has a minimum of 2 servers on the local network for redundancy, more where necessary – Local Nodes • A typical F-Root deployment at a exchange point or inside of an ISP network • Announces 192.5.5.0/24 and 2001:500:2f::/48 with NO_EXPORT
Slide 32: F-Root
F-Root 192.5.4.0/23 2001:500:2E::/47 Global Node Global Node ASN ASN NO_EXPORT ASN ASN ASN NO_EXPORT 192.5.5.0/24 192.5.5.0/24 2001:500:2F::/48 2001:500:2F::/48 Local Node Local Node IX IX IX 192.5.4.0/23 192.5.4.0/23 192.5.5.0/24 192.5.5.0/24 2001:500:2E::/47 2001:500:2E::/47 2001:500:2F::/48 2001:
Slide 33: F-Root
F-Root • Why 3 levels? – A strong desire to keep local traffic local • Local nodes may be deployed in bandwidth starved areas, like behind satellite links, and thus shouldn’t draw in queries from far away • Provide an incentive for local ISP’s to peer with the local F-Root instance – Diversity in th
Slide 34: F-Root
F-Root • Zone transfers are not officially supported, but allowed – If the long lived TCP connections fail ISC does not consider it an outage • Prior to IPv6 and DNSSEC deployment TCP queries were extremely rare – 0.00%, before DNSSEC – 0.2-0.4% after DNSSEC – Most DNS implementations handle a non-r
Slide 35: F-Root
F-Root
Slide 36: Summarize Anycast
Summarize ANYCAST
Slide 37: Summary
Summary • Anycast is a routing scheme that can be useful when deploying some applications • There are some protocol level implications that must be considered when designing an Anycast deployment • DNS is generally well suited to Anycast deployments, and is one of the most popular services to Anycas
Slide 38: Learn ISC Events
Learn ISC EVENTS
Slide 39: Events and Trainings
Events and Trainings www.isc.org/webinars www.isc.org/support/training • Despliegue y Experiencia • 3-Day IPv6 Fundamentals Operativa con Anycast – 4-6 June 2012, Amsterdam • 15 May 2012 • 2-Day DHCP Workshop • Cyber Crime Remediation – 7-8 June 2012, Amsterdam • 22 May 2012 • 2-Day Intro DNS & BIND
Slide 40: Special Offer
SPECIAL OFFER 18% discount on any training sessions, now to 30 September 2012! A thank you for attending! Coupon code in the follow up e-mail you will receive from this webinar. www.isc.org/support/training
Slide 41: Question and Answer
Ask QUESTION AND ANSWER
Slide 42: Keep in Touch
Keep in Touch www.facebook.com/InternetSystemsConsortium www.linkedin.com/company/internet-systems-consortium www.twitter.com/ISCdotORG
Internet Systems Consortium — Technical Presentation (May 2012) on 6ix