Agilent Technologies, Inc. · NYSE:A
Agilent Technologies — Technical Presentation (2001)
42 slides from Agilent Technologies's technical presentation, Jul 11, 2001
Slide 1: Next Generation Router Test Methodology
Making IP Packets and Protocols Work - North America 2001 Next Generation Router Test Methodology
Slide 2: Introduction
Making IP Packets and Protocols Work Introduction w In this module … w we’ll examine a “popular” magazine testing methodology w examine some of the characteristics of the Internet and apply these to specific test scenarios w examine the specifics of some routing protocol performance tests w take a l
Slide 3: Some (more) Definitions
Making IP Packets and Protocols Work Some (more) Definitions w Packet loss w Number of packets lost (usually per second) w Misdirected packets w Different from packet loss - the packet wasn’t lost, but it came out of an unexpected port on the router (misdirected) w Errored packets w A packet with a
Slide 4: Some (more) Definitions
Making IP Packets and Protocols Work Some (more) Definitions w Packet latency w The time that it takes for a packet to go from point A to point B (today, we care about latency through a router) w For packets with a test payload, the time from when the IP packet leaves the IP layer, and the time at w
Slide 5: Review Current Performance Tests
Making IP Packets and Protocols Work Review Current Performance Tests w In the September 21, 1999 issue of Data Communications, a test was conducted against Juniper’s M40 router: w http://www.networkmagazine.com/article/DCM20000510S0017 w This style of test was recently repeated by Avici and the res
Slide 6: Test Configuration
Making IP Packets and Protocols Work Test Configuration Route injection Packet Generation Packet Generation
Slide 7: Methodology
Making IP Packets and Protocols Work Methodology w Throughput tests: w First, advertised 36,864 /17 prefixes (then, repeated with 73,728 prefixes). w Test repeated for both sizes with /22 length prefixes w “Fully meshed traffic” - traffic was delivered from every tester interface to every other test
Slide 8: Methodology (cont.)
Making IP Packets and Protocols Work Methodology (cont.) w Maximum forwarding table size test: w Used the QA Robot to continuously advertise blocks of 40,000 /22 routes w Repeat until the routers do not advertise out all routes that have been advertised in
Slide 9: Methodology (cont.)
Making IP Packets and Protocols Work Methodology (cont.) w Route flap test: w 73,728 /22 primary and secondary routes were advertised into the router w Packets were generated over all routes w Then, over a 30 second interval, 12,288 primary routes were withdrawn, then readvertised w Verify that pack
Slide 10: Some shortcomings ...
Making IP Packets and Protocols Work Some shortcomings ... w Complex configuration for narrowing down performance issues: w preferable to test against a single router, but to test against high-speed interfaces depends on test equipment availability w Uses /17 routes OR /22 routes w Not a realistic r
Slide 11: Some shortcomings … (cont.)
Making IP Packets and Protocols Work Some shortcomings … (cont.) w Finally, the route flapping test is actually pretty good: w 1/6th of the routes were withdrawn during 1 second, and then re-advertised after 30 seconds w Compared to the Internet (up to hundreds of route flaps per second) this is mor
Slide 12: How to make things more realistic ...
Making IP Packets and Protocols Work How to make things more realistic ... w We can build up some realistic tests by studying the Internet, and applying some of the models to our test campaign
Slide 13: Testing Forwarding Capabilities
Making IP Packets and Protocols Work Testing Forwarding Capabilities w When testing the forwarding capability of a router, we are testing: w ability of the router to process incoming packets (decrement TTL, check IP header checksum) w look up the destination within the forwarding table w get the pac
Slide 14: Useful Reference
Making IP Packets and Protocols Work Useful Reference w To see how Juniper tested the performance of their router, take a look at: w http://www.juniper.net/products/performance/250002.html
Slide 15: Testing the Forwarding Table
Making IP Packets and Protocols Work Testing the Forwarding Table w The first thing a router does (ok, second thing after TTL & checksum stuff) is to match the destination address against network prefixes in the forwarding table: 15.23.0.0 /16 A router must perform a 21.23.9.0 /24 longest prefix mat
Slide 16: Testing the Forwarding Table
Making IP Packets and Protocols Work Testing the Forwarding Table w This is the distribution of prefixes inside a real router 60.0% 54.3% 50.0% 40.0% 30.0% 20.0% 10.1% 10.0% 8.0% 8.2% 6.5% 3.9% 4.4% 1.3% 2.5% 0.0% /16 /17 /18 /19 /20 /21 /22 /23 /24
Slide 17: Testing the Forwarding Table
Making IP Packets and Protocols Work Testing the Forwarding Table This is how big forwarding tables are (and how big they’re getting) ...
Slide 18: Testing the Forwarding Table
Making IP Packets and Protocols Work Testing the Forwarding Table w So … to test forwarding performance, you must install a forwarding table that’s realistic w Plus you gotta generate traffic that exercises the forwarding table: w this is actually very key - a router has to lookup destination addres
Slide 19: Routing Protocol Testing
Making IP Packets and Protocols Work Routing Protocol Testing w Realistic traffic generation and dynamic routing changes are equally important w The Internet is not stable w Routers within the Internet core exchange a total of between 3 and 6 million routing prefixes each day w A single BGP update t
Slide 20: Routing Protocol Testing
Making IP Packets and Protocols Work Routing Protocol Testing Fluctuation of active BGP addresses within one AS over 14 days Source: http://www.telstra.net/ops/bgptable.html
Slide 21: Route Flapping
Making IP Packets and Protocols Work Route Flapping 3 Ensure traffic is 1 forwarded with no Send traffic through change in latency, the router frame loss or Router under Test throughput AS AS 200 300 netid AS route port 136.12.128 /24 200,400 2 128.47.23 /24 300,500 1 Routes used to forward traffic
Slide 22: Route Flap Convergence
Making IP Packets and Protocols Work Route Flap Convergence 1 AS AS Send routing update. AS_PATH = 300,400,500 300 400 Traffic initially 1 2 received on port 2 Simulated Router under Test 2 Transmit traffic into the Networks router, destined for network within 3 AS 500 Send routing update. 3 AS_PATH
Slide 23: Testing the switch fabric
Making IP Packets and Protocols Work Testing the switch fabric w A router’s switching fabric is a pretty complex device: w has some pretty complex allocation/memory management to get packets through quickly w the ultimate test of a router is to pound it with many small packets, on all ports, at wire
Slide 24: Testing the switch fabric
Making IP Packets and Protocols Work Testing the switch fabric w First, benchmark the behavior of a single stream w measure latency, throughput, loss (hopefully zero) of this stream w use a single destination
Slide 25: Testing the switch fabric
Making IP Packets and Protocols Work Testing the switch fabric w Then, see what happens when the fabric is loaded with unidirectional traffic w measure those same parameters again. w This loads the fabric, but takes it easy on the forwarding engines
Slide 26: Testing the switch fabric
Making IP Packets and Protocols Work Testing the switch fabric w Now, make it bi-directional w each port is both receiving and transmitting packets w any change in performance?
Slide 27: Testing the switch fabric
Making IP Packets and Protocols Work Testing the switch fabric w Finally, the full mesh test w combines fabric stress with routing stress w each packet must be looked up in the forwarding table w forces the forwarding table to be exercised w also, reveals weaknesses in allocation mechanisms. Just a
Slide 28: Testing the switch fabric
Making IP Packets and Protocols Work Testing the switch fabric w Packet lengths w … are obviously going to play a role in router performance! w Short packets w lots per second w stresses table lookup plus provides maximum number of events per second w Mixed lengths: w stress memory management routin
Slide 29: Testing the switch fabric
Making IP Packets and Protocols Work Testing the switch fabric w Internet Mix (Imix): 40 bytes 552/576 bytes About 1000 X 1500 bytes more 40 byte frames than the average
Slide 30: Multicast
Making IP Packets and Protocols Work Multicast w Multicast traffic is a performance concern w a single packet has to be sent multiple times from one port to possibly many ports w this is easy with a shared memory fabric, but really hammers a crossbar fabric w lots of multicast traffic will really sl
Slide 31: Multicast (cont.)
Making IP Packets and Protocols Work Multicast (cont.) w First step is to examine the impact on multicast traffic by the router: w set up a multicast group across some number of ports on the router w send in packets on one interface destined for this multicast group w examine latency variation betwe
Slide 32: Multicast (cont.)
Making IP Packets and Protocols Work Multicast (cont.) w Next step: do multicast packets block unicast traffic? w A router may delay unicast packets behind multicast packets as it copies the multicast packet out to each destination. w Test this by sending in a mix of multicast and unicast traffic w
Slide 33: Filters and Billing
Making IP Packets and Protocols Work Filters and Billing w Once upon a time, a service provider asked a router manufacturer to add billing capabilities to their router w the service provider wanted to install auditing capabilities in order to figure out how many packets it was routing within its own
Slide 34: A Final Note on Switch Fabric Testing
Making IP Packets and Protocols Work A Final Note on Switch Fabric Testing w A traditional technique for testing the switch fabric in an ATM switch: w inject cells in one interface w switch through fabric to another interface w loop the cell back into another interface w repeat for every interface
Slide 35: A Final Note on Switch Fabric Testing
Making IP Packets and Protocols Work A Final Note on Switch Fabric Testing w Switch had to be configured so that a cell with a vpi/vci received on one interface was switched to a specific output interface. w Since vpi/vci is relevant only to one physical interface, this could be a “normal” configura
Slide 36: Reliability Tests
Making IP Packets and Protocols Work Reliability Tests w In the grand scheme of things, performance numbers are important (especially for marketers), but reliability is almost (and maybe more) important. w Some manufacturers adopt the mantra that their router “cannot fail inside a network”. This mea
Slide 37: Finally a few notes from “the real world”
Making IP Packets and Protocols Work Finally a few notes from “the real world” w Some things that we’ve observed when testing routers: w what happens when a router gets loaded? w What happens with big routing tables? w What happens with traffic loads on many ports? w What happens at the performance
Slide 38: What happens when the switch slows down?
Making IP Packets and Protocols Work What happens when the switch slows down? w If the switch doesn’t work so good … you’ll see: w Packet latency growth, followed by packet loss w indicates that the switching fabric is choking, that buffers to the ingress of the fabric are filling, and then packets
Slide 39: Some things we’ve seen …
Making IP Packets and Protocols Work Some things we’ve seen … w Forwarding table in excess of ~60,000 entries can crash a “popular” router: w this is a function of the amount of memory available inside the router w A “Brand X” router, near the performance threshold, would exhibit different packet lo
Slide 40: Some things we’ve seen … (cont.)
Making IP Packets and Protocols Work Some things we’ve seen … (cont.) w “Brand Y” appears to shuffle packets: w Lots of sequence errors were received (but, packets weren’t being lost) w The switch was sending different flows, between the same pair of ports, along different paths within the switch fa
Slide 41: Some things we’ve seen … (cont.)
Making IP Packets and Protocols Work Some things we’ve seen … (cont.) w Packet loss during route flaps: w “Brand Y” router loses a significant number of packets during a route flaps, but: w “Brand X” router loses almost none (extremely quick to update forwarding tables and reroute traffic)!
Slide 42: Conclusion
Making IP Packets and Protocols Work Conclusion w Its all up to you … w Testing a router depends upon the test objectives: w Reliability? w Performance? w Features? w Flexibility?