Recent from talks
Contribute something
Nothing was collected or created yet.
Anycast
View on Wikipedia
| Communication protocol | |
Visualization of anycast routing | |
| Purpose | To route traffic to the closest server |
|---|---|
| Developer(s) | Craig Partridge, Trevor Mendez, Walter Milliken at BBN |
| Introduction | 1989 |
| RFC(s) | 1546, 2526, 4291, 4786... |
Anycast is a network addressing and routing methodology in which a single IP address is shared by devices (generally servers) in multiple locations. Routers direct packets addressed to this destination to the location nearest the sender, using their normal decision-making algorithms, typically the lowest number of BGP network hops. Anycast routing is widely used by content delivery networks such as web and name servers, to bring their content closer to end users.
History
[edit]The first documented use of anycast routing for topological load-balancing of Internet-connected services was in 1989;[1][2] the technique was first formally documented in the IETF four years later.[3] It was first applied to critical infrastructure in 2001 with the anycasting of the I-root nameserver.[2]
Early objections
[edit]Early objections to the deployment of anycast routing centered on the perceived conflict between long-lived TCP connections and the volatility of the Internet's routed topology. In concept, a long-lived connection, such as an FTP file transfer (which can take hours to complete for large files) might be re-routed to a different anycast instance in mid-connection due to changes in network topology or routing, with the result that the server changes mid-connection, and the new server is not aware of the connection and does not possess the TCP connection state of the previous anycast instance.
In practice, such problems were not observed, and these objections dissipated by the early 2000s. Many initial anycast deployments consisted of DNS servers, using principally UDP transport.[4][2] Measurements of long-term anycast flows revealed very few failures due to mid-connection instance switches, far fewer (less than 0.017%[5] or "less than one flow per ten thousand per hour of duration"[1] according to various sources) than were attributed to other causes of failure. Numerous mechanisms were developed to efficiently share state between anycast instances.[6] And some TCP-based protocols, notably HTTP, incorporated "redirect" mechanisms, whereby anycast service addresses could be used to locate the nearest instance of a service, whereupon a user would be redirected to that specific instance prior to the initiation of any long-lived stateful transaction.[1][7]
Internet Protocol version 4
[edit]Anycast is implemented in the Internet via Border Gateway Protocol (BGP), where multiple hosts (usually in different geographic areas) are given the same anycast IP address and they all announce it to their BGP table. Routers consider these to be alternative routes to the same destination, even though they are actually routes to different destinations with the same address. Routers will often select the path with fewer hops, which leads to the one closer to the client as being selected (often faster). It doesn't always happen, as some routers will choose other metrics (least congested, cheapest).
Internet Protocol version 6
[edit]Anycast is supported explicitly in the IPv6 addressing architecture.[8] The lowest address within an IPv6 subnet (interface identifier 0) is reserved as the "Subnet Router" anycast address. In addition, the highest 128 interface identifiers within a subnet are also reserved as anycast addresses.[9]
| Subnet Prefix | interface identifier | CIDR notation | |
|---|---|---|---|
| Subnet router | any | :: | ::0/124 |
| Anycast | any | ffff:ffff:ffff:ff80 to ffff:ffff:ffff:ffff | ::ffff:ffff:ffff:ff80/121 |
| Mobility Support | any | ffff:ffff:ffff:fffe | ::ffff:ffff:ffff:fffe/124 |
Most IPv6 routers on the path of an anycast packet through the network will not distinguish it from a unicast packet, but special handling is required from the routers near the destination (that is, within the scope of the anycast address) as they are required to route an anycast packet to the "nearest" interface within that scope which has the proper anycast address, according to whatever measure of distance (hops, cost, etc.) is being used.
The method used in IPv4 of advertising multiple routes in BGP to multiply-assigned unicast addresses also still works in IPv6, and can be used to route packets to the nearest of several geographically dispersed hosts with the same address. This approach, which does not depend on anycast-aware routers, has the same use cases together with the same problems and limitations as in IPv4.
Applications
[edit]With the growth of the Internet, network services increasingly have high-availability requirements. As a result, operation of anycast services has grown in popularity among network operators.[10]
Domain Name System
[edit]All Internet root nameservers are implemented as clusters of hosts using anycast addressing.[11] All 13 root servers A–M exist in multiple locations, with 11 on multiple continents. (Root servers B and H exist in two U.S. locations.)[12][13][14] The servers use anycast address announcements to provide a decentralized service. This has accelerated the deployment of physical (rather than logical) root servers outside the United States. Many commercial DNS providers have switched to an IP anycast environment to increase query performance and redundancy, and to implement load balancing.[2]
IPv6 transition
[edit]In IPv4 to IPv6 transitioning, anycast addressing may be deployed to provide IPv6 compatibility to IPv4 hosts. This method, 6to4, uses a default gateway with the IP address 192.88.99.1.[15] This allows multiple providers to implement 6to4 gateways without hosts having to know each individual provider's gateway addresses. 6to4 has been deprecated[16] in response to native IPv6 becoming more prevalent.
Content delivery networks
[edit]Content delivery networks may use anycast for actual HTTP connections to their distribution centers, or for DNS. Because most HTTP connections to such networks request static content such as images and style sheets, they are generally short-lived and stateless across subsequent TCP sessions. The general stability of routes and statelessness of connections makes anycast suitable for this application, even though it uses TCP.[5][1]
Connectivity between Anycast and Multicast network
[edit]Anycast rendezvous point can be used in Multicast Source Discovery Protocol (MSDP) and its advantageous application as Anycast RP is an intra-domain feature that provides redundancy and load-sharing capabilities. If the multiple anycast rendezvous point is used, IP routing automatically will select the topologically closest rendezvous point for each source and receiver. It would provide a multicast network with the fault tolerance requirements.[17]
Security
[edit]Anycast allows any operator whose routing information is accepted by an intermediate router to hijack any packets intended for the anycast address. While this at first sight appears insecure, it is no different from the routing of ordinary IP packets, and no more or less secure. As with conventional IP routing, careful filtering of who is and is not allowed to propagate route announcements is crucial to prevent man-in-the-middle or blackhole attacks. The former can also be prevented by encrypting and authenticating messages, such as using Transport Layer Security, while the latter can be frustrated by onion routing.
Reliability
[edit]Anycast is normally highly reliable, as it can provide automatic failover without adding complexity or new potential points of failure. Anycast applications typically feature external "heartbeat" monitoring of the server's function, and withdraw the route announcement if the server fails. In some cases this is done by the actual servers announcing the anycast prefix to the router over OSPF or another IGP. If the servers die, the router will automatically withdraw the announcement. "Heartbeat" functionality is important because, if the announcement continues for a failed server, the server will act as a "black hole" for nearby clients; this is the most serious mode of failure for an anycast system. Even in this event, this kind of failure will only cause a total failure for clients that are closer to this server than any other, and will not cause a global failure. However, even the automation necessary to implement "heartbeat" routing withdrawal can itself add a potential point of failure, as seen in the 2021 Facebook outage.
Mitigation of denial-of-service attacks
[edit]In denial-of-service attacks, a rogue network host may advertise itself as an anycast server for a vital network service, to provide false information or simply block service.
Anycast methodologies on the Internet may be exploited to distribute DDoS attacks and reduce their effectiveness: As traffic is routed to the closest node, a process over which the attacker has no control, the DDoS traffic flow will be distributed amongst the closest nodes. Thus, not all nodes might be affected. This may be a reason to deploy anycast addressing.[18] The effectiveness of this technique depends upon maintaining the secrecy of any unicast addresses associated with anycast service nodes, however, since an attacker in possession of the unicast addresses of individual nodes can attack them from any location, bypassing anycast addressing methods.[19]
Local and global nodes
[edit]Some anycast deployments on the Internet distinguish between local and global nodes to benefit the local community, by addressing local nodes preferentially. An example is the Domain Name System. Local nodes are often announced with the no-export BGP community to prevent hosts from announcing them to their peers, i.e. the announcement is kept in the local area. Where both local and global nodes are deployed, the announcements from global nodes are often AS prepended (i.e. the AS is added a few more times) to make the path longer so that a local node announcement is preferred over a global node announcement.[20]
See also
[edit]- Multihoming
- Line hunting, for an equivalent system for telephones
References
[edit]- ^ a b c d Woodcock, Bill (June 1996). "Best Practices in Anycast Routing" (PDF). Packet Clearing House.
- ^ a b c d Hernandez, Gael (October 10, 2017). "Building and Operating a Global Anycast Network" (PDF). Eurasia Network Operators Group.
- ^ C. Partridge; T. Mendez; W. Milliken (November 1993). Host Anycasting Service. Network Working Group. doi:10.17487/RFC1546. RFC 1546. Informational.
- ^ Woodcock, Bill (November 14, 2019). "TCP and Anycast". NANOG mailing list archive. North American Network Operators Group.
- ^ a b Levine, Matt; Lyon, Barrett; Underwood, Todd (June 2006). "TCP Anycast: Don't Believe the FUD - Operational experience with TCP and Anycast" (PDF). North American Network Operators Group.
- ^ Herrin, William. "Anycast TCP Architecture". Retrieved October 11, 2021.
- ^ Katz-Bassett, Ethan; Gao, Ryan (July 2019). "Impact of TCP Loss on Regional Application Performance" (PDF). Microsoft.
Azure Frontdoor uses anycast redirection to direct users to a nearby edge.
- ^ R. Hinden; S. Deering (February 2006). IP Version 6 Addressing Architecture. Network Working Group. doi:10.17487/RFC4291. RFC 4291. Draft Standard. Obsoletes RFC 3513. Updated by RFC 5952, 6052, 7136, 7346, 7371 and 8064.
- ^ D. Johnson; S. Deering (March 1999). Reserved IPv6 Subnet Anycast Addresses. Network Working Group. doi:10.17487/RFC2526. RFC 2526. Proposed Standard.
- ^ J. Abley; K. Lindqvist (December 2006). Operation of Anycast Services. IETF Network Working Group. doi:10.17487/RFC4786. BCP 126. RFC 4786. Best Current Practice 126.
- ^ T. Hardie (April 2002). Distributing Authoritative Name Servers via Shared Unicast Addresses. Network Working Group. doi:10.17487/RFC3258. RFC 3258. Informational.
- ^ Home-page B-root DNS server, visited 8 Feb. 2015
- ^ "Report on Root Nameserver Locations". Packet Clearing House. Retrieved February 21, 2011.
- ^ "Root Server Technical Operations Assn". root-servers.org. Retrieved February 16, 2013.
- ^ C. Huitema (June 2001). An Anycast Prefix for 6to4 Relay Routers. Network Working Group. doi:10.17487/RFC3068. RFC 3068. Informational. Obsoleted by RFC 7526.
- ^ O. Troan (May 2015). B. Carpenter (ed.). Deprecating the Anycast Prefix for 6to4 Relay Routers. Internet Engineering Task Force. doi:10.17487/RFC7526. BCP 196. RFC 7526. Best Current Practice 196. Obsoletes RFC 3068 and 6732.
- ^ "Anycast Rendezvous Point". Cisco Systems. June 1, 2001.
- ^ "ICANN Factsheet on root server attack on 6 February 2007" (PDF). Factsheet. The Internet Corporation for Assigned Names and Numbers (ICANN). March 1, 2007. Retrieved February 21, 2011.
- ^ Metz, C. (2002). "IP Anycast: Point-to-(Any) Point Communication (sign-in required)". IEEE Internet Computing. 6 (2). IEEE: 94–98. doi:10.1109/4236.991450.
- ^ Oki, Eiji; Rojas-Cessa, Roberto; Tatipamula, Mallikarjun; Vogt, Christian (April 24, 2012). Advanced Internet Protocols, Services, and Applications. John Wiley & Sons. pp. 102 & 103. ISBN 978-0-470-49903-0. Archived from the original on January 5, 2020.
External links
[edit]- Best Practices in IPv4 Anycast Routing Tutorial on anycast routing configuration.
Anycast
View on GrokipediaFundamentals
Definition and Principles
Anycast is a network addressing and routing methodology in which a single IP address, known as an anycast address, is assigned to multiple hosts or network interfaces located in different geographic or topological locations.[5] When a packet is sent to this anycast address, the network's routing infrastructure directs it to one of the available destinations, typically the topologically closest or most optimal one based on routing metrics such as path length or latency.[6] This can lead to IP geolocation services reporting the location of the nearest endpoint rather than the service provider's headquarters.[7] This approach enables efficient service delivery by leveraging the Internet's routing protocols to provide redundancy and proximity without requiring clients to know the specific locations of the servers.[5] The core principles of anycast rely on standard unicast routing mechanisms, where the same IP prefix is advertised from multiple sites using protocols like the Border Gateway Protocol (BGP).[6] Routing decisions are made by the network using shortest-path algorithms, such as those in Interior Gateway Protocols (IGPs) or BGP's path selection, to forward packets to the nearest endpoint based on metrics like hop count or round-trip time.[5] Unlike connection-oriented protocols, anycast provides no inherent session affinity, meaning successive packets from the same source may be routed to different endpoints, which supports stateless operations but requires applications to handle potential reordering or state synchronization if needed. In operation, a client transmits a packet destined for the anycast address, and the routing system—guided by BGP advertisements from the anycast sites—delivers it to one of the endpoints, often the one with the shortest path in the routing topology.[6] This selection is dynamic and can shift if network conditions change, such as during failover when a site withdraws its route advertisement.[5] Anycast differs from traditional load balancing, which typically operates at the application layer or uses explicit distribution algorithms; instead, anycast achieves geographic load distribution inherently through IP routing, without needing client-side or proxy-based decisions.[5] Anycast complements other IP addressing paradigms, including unicast for one-to-one communication, multicast for one-to-many, and broadcast for local network-wide delivery.Comparison to Other IP Addressing Methods
Unicast addressing facilitates one-to-one communication in IP networks, where a unique IP address identifies a single network interface, ensuring packets are delivered exclusively to that endpoint.[3] This method supports reliable point-to-point interactions, such as web browsing or file transfers, but becomes inefficient for data replication to multiple recipients, as it requires establishing separate connections for each.[1] Scalability in unicast relies on hierarchical routing to manage large address spaces, though it offers limited inherent redundancy against endpoint failures.[3] Multicast addressing supports one-to-many communication by assigning IP addresses to groups of interfaces, allowing a single packet transmission to reach all group members efficiently.[3] It relies on protocols like Internet Group Management Protocol (IGMP) for host-to-router signaling and Protocol Independent Multicast (PIM) for router-to-router tree construction, making it suitable for applications such as video streaming or software updates.[8] However, multicast demands dedicated network support for group management and can introduce scalability challenges in wide-area deployments due to the complexity of maintaining multicast trees.[1] Broadcast addressing in IPv4 enables one-to-all delivery within a local subnet, where packets with a broadcast IP (e.g., 255.255.255.255) are flooded to every device on the segment, simplifying tasks like address resolution via ARP. This approach is confined to link-local scope to prevent global propagation, but it risks network congestion and inefficiency in larger or interconnected subnets, leading to higher collision rates in shared media.[1] IPv6 eliminates native broadcast, replacing it with link-local multicast to mitigate these issues.[3] In contrast, anycast uses unicast IP addresses shared across multiple interfaces but directs packets to the "nearest" one based on routing metrics, typically via protocols like BGP, achieving one-to-nearest delivery without explicit group management.[1] This hybrid model enhances fault tolerance by automatically rerouting traffic to available replicas upon failure and reduces latency for global services, though it may disrupt stateful protocols like TCP if sessions switch endpoints mid-connection.[1] Unlike multicast's overhead for group joins or broadcast's flood risks, anycast scales through distributed replication while leveraging existing unicast infrastructure, though global deployments can strain routing tables.[1]| Addressing Method | Delivery Model | Protocol Support | Scalability Implications | Common Failure Modes |
|---|---|---|---|---|
| Unicast | One-to-one | TCP, UDP, standard IP routing | Efficient for point-to-point; requires multiple streams for replication | Single point of failure at endpoint; destination unreachability |
| Multicast | One-to-many | IGMP, PIM, MLD | Efficient bandwidth use but complex group management limits wide-area adoption | Group membership errors; multicast tree failures |
| Broadcast | One-to-all (subnet) | ARP, limited to link layer | Simple for local use; prone to congestion in large segments | Network flooding and collisions; no global scope |
| Anycast | One-to-nearest | BGP, unicast routing protocols | Improves redundancy and load balancing; routing table bloat in global use | Route flapping; stateful session disruptions |
History
Origins and Early Development
The concept of anycast was first formally proposed in November 1993 in RFC 1546, titled "Host Anycasting Service," as an extension to IP services.[2] Authored by Craig Partridge, Tony Mendez, and William Milliken, the informational RFC defined anycast as a stateless mechanism for best-effort delivery of datagrams to at least one—and ideally only one—host providing the service at a shared anycast address. It emphasized compatibility with IP's connectionless nature, allowing successive packets to the same address to be routed to different hosts without consistency guarantees. Motivations included simplifying server location for services like DNS and FTP using a single virtual address, with suggestions for a dedicated IP address class to aid identification and autoconfiguration.Initial Objections and Adoption Challenges
Early proponents of anycast faced significant objections in the 1990s, primarily centered on the potential for BGP route flapping and convergence delays caused by multiple advertisements of the same prefix from dispersed sites. The 1998 IAB Routing Workshop discussed general risks of route instability in BGP, which applied to anycast deployments and raised concerns about propagation across the Internet given the era's limited router processing capabilities for rapid route changes.[9] Another major objection involved the incompatibility of anycast with stateful protocols like TCP, where endpoint switching during a session could break connections by routing packets to different receivers. RFC 1546 explicitly addressed this by recommending that anycast be used only for initial TCP SYN packets, with subsequent responses switching to unicast addresses to maintain session continuity. Technical challenges compounded these issues, including early routers' lack of native anycast support, which often resulted in suboptimal or asymmetric routing paths that favored certain replicas over others.[2][1] Resolution efforts began with the introduction of BGP route flap dampening in RFC 2439 (November 1998), which penalized unstable routes to mitigate flapping without fully suppressing valid anycast announcements.[10] Controlled tests and pilot projects, such as early DNS anycast deployments for root servers starting in the early 2000s, demonstrated practical stability by showing minimal convergence times under normal conditions and effective load distribution. These efforts shifted consensus during IETF meetings, including the 1999 Anycast BOF at IETF 46, which discussed scalability and other concerns, with later empirical evidence from implementations addressing initial fears.[1] By the early 2000s, growing adoption in DNS infrastructure—evidenced by eight of 13 root servers using anycast by 2007—validated these mitigations and paved the way for broader protocol integration.[1][11]Technical Implementation
Anycast in IPv4
In IPv4, anycast addresses are allocated without a dedicated reserved range, distinguishing them from unicast or multicast addresses, and are instead drawn from standard unicast blocks assigned by Regional Internet Registries (RIRs), such as /24 prefixes for broader coverage or /32 host routes for individual node advertisement.[4] These addresses must remain unique within the global routing domain to avoid conflicts, and for host anycast implementations, a /32 prefix is typically used to precisely target a single service instance while allowing multiple dispersed nodes to advertise the same address.[4] Subnet anycast, in contrast, employs a shared prefix (e.g., /24) across multiple nodes within the same local network segment, enabling load distribution among servers at a single site without requiring global propagation.[1] The scarcity of IPv4 addresses heightens the efficiency of anycast by permitting multiple service instances to share one address, thereby conserving the limited 32-bit address space for broader network utility.[4] Routing for IPv4 anycast relies primarily on the Border Gateway Protocol (BGP) for global dissemination, where the same prefix is advertised from multiple Autonomous Systems (ASes) to direct traffic toward the nearest instance based on topological proximity.[1] BGP path selection prioritizes attributes such as the AS_PATH length—favoring shorter paths to minimize hops—and the Multi-Exit Discriminator (MED) to influence entry points into an AS, ensuring packets reach the optimal anycast endpoint. Within internal routing domains, Interior Gateway Protocols (IGPs) like OSPF or IS-IS may propagate host routes (/32), but global anycast typically uses covering prefixes to align with BGP's policy-driven nature.[4] However, in Classless Inter-Domain Routing (CIDR) environments, longest prefix match semantics can introduce issues; if one anycast node advertises a more specific route (e.g., /25 within a /24 anycast prefix), it may attract disproportionate traffic, overriding proximity-based selection and potentially causing imbalances or blackholing.[4] IPv4 anycast faces inherent limitations tied to the protocol's architecture, including challenges with Network Address Translation (NAT) traversal, where stateful middleboxes or firewalls may fail to handle transitions from anycast to unicast addresses in return paths, leading to session disruptions.[1] The address exhaustion in IPv4 further amplifies scaling constraints, as widespread anycast deployment increases BGP table sizes with additional route advertisements, straining router resources without the expansive space available in IPv6.[1] For intra-site load distribution, subnet anycast proves effective, as seen in deployments where multiple DNS resolvers within an ISP's local network share a common prefix, balancing queries across servers while maintaining consistent routing within the subnet.[4] A basic configuration for advertising an IPv4 anycast prefix from dispersed routers involves BGP announcements of the same route from multiple locations. For example, on two routers in different ASes sharing a /24 anycast prefix (e.g., 192.0.2.0/24), the pseudo-configuration might resemble:Router1 (AS 10000):
router bgp 10000
network 192.0.2.0 mask 255.255.255.0
neighbor 203.0.113.1 remote-as 64496 # Upstream provider
address-family ipv4
neighbor 203.0.113.1 activate
neighbor 203.0.113.1 send-community
Router2 (AS 20000):
router bgp 20000
network 192.0.2.0 mask 255.255.255.0
neighbor 198.51.100.1 remote-as 64496 # Different upstream
address-family ipv4
neighbor 198.51.100.1 activate
neighbor 198.51.100.1 send-community
Router1 (AS 10000):
router bgp 10000
network 192.0.2.0 mask 255.255.255.0
neighbor 203.0.113.1 remote-as 64496 # Upstream provider
address-family ipv4
neighbor 203.0.113.1 activate
neighbor 203.0.113.1 send-community
Router2 (AS 20000):
router bgp 20000
network 192.0.2.0 mask 255.255.255.0
neighbor 198.51.100.1 remote-as 64496 # Different upstream
address-family ipv4
neighbor 198.51.100.1 activate
neighbor 198.51.100.1 send-community