Cisco OTV – Overlay Transport Virtualization – Part 2

In part 1 we configured OTV using multicast as the control plane transport method. But if you don’t want to use multicast, you can use unicast instead. The main difference is that you’ll also need to designate at least one OTV Edge device to act as an Adjacency Server.

Adjacency Server

With OTV in multicast mode, the underlying multicast infrastructure handled packet replication to each of the remote OTV devices, and also allowed the dynamic discovery of OTV peers. With unicast mode, each edge device must send a unicast copy of the packet to each remote device. The question is – how do they know which devices should receive each packet?

The adjacency server function allows a router to learn of, and distribute lists of edge devices, so all members of the OTV domain are aware of each other. The adjacency server creates a Unicast Replication List (URL) and distributes this list to each edge device, updating as necessary when devices join or leave.

Once you have your adjacency servers configured, adding a new site is as easy as configuring the new device with the address of the two Adjacency servers, who will then distribute information about the new device to the rest of the overlay.

Configuration

Since OTV can use only one mode at at time (unicast OR multicast) you must completely remove any multicast specific OTV commands before you can add the adjacency server config.

So we’ll start by removing our multicast control-group information from both routers:

interface Overlay1
no otv control-group 239.1.1.1
no otv data-group 232.1.1.0/28

Since our topology only has two routers, we’ll only configure OTV-RTR1 to act as the adjacency server, and then point it to itself:

interface Overlay1
otv adjacency-server unicast-only
otv use-adjacency-server 10.80.0.2 unicast-only

On OTV-RTR2, we’ll only specify the adjacency server address:

interface Overlay1
otv use-adjacency-server 10.80.0.2 unicast-only

Now let’s look at the status of the overlay:

OTV-RTR1#sh otv overlay1
Overlay Interface Overlay1
 VPN name                 : None
 VPN ID                   : 1
 State                    : UP
 AED Capable              : Yes
 Join interface(s)        : GigabitEthernet0/0/1
 Join IPv4 address        : 10.80.0.2
 Tunnel interface(s)      : Tunnel1
 Encapsulation format     : GRE/IPv4
 Site Bridge-Domain       : 100
 Capability               : Unicast-only
 Is Adjacency Server      : Yes
 Adj Server Configured    : Yes
 Prim/Sec Adj Svr(s)      : 10.80.0.2

We can see that this router is an adjacency server, and that it has an adjacency server configured. On OTV-RTR2:

OTV-RTR2#sh otv overlay1
Overlay Interface Overlay1
 VPN name                 : None
 VPN ID                   : 1
 State                    : UP
 AED Capable              : Yes
 Join interface(s)        : GigabitEthernet0/0/1
 Join IPv4 address        : 10.70.0.2
 Tunnel interface(s)      : Tunnel1
 Encapsulation format     : GRE/IPv4
 Site Bridge-Domain       : 100
 Capability               : Unicast-only
 Is Adjacency Server      : No
 Adj Server Configured    : Yes
 Prim/Sec Adj Svr(s)      : 10.80.0.2

We can also see that the OTV adjacency is up:

OTV-RTR1#sh otv adj
Overlay 1 Adjacency Database
Hostname                       System-ID      Dest Addr       Up Time   State
OTV-RTR2                       c08c.6008.0f00 10.70.0.2       00:47:50  UP

We can also see that the URL on each router contains the opposite edge device:

OTV-RTR1#sh otv adjacency-server replication-list
Overlay 1 Unicast Replication List Database
Total num: 1

Dest Addr       Capability
10.70.0.2       Unicast

OTV-RTR2#sh otv adjacency-server replication-list
Overlay 1 Unicast Replication List Database
Total num: 1

Dest Addr       Capability
10.80.0.2       Unicast

AED

I mentioned a term in the previous post that I would like to revisit. The AED, or Authoritative Edge Device. To get some background, let’s reexamine our topology:

Sample OTV topology

In the first topology there is only one entry and exit point for overlay traffic. But in a production environment, you will likely have redundant edge devices. What happens then, when we add another OTV edge device to a site?

otv-topology-2

We’ll go ahead and add another router on the left side of the diagram, and call it OTV-RTR1A.  Now we have two interfaces on the same Vlan that don’t participate in STP, which means it’s possible for a loop to form. To prevent this from happening, edge devices at the same site will elect an AED, which is then the only device allowed to forward traffic for the overlay. This functionality applies to traffic going both directions — the AED will be the only device allowed to both encapsulate frames into IP packets towards the overlay, and decapsulate OTV packets and forward frames to the local LAN.

It’s also important to note that an AED will be elected per VLAN for automatic load-balancing. One edge device will be the AED for the even number VLANs, and the other for the odd VLANs. As far as I know, this is not configurable.

AED Election

Remember that site ID and site VLAN we configured in part 1? The Site ID is used to identify Edge devices in the same site, and the Site VLAN is used for communication between the edge devices. Another value, the OTV System ID, is used to elect the AED.

The System ID is a combination of the IS-IS system ID (viewed with the show otv isis protocol command) and the site identifier. You can view the OTV system ID with the show otv site command.

Using our new topology, let’s go back and add the site vlan to the inside interface on OTV-RTR1 and OTV-RTR1A (RTR1A has already been configred for OTV, using RTR1 as the adjacency server):

OTV-RTR1(config)#interface Gig0/0/0
OTV-RTR1(config-if)#service instance 100 ethernet
OTV-RTR1(config-if-srv)#encapsulation dot1q 100
OTV-RTR1(config-if-srv)#bridge-domain 100
OTV-RTR1(config-if-srv)#^Z
OTV-RTR1#

OTV-RTR1A(config)#interface Gig0/0/0
OTV-RTR1A(config-if)#service instance 100 ethernet
OTV-RTR1A(config-if-srv)#encapsulation dot1q 100
OTV-RTR1A(config-if-srv)#bridge-domain 100
OTV-RTR1A(config-if-srv)#^Z
OTV-RTR1A#

Now let’s verify that the two edge devices recognize each other, and that one has been elected AED.

OTV-RTR1#sh otv site
Site Adjacency Information (Site Bridge-Domain: 100)

Overlay1 Site-Local Adjacencies (Count: 1)

Hostname       System ID      Last Change Ordinal    AED Enabled Status
*OTV-RTR1      001E.4962.5400 00:02:20    0          site       overlay
 OTV-RTR1A     001E.F6B5.2600 00:02:20    1          site       overlay

OTV-RTR1A#sh otv site
Site Adjacency Information (Site Bridge-Domain: 100)

Overlay1 Site-Local Adjacencies (Count: 1)

Hostname       System ID      Last Change Ordinal    AED Enabled Status
OTV-RTR1       001E.4962.5400 00:02:39    0          site       overlay
*OTV-RTR1A     001E.F6B5.2600 00:02:39    1          site       overlay

The two devices have formed a ‘site-local adjacency’. Who’s the AED?

OTV-RTR1#sh otv vlan authoritative
Key:  SI - Service Instance

Overlay 1 VLAN Configuration Information
 Inst VLAN  Bridge-Domain  Auth  Site Interface(s)
 0    250   250            yes   Gi0/0/0:SI250
 Total VLAN(s): 1
 Total Authoritative VLAN(s): 1

OTV-RTR1A#sh otv vlan authoritative
Key:  SI - Service Instance

Overlay 1 VLAN Configuration Information
 Inst VLAN  Bridge-Domain  Auth  Site Interface(s)
 Total VLAN(s): 1
 Total Authoritative VLAN(s): 0

We can see that OTV-RTR1 is the AED at the site.

Failover

To test a failover scenario, we’ll simply shutdown the inside interface on OTV-RTR1. Once the interface is down, you’ll notice that OTV-RTR1A now considers itself authoritative for VLAN 250:

OTV-RTR1A#sh otv vlan authoritative
Key:  SI - Service Instance

Overlay 1 VLAN Configuration Information
 Inst VLAN  Bridge-Domain  Auth  Site Interface(s)
 0    250   250            yes   Gi0/0/0:SI250
 Total VLAN(s): 1
 Total Authoritative VLAN(s): 1

And when we examine the OTV event log, we see the following entry:

[09/16/13 19:44:24.724 23C9 490] OTV-APP-ISIS: AED set to UP for overlay 1 bd 250

Keep in mind that OTV is using IS-IS under the hood, so the failover processes are dependent on IS-IS timers.

Also, it’s important to note that the failover process does interrupt traffic. There is a short window of about 8-9 seconds where the remote OTV device doesn’t have an OTV route from the other site. It’s hard to grab the output exactly on time, though, so you’ll have to test in your environment to verify. In the outputs below, notice how the route for Sw-1 (0009.b709.4b80) is withdrawn and then is inserted again with OTV-RTR1A as the next hop:

OTV-RTR2#sh clock
14:05:05.745 MST Mon Sep 16 2013
OTV-RTR2#sh otv route

Codes: BD - Bridge-Domain, AD - Admin-Distance,
   SI - Service Instance, * - Backup Route

OTV Unicast MAC Routing Table for Overlay1

Inst VLAN BD     MAC Address    AD    Owner  Next Hops(s)
----------------------------------------------------------
0    250 250   0009.b709.4b80 50    ISIS   OTV-RTR1
0    250 250   0009.b717.7880 40    BD Eng Gi0/0/1:SI250

OTV-RTR2#show clock
14:05:16.767 MST Mon Sep 16 2013
OTV-RTR2#sh otv route

Codes: BD - Bridge-Domain, AD - Admin-Distance,
    SI - Service Instance, * - Backup Route

OTV Unicast MAC Routing Table for Overlay1

Inst VLAN BD     MAC Address    AD    Owner  Next Hops(s)
----------------------------------------------------------
0    250 250   0009.b717.7880 40    BD Eng Gi0/0/1:SI250

OTV-RTR2#show clock
14:05:25.800 MST Mon Sep 16 2013
OTV-RTR2#sh otv route

Codes: BD - Bridge-Domain, AD - Admin-Distance,
    SI - Service Instance, * - Backup Route

OTV Unicast MAC Routing Table for Overlay1

Inst VLAN BD     MAC Address    AD    Owner  Next Hops(s)
----------------------------------------------------------
0    250 250   0009.b709.4b80 50    ISIS   OTV-RTR1A
0    250 250   0009.b717.7880 40    BD Eng Gi0/0/1:SI250

Conclusion

As you can see, OTV is still fairly simple to configure, but there’s a lot of stuff going on behind the scenes to make it work. Before you implement anything in production, be sure to have a good understanding of what’s actually happening.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s