Skip to Content

Campus Network Outage 9/10 6:00-6:30am; First Nexus 7010 in Production

Printer-friendly versionPrinter-friendly version

On Friday, Sept. 10th, there will be a campus network outage between 6:00-6:30am. During this outage, all campus core routers will be modified to run Rapid-PVST spanning tree, rather than the older PVST. As the names imply, Rapid-PVST is like PVST, but faster. While PVST takes 30-40 seconds to reconfigure the network after a topology change -- dropping traffic during the process -- Rapid-PVST takes less than 5 seconds. Changing to Rapid-PVST is required by the new campus core routers, and should be helpful when we start upgrading intra-core links to 10Gb. During the same outage window, some departments will have their routing migrated to the N7K; this has been announced to the CSF list. At some point in the future, another brief outage will be required to migrate various building switch connection to the new N7K.

The first Nexus 7010 (N7K) is moving production traffic. Site preparation is essentially complete at one core location, and the N7K has a vlan-tagged link to the old 6506 router. Portions of the old configuration were converted by a locally-developed script and imported to the N7K, so the N7K has all the necessary VLAN, routed VLAN interface, and ACL information. Those subnets supporting NOC services are now routed by the N7K.

The new router configurations use CoPP (Control-Plane Policing) to help protect the router from various attacks and other evil, and CoPP requires careful cofiguration. Although it did not impact any users, a couple of issues with CoPP became obvious after the migration of NOC subnets. The first problem was that DHCP did not work. A client would request an address, the router would forward the request to the DHCP server, the server would respond, but the router would drop the response and the client would still need an address. This is typical of a CoPP problem, and was easily resolved by adding a clause to the CoPP configuration to accept DHCP replies from the DHCP servers. The other problem involved a multicast source which was not recognized by the router. Again, CoPP was preventing the control plane from seeing most multicast traffic, so it did not learn about the multicast source. A small modification to CoPP and the multicast source was recognized and accessible to other on-campus clients. I'm certain there will be additional tweaks to CoPP in the coming months, but I hope the majority of the configuration is adequate to our needs.

This is an exciting -- and slightly scary -- process, and I really appreciate everyone's patience during these events. I also welcome any questions or comments regarding the migration process, the new equipment, or campus networking in general.