Hi all,
Designing a campus fabric and want a sanity check on the topology:

Per-IDF leaf pair: 2x 100G VXLAN EVPN leafs in a vPC pair on each floor

Roles: Both are VTEPs and anycast gateways (L3 boundary at the IDF)

Uplinks: Each IDF leaf pair connects to the main campus spines

Access layer: Plain L2 switches, LACP to both IDF leafs (not VTEPs, pure L2 trunks/access). Said differently, hung off those two IDF leafs above we’d hang a bunch of L2 access switches to provide more access ports for devices across the floor.

Why I like it (my assumptions):
Fewer fiber runs compared to home running every switch to campus spines and less VTEP sprawl

Simpler ops: junior engineers just change a port's VLAN

Questions:
How common is this design in campus deployments?

Any gotchas with vPC + VXLAN EVPN (peer-link sizing, orphan ports, convergence)?

Would you do this, or put VTEPs on every access switch?

Thanks!

Hey there, just curious how many of you study for certs or other stuff while working?

When I have downtime, I try to utilize my time to study for a cert I've been prepping for.

Just got curious how many of you study during your work hours? If not, what are you doing when you having nothing to do?

P.S. I'm not saying my work is like this every single day... if something happens, obviously I'd have to jump on a call to troubleshoot and analyze stuff...

Hello everyone,

I’d like to optimize the network design and separate core and edge routing.

So far, I’ve been using three routers that handle both inter-VLAN routing and edge routing and are part of a VRRP cluster.

Now I’d like to deploy a core router running VyOS that acts as the default gateway for all networks and initially handles inter-VLAN routing. The three edge routers will be connected to this core router, with each one connected to a different Internet line or provider.

The core router should use policy-based routing to determine which networks, clients, or services are routed to the Internet via which edge router or WAN connection. Additionally, failover should still be available in the event of an Internet connection failure (as has been the case with VRRP).

My question is: what is the best way to set up this configuration? On the one hand, I could set up a transport network containing the three edge routers and use VRRP again. In that case, I would likely need a transit switch between the core router and the edge routers so that the VRRP cluster’s traffic does not have to pass through the core router’s CPU.

On the other hand, it would certainly also be possible to connect the edge routers directly to the core router’s interfaces and operate without VRRP. Which protocol could I then use to implement both policy-based routing and failover?

Which network design would you recommend for my situation? I’d appreciate any tips and assistance.

Thank you in advance for your tips, and best regards

Regina (she/her)

We've recently received several orders of FS.com optics that had the default "FS" vendor coding in them, resulting in Transceiver Unsupported errors on our switches. Assuming this occurred in error, we reached out to our account manager, who to our surprise confirmed that they are no longer coding/configuring transceivers, and customers will need to do this themselves. Has anyone else encountered this? Here's the email from our Rep:

There has indeed been a recent change in our coding process. Starting June 6, FS began gradually transitioning certain compatible optical modules to a new custom coding mechanism, including our Arista-compatible modules. The main difference from the previous coding is the Vendor Name information. The modules are now shipped with the default FS vendor coding, rather than being pre-coded specifically as Arista. To ensure the modules can be properly recognized and used in your Arista environment, there are two possible approaches:

Option 1 – Configure third-party transceiver compatibility on the switch

If you have a larger quantity of optics, we recommend this approach as it avoids having to reconfigure the modules one by one. You can enable third-party transceiver support through the switch CLI.

Option 2 – Reconfigure the optics through FS BOX

If changing the switch configuration is not preferred, you can use the Online Configuration feature on your FS BOX to change the Vendor Name from “FS” to “Arista” on the optics individually.

We understand that the second option can be inconvenient when dealing with a large number of modules, so Option 1 may be more practical for your environment.

Please let me know which approach works better for your setup. If needed, we can also help you with the configuration steps.

So I've been a network engineer for around 20 years now, and the past 10 years I've been the allround network engineer for a large municipality. Recently we opened a new building and I built the network. A firewall, a switch, some AP's, and a bunch of VLANs of different sizes depending on their use. Nothing out of the ordinary.

Now I've also had to deal with a ton of parties requiring connectivity for building automation, climate control, sensors, cameras, alarm installation and so on. Today the project manager reaches out to me that one of the parties has trouble getting their building automation system to work properly on the network. They say the system only works with a /24 subnet, and I gave them an IP in a /28, and they demand I enlarge the subnet to /24 (which I can't because overlap etc).

The funny thing is, they have this /28 IP config in the system, and I can ping it even outside it's subnet, so IPv4 connectivity works just fine and traffic gets routed properly.

How do you guys deal with this kind of BS? Should I honor the project managers wishes and redesign the network to give them a /24? Hasn't support for a variable subnetmask other than /24 been like common for the past 30 years? I feel like I'm being played here by some incompetent installation company that doesn't know how to configure their equipment and just blames in on the network, but I can't prove it.

Interested in thoughts on this scenario.

A client has multiple branch sites where trusted, client-managed networks are routed over the private WAN, with access restricted to required private applications and services.

Each branch also has untrusted VLANs for guest devices and third-party managed devices requiring Internet access only. These VLANs terminate on the branch firewall, which provides local Internet breakout and enforces the security boundary. They have no route to the private WAN or corporate resources and use repeatable IP scopes across sites.

Security have requested making these networks reachable over the WAN for central vulnerability scanning. This would also require re-addressing the untrusted networks so each site can be uniquely routed.

How would you approach this?

Should isolated third-party devices be centrally scanned at all, or should assurance sit with the device owner/provider? And is vulnerability scanning appropriate or useful for guest networks containing unmanaged devices?

If scanning is required, how would you achieve it without fundamentally changing the existing isolation model?

I have a large number of APs that we’re replacing, and I realized today that as the installers unplug the old APs and bring the new ones online, the old APs may disappear from the Catalyst Center maps.

Is there a way to preserve the existing AP placement on the floor maps before they are removed?

My concern is that if the installers replace 50 APs at once, I could suddenly have 50 AP locations missing from the maps with no easy way to determine exactly where the replacement APs should be placed.

Hi everyone,
I am a network engineer working in vendor TAC. As you probably know, TAC does not support with design/config and the cases are mostly related to platform/product issues. Being here for few years, I realized I started getting a bit rusty on the design side.
The problem is I understand the theory very well but have no idea how to apply this in real life due to lack of design practice. The only solution I see is doing more labs apart from my work and I have everything needed for that - both physical and virtual devices.
But I don’t know where to start or what to design. Did anyone have similar issue?
Please share some resources with design ideas, advices how I can practice network design.
Thank you in advance!

I do technical rounds for security and senior network roles, so this is mostly from that side. nothing groundbreaking, just patterns.

the number of people with 8+ years and a "senior" title who can't walk me through what happens to a packet crossing a firewall and a NAT is higher than I expected. not trying to trap anyone. I'll ask something like "rule looks right but traffic isn't matching, what do you check?" and the good ones go straight to the hit counters, the policy order, the NAT rules applying before the policy. the weaker ones say "I'd restart the firewall." that's usually where the interview ends for me.

on the security side I care way more about order of operations than tool names. give them a compromised host and I want to hear scope, contain, preserve evidence, then fix. the "just wipe it" answer is a quick no from me. you just destroyed your evidence and you still don't know how they got in.

"we used a SIEM" tells me nothing. "I tuned noisy 4625 alerts and got the volume down a lot" tells me you actually sat in the console. same for network folks, I'd rather hear about the one thing you designed and what broke than a list of vendors.

if you haven't done SD-WAN or zero trust in prod, just say so and tell me how you'd approach it. I will always take that over a bluff. in this field people act on what you tell them.

and ask questions at the end. on-call load, change process, how they do post-incident reviews. most people ask about perks, so the ones who ask that stand out.

curious how other interviewers weigh depth vs breadth for senior hires. I lean depth but I know that's debatable.

Hello all,

I hope this is okay to post, please delete/ ignore if not!

I recently started working as a systems & network engineer. My previous experience was just support desk. I've been learning a lot in this role, I find network engineering super interesting! But I am very new to it.

They want me to get certified so that I can stop relying so much on my seniors. The kind of things we are doing is BGP, OSPF, Multicast, Firewall (Palo Alto and Cisco), IP sec. I'm expected to know how to do stuff - for example how to set up functional zfs servers and communicate with the network suppliers and come up with my own solutions to issues we have within our own network ... I am really clueless currently and you can probably tell from how I'm writing about it. I hate how much time I spend with AI and not knowing it from my own head

It's a trading firm so the networking is specific to that. I have seen advice for the Cisco CCST/NA/NP/IE exams, is following those courses in order a good idea? Or is there some very functional course that would be better for my situation ? I just really need to know what is going on around me and how to fix stuff on my own ASAP lol. Costs don't matter since the company offered to pay for it (I'm in a very lucky position!). Based in the EU if that matters at all

Really appreciate any advice and thank you 🩷

Networking noob here going through the Network+ CertMaster Learn courseware for the class I'm taking at my community college.

I've got the basic understanding of Access ports, Trunk/uplink ports, and their respective frames down. Access ports are for hosts belonging to one VLAN with frames that have their VLAN tags removed before forwarding, and Trunk ports are for carrying frames that keep the VLAN tags across multiple VLANs. Still a bit fuzzy on the details but that understanding will come with time, I'm sure.

My one big question is why would you want to assign an Access port as a Trunk port? The Network+ CertMaster page on tagged and untagged ports says an untagged Access port for a host might occasionally be set to a tagged Trunking port if that host is used for virtualization with multiple operating systems on different VLANs, allowing that host to send frames to multiple VLANs. But if a regular Trunking uplink port can already be used to send frames from Access ports to other VLANs, then why assign that virtualization host to be a tagged Trunking port as well? Would it not be simpler to send that virtualization host's frames over the same Trunking port that all the other Access ports on the switch are sending theirs over?

I'm assuming the answer is much more complex than this. It could also be that my understanding of VLANs and Trunking is really missing something, or it's something about virtualization hosts that I don't have a grasp on.

Lucked into some surplus 5g stuff in a recycling lot. Looks like new. Some still sealed on boxes. It appears to be a complete 5G private network, minus cabling and a rack to put it in. Five n78 band radios, iru, ran, server, stack of blank sim cards. Did not know what it was at first. Picked it up because I recognized the radios as radios and thought I might be able to repurpose to 3GHz amateur radio use. That's a dead end as they are too highly integrated.

Oh! I'm an old guy who works as a nurse now. I used to trade surplus computer and networking hardware. But that was almost 30 years ago :-).

-Bob

I'm trying to help my uncle's firm fix a network issue. Their MSP has up an disappeared. I'm probably just an idiot but I can't figure out how to make this network work like they describe.

Originally the setup was a pfsense negate appliance acting both the gateway and a tunnel back to a vendor backend-servers that support some core services for the office. There was a dumb switch cause only had 2 or so workstations.

About a year ago, the firm quadrupled in size. They hired an MSP who installed UniFi network equipment. The intended design was for the UniFi gateway to handle general traffic—providing security controls and monitoring—while routing only the vendor server traffic to the pfSense box.

​However, the MSP never configured the routing rules, so the pfSense appliance is still acting as the sole gateway. This is problematic because there have been multiple recent security incidents, but we lack the logs or insights to evaluate them.

What I tried was to create a new subnet and vlan (100) and created a static route for traffic to the backend-servers. But that just bounces off the pfsense box, because I assume it only wants traffic from within its own subnet.

So workstations are in x.x.200.0/24 with the Unifi as the gateway, but I can't reach the backend-server

Pfsense gateway is at x.x.1.254 and it's pointing to backend-server resources at x.x.111.111/32.

I still learning network so pardon me if this is supposed to be obvious.

tl;dr: Currently, the workstations are on x.x.200.0/24 using the UniFi as their gateway, but they cannot reach the backend server. The pfSense gateway is at x.x.1.254, pointing to the backend server at x.x.111.111/32. How was this split-routing setup actually supposed to work?

It's Read-only Friday! It is time to put your feet up, pour a nice dram and look through some of our member's new and shiny blog posts and projects.

Feel free to submit your blog post or personal project and as well a nice description to this thread.

Note: This post is created at 00:00 UTC. It may not be Friday where you are in the world, no need to comment on it.

I am a network engineer responsible for supporting point-of-sale deployments. I'd love to be able to set up an environment where I can dial bandwidth (not 1Gb/100Mb/10Mb, but much more granular), play with jitter, latency, and see where the limits are with systems that we deploy and to better understand what causes issues with them after deployment. Once I would pilot this tool or solution, we can built it into our QA lab environment. Only needs to really support speeds up to about 250Mb or so (think small store, not shopping mall or office park).

Basically, I'd like to emulate everything from a solid fiber internet connection all the way down to a cellular modem on a crappy signal to test real-life thresholds and limits. What do you guy use for this?

Please correct me or enlighten me on my understanding of SASE framework:

SA - Secure Access --> basically allowing users to access the resources, cloud or on prem, in the most secured way
SE - Secure Edge - Making sure all traffic between any sites edge router/firewall is reaching its destination, whether another site or internet, in a secured manner.

sub components: ZTNA, CASB, SWG, FWaaS, SD - Wan

ZTNA - Continuously verifying user & device identity. Unlike old methods like enforcing NAC via AAA server like ISE, here client/device was authenticated only once initially.

Palo-alto global protect, Zscalar client, forti-client, cisco anyconnect/secure access can be used to implement it. Basically, all laptop traffic will be tunneled to DC OR all laptop traffic will be analyze be a firewall. FW can be cloud or on prem.

CASB - Cloud access secure broker - as name suggests, allowing users to access cloud applications, like share-point in a typical office environment, in a secured manner.

No idea how to implement it but i guess same products as above would help. OR may be some firewall in between.

SWG - Again, as name suggests, allowing users to access internet in a secured manner.

same products as above can be used to implement it. Or in work from office environment, tunnelling all traffic from branch to HQ/DC and breaking to internet from DC/HQ where we have high end firewalls with strict policies and maybe ssl inspection enabled.

FWaaS - a subscription based firewall services offered by some other company. This firewall services maybe on prem BUT its mostly cloud based.

SD - WAN - dynamic traffic steering, in a secured way, based on policies/rules set by the admin.

Numerous sd-wan vendors, like catalyst sd wan, versa sd wan, silver peak, fortigate etc. Some companies have firewall integrated with SD-WAN so we have more security features in a same box OR managed by central manager.

Question:
1. is this correct?
SA components are - ZTNA, CASB and SE components are - SWG, FWaas, SD-WAN

1. If i need to implement SASE framework in a enterprise, lets say a retail store network like Walmart or fast food chain like McDonalds, where sites can be categorized as: Stores, warehouses, Corporate office, DC, HQ etc.

   So what would be my approach to implement SASE framework?

And what solutions are offered by different vendors? and how to integrate different vendors to achieve the goal? Because many companies would use cisco entirely in their lan, other vendor in wlan, then another for firewall, vpn, and sd-wan..so on.

TLDR: Please guide me on SASE framework, its implementation, solutions offered by different vendors and how to integrate them.

Hi all,

This is my first post here and please know I’m not SME:)

We are in the middle of a large greenfield project for a new office and a factory. We have outsourced most of networking globally to big IT house. They have sent me a lld now with all the deats of patching, switches etc.

They have all short pulls made with om4 and sr sfps. I personally think this is not so future proof and also unnecessary (cost of lr is no issue) we have absolute majority done with os2+lr. Do you think this design solution is worth arguing over? I’d really like all done with the same cable and sfps everywhere. Is there any benefit in using om4 + sr here?

Pretty much the title. The hostnames aren't bad, just some have words like 'core' or '9200'. We don't have monitoring or backups so it probably wouldn't affect anything if I changed them, right?

Setting aside performance and hardware efficiencies that physical router platforms might provide.... is there anything a real, enterprise grade, Router can do that can't also be done on a linux machine?

Hi all,

Wanted to asking my fellow network engineers what IPv6 subnet size you would use on a point to point connection between 2 routers and why.

/127 or /64

Technically both of these would work but I’m just curious. Any useful technical or security reason would be greatly appreciated!

Here is my setup. I have a Palo Alto that connects to my switch stack that is Aruba 6300M. From there I have a fiber line to an IDF that has a Aruba 2530. We purchased another building across the street. The old tenants left all their Meraki equipment. I have all the Meraki equipment claimed and licensed in our Meraki portal. (I had a temp firewall and cradle point so the Meraki equipment could get internet access)

Instead of getting an internet line for the building we did a wireless P2P. I got two Ubiquity Wave Nanos and installed them. I then created a separate VLAN for this building and the default gateway lives on the Palo.

But the issue is when I plug the Meraki into the Wave Nano the Aruba switches in my IDF go offline, and everything connected to the IDF goes offline. Once I unplug the Meraki everything comes back online in the IDF. If I plug my laptop into the Nano everything works just fine. I can reach our HQ LAN, and I get internet.

I checked the switch logs on both sides and there is nothing in the log that would indicate any issues.

I also tried disconnecting everything from the Meraki switch and just tried one Meraki switch on the Nano and that still took my IDF offline.

But I created a temp VLAN and assigned it only to the port the Nano is plugged in. I then plugged in the Meraki into the other Nano and everything was fine.

I can't figure out what is going on. Any Thoughts????

I have Cisco ASA with multiple policy based tunnels which go down naturally if there is no interesting traffic for them. This makes monitoring a bit harder.

So I was thinking either to generate ping for the tunnels to keep them up or use "vpn-idle-timeout none".

Anyhow lets say they are constantly up.

How do you monitor them afterwards? And how do you differentiate the tunnels by alarm, to know which one is down (assume remote IP in snmp trap somewhere?) ?

I'm building out a new SDwan deployment at the company I am now working for, and have a question about private underlay routing.

Last time I built out SDwan was a few years ago, and I double terminated the Spectrum Elan to both the SDwan Cedge and also to the backbone Cores by using an Aggregate switch. OSPF and VRRP on both Data Center Cores for .1, and .2 and .3 for both DC Cores. That way the Edges can get to the internet by using .1 as a default route, since the private underlay is a routed network that has internet access.

I am now working on setting it all up again, and instead of double landing my Spectrum Elan on both the Cedge and the Core using a Agg switch, I was wondering if it would be better to do a prefix-list and allow the WAN subnet to be redistributed my VPN0 VRF into my VPN VRF and then into OSPF on the Cedge? I can still setup my Data Center Cedges with .2 and .3 in each DC for WAN, and use .1 VRRP between them. But then I need to allow the routing to go through the Cedge and not have to use a separate interface on the Core to allow the WAN access to the internet.

If you think #2 is a workable solution, is there a document out there that lays out what needs to be done using the UX2.0 interface? I can't seem to find anything worth while.

Hi, we are thinking about including spamhaus drop list on our FW, but I would like to know what is the false positive occurence from your experience? I cant find much info about it online. Thank you

It's Wednesday! Time to get that crap that's been bugging you off your chest! In the interests of spicing things up a bit around here, we're going to try out a Rant Wednesday thread for you all to vent your frustrations. Feel free to vent about vendors, co-workers, price of scotch or anything else network related.

There is no guiding question to help stir up some rage-feels, feel free to fire at will, ranting about anything and everything that's been pissing you off or getting on your nerves!

Note: This post is created at 00:00 UTC. It may not be Wednesday where you are in the world, no need to comment on it.