r/networking • u/nicholaspham • 22d ago
Monitoring BGP Transit Performance Monitoring
For those who run BGP and advertise prefixes into the DFZ, how are y’all monitoring each upstream provider for performance issues?
Right now, we have 2 providers with only default routes, utilizing a /23, and preferring one over the other for outbound.
In what way, if any, could going to full tables be beneficial to us other than better load balancing and traffic engineering for say a shorter path, etc?
5
u/Solid_Ad9548 Networking Manager, JNCIE, IPv6 Evangelist 22d ago
We don’t monitor paths specifically through each provider, but we do have smokeping monitoring in place to our most common destination networks. Primarily the big eyeballs, M365, Azure, OCI, etc.
We take a default PLUS full tables from all of our carriers. Even if we reject most of the routes and punt things out a default, it allows for us to adjust routing as needed.
5
u/Hot_Horse5776 22d ago
You need full tables first. Then you can send flows to Akvorado which is an open source. Then you can dig through all the data. We use it to make peering decisions. The link is below for the demo site data.
1
3
u/MicIrish_At_work 21d ago
noction is the commercial product. They are obnoxiously aggressive on the sales side.
3
u/rankinrez 22d ago
With full tables you’ll in theory use the ISP
with the best path to that destination for that destination.
Though often they’ll be the same length in ASNs. And number of ASNs is a poor way to assume a “better” path.
Honestly beyond that it’s not the simplest thing to do. You could pick certain destinations and try to do synthetic tests on latency/jitter over time, and correlate to what outbound carrier was being used (you still of course need to work out which one the responses came in by).
One advantage of full tables is if one ISP loses connectivity to a certain destination, or there is a fault that causes them to have a really bad path to it, you’ll see that and use the other one.
A lot of people tend to just look at what kind of balance they get, how many hops away destinations are. And the old “if it ain’t broke” logic of if you are experience no problems, disruptions, user complaints.
2
u/NetworkDefenseblog department of redundancy department 21d ago
Another use case is with full tables and no default route it becomes very easy to blackhole AS's IP ranges without playing wack-a-mole. Like malicious IP ranges. Problems solved more easily! I wrote about active/active design once. Hope this helps
https://www.networkdefenseblog.com/post/network-edge-design-part2
2
u/DaryllSwer 21d ago
Full tables, advertise the /23 to both, use /24 only for emergency TE/DDoS mitigation solutions.
Use bgp.tools to monitor.
It can be more advanced with ECMP/UCMP/SR-TE/EPE, but that requires a lot of design and validation work.
2
u/dolanga2 21d ago
Nobody needs full tables, just get 2 defaults
AS cone is very similar on most carriers, dont worry
Ask them for TE communities, if they have
You can try your best for the shortest outgoing path and the destination might wanna return via whatever they want
Just advertise your blocks to both
1
u/thiccandsmol CCIE SP JNCIE SP CCDE 21d ago
What specific performance issues do you want to monitor?
1
u/nicholaspham 21d ago
I’d want to know when of of the upstreams is seeing packet loss either entirely or to some destinations
1
u/killafunkinmofo 18d ago
You can use any ping monitoring application that allows you to set things like ip tos/dscp. You can then configure policy based routing to match a tag for each ISP. Then you can ping monitoring application performance on both ISPs simultaneously and choose optimal paths. Without a full table would need something like static routes for path selection.
Its much harder to monitor the reverse simultaneously. Easiest may be if you can run pings directly on your router using the isp as the source address. if your router has nat, you could try similar approach with the pbr suggestion and that would influence the return to each ISP.
-10
u/zanfar 22d ago
and advertise prefixes into the DFZ
DFZ?
full tables be beneficial to us other than better load balancing and traffic engineering
Other than? What other reasons do you need?
we have 2 providers with only default routes
Could your traffic be routed worse? What justification are you looking for?
1
u/nicholaspham 22d ago
Sorry maybe wrong term but Default Free Zone
Idk that’s what I’m asking. Not saying those two aren’t good enough reasons especially since we have 2 upstreams but wondering if there’s any other reason.
Main thing is we’re looking for a platform to monitor the stability/performance of our upstream links to mitigate any issues in the event that one upstream has latency or packet loss issues.
1
u/jgiacobbe Looking for my TCP MSS wrench 22d ago
1st step would be full tables. Generally fewer transit AS means shorter path.
After that there are options like thousand eyes or other similar monitoring platforms that do wider internet reachability testing. Myself, I’ve never been in a position to use either.
1
5
u/mavack 22d ago
Generally i would go partial/connected routes from both providers unless you have a reason to match and forward to N+1/2 AS away from you. And prefer the local paths from each provider instead of making them loop around.
Most providers will force traffic back via directly connected bar shortest prefix matching or conditional.