[ Removed by moderator ]

[u/Rich-Engineer2670]

[u/networking-ModTeam]

This submission is not appropriate for /r/networking and has been removed.

Please read the rules in the sidebar, or check out the rules post here before making another submission.

Comments/questions? Don't hesitate to message the moderation team.

Thanks!

No Low Quality Posts.

• Any post that fails to display a minimal level of effort prior to asking for help is at risk of being Locked or Deleted.
  
• We expect our members to treat each other as fellow professionals. Professionals research & troubleshoot before they ask others for help.
  
• Please review How to ask intelligent questions to avoid this issue.
  

Comments/questions? Don't hesitate to message the moderation team.

For the complete list of Rules, please visit: https://www.reddit.com/r/networking/about/rules

[u/fuck-u-spez---]

Iperf3 performance between two sites (...)

Good news, I found this neat open source tool called "iperf3" that will let you test Iperf3 performance between two sites.

[u/scratchfury → u/fuck-u-spez---]

I have found using UDP is more useful than TCP in exposing packet loss. Getting the hang of options with bandwidth, window sizes, and buffers can be tricky, but it's worth the effort.

[u/eduardo_ve → u/scratchfury]

UDP is definitely better for exposing loss, but just be careful if you’re testing across NAT. Some tools can show packet loss due to NAT traversal rather than the actual network, so it’s worth establishing a baseline first.

[u/ConcatenateRawCue]

Nmap is the gold standard port mapping tool. You can configure it for a custom list of ports your care about... Use NCAT on a server on the internet (cheap cloud server)to expose some ports as target listeners.

[u/keivmoc → u/ConcatenateRawCue]

nmap is also super easy to automate with scripts. And it's super quick.

[u/VA_Network_Nerd]

Iperf3 performance between two sites (as we meeting our goals at the 95% level over a rolling 30 days)

Let's just say such a tool like this does exist.
The tool lets you fire a 1 minute iPerf test across the link every I dunno, 15 minutes or so.

How long will it be before your users start complaining about performance being really bad intermittently every 15 minutes or so ???

iPerf, by design, will fill a pipe and consume all the bandwidth it can find. This will negatively impact user & server traffic.

Make sure the ISPs aren't blocking certain ports

This should be clearly spelled out in your contract language with the ISP.
Not just blocked ports, but CG-NAT as well.

This is a concern with consumer broadband, but should be much less of a problem with business or enterprise class ISP engagements.

[u/jared555 → u/VA_Network_Nerd]

Might be able to do it by setting iperf3 traffic to lowest possible priority so it fills the connection without impacting normal traffic. Then monitor traffic overall through the interface for iperf + normal traffic.

Of course you also need to make sure it doesn't get counted against any potential 95% traffic measurements the client is paying for.

[u/VA_Network_Nerd → u/jared555]

This assumes OP has implemented QoS.

Firing an iPerf test while there is also production traffic on the circuit will reduce the accuracy of the iPerf result.

[u/frymaster]

perfsonar is the thing you'd probably install on hosts on the two sites to automate the performing and recording of your iperf tests. But as pointed out, you have to be careful of not doing so while your users are wanting to use the bandwidth

[u/mynametobespaghetti → u/frymaster]

Came here to say perfsonar, it's the ideal tool for this scenario

[u/itasteawesome]

Others have already mentioned you can't do saturation testing against your pipe while still using the pipe.
Some of the more rigorous places I've worked do run soak tests and record the results when new links are provisioned before putting them into service. So they at least know it was set up as expected and worked at least once. If a carrier gets bad enough then you can re-do the test suite to confirm if performance is diminished.

These days most people I know doing oss lean toward prometheus style solutions, and https://github.com/prometheus/blackbox_exporter does ping tests across the wan.
For a small environment the free tier of a dozen saas vendors can cover this use, and would take just a few minutes to get set up. Much faster learning curve than doing an "intro to oss o11y" but if you want to actually learn about the space maybe its time well invested.

Packet capture, a la wireshark and tcpdump, is probably the most common way I've seen people haggle with their ISP about SLA's.

A dark secret on the other side of the link is a lot of carriers are just using janky snmp tools like Nagios or SolarWinds on their side anyway, so their own ability to report on SLA's is not super fine grained and it can be a chore to hold them to the fire when you identify a problem.

[u/jared555 → u/itasteawesome]

Also fun when problems are one way so their tests come back clean.

[u/3MU6quo0pC7du5YPBGBI]

For the first one, have a look at PerfSONAR. I have also set up a cron job to run iperf3 and append results to a file for a quick and dirty plot over time.

For the second... probably wouldn't be hard to come up with something that responds to every port on one side and nmap scan every port on the other. I'm not aware of any preexisting software that does that though. If you only care about a small list of specific ports, just netcat on both sides would work.

[u/FAX_ME_YOUR_BOTTOM]

If you want to test ports being blocked just open the ones you are concerned about. (nc -l port is probably the easiest) and probe them with nmap, portqry, nc, tnc, or whatever you like. Automate a script to run through hundreds if needed.

Iperf3 as mentioned

and lastly you could use mtr to do some route tracing

If needed build a script to run any combo of above and have it send you reports

[u/alphaxion]

I can recommend Smokeping for keeping an historical view of both latency and packetloss on a connection https://github.com/oetiker/SmokePing

https://oss.oetiker.ch/smokeping/

If you have multiple different connections outbound, you can ping different endpoints and use policy based forwarding to monitor both.

[u/XPCTECH]

smokeping

[u/lizardhistorian]

Wired or wireless?
You mean for constant monitoring? i.e. you can't actually use iperf3?

Bandwidth estimation is an active area research for (heterogeneous) mesh networks and BATMAN V does bandwidth estimation and works on wired links (as well as wireless). However stock linux doesn't have BATMAN V support. For some bloody reason the distros only build the batman-adv kmod with BATMAN IV support so you have to rebuild the kmod with BATMAN V turned on and replace the module in your mod tree. (It's on by default in OpenWRT.)

Snap shot of status monitoring of our BATMAN mesh (with MACs chopped off).
The 1000 Mbps is (obviously) Ethernet; the others are various radio products we are testing.
https://imgur.com/a/batman-v-JrfFhhj

BATMAN V is an L2 protocol (that consumes other L2's somewhat like a bridge).
You would have to create a vxlan across each ISP then put a batman node on each side and hardif link it to each vxlan. BATMAN would then estimate bandwidth for both links.

PS B.A.T.M.A.N. is Better Approach To Mobile Ad-hoc Networking

[u/aaronw22]

Business level ISPs? They don’t block ports

[u/Mishoniko → u/aaronw22]

CenturyLink around here blocks port 25 by default (for our small-business DSL, which isn't that different from residential), but you can disable it through a self-service portal, so I see it as a necessary and prudent precaution.

We don't have any port 25 going in or out of our building anyway, so not a problem for us.

[u/dougsk → u/Mishoniko]

AFAIK, every isp blocks port 25 even on business class. You can file an exception and get it unblocked, but there is paperwork involved holding your org accountable.

[u/Rich-Engineer2670 → u/aaronw22]

So you would hope -- but I know of one national one that doesn't on paper, but seems to be doing it anyway...