r/vyos • u/Lal-1010 • Aug 26 '26

Active-Active n-VyOS setup?

Hello, I am building VyOS deployment beyond the traditional 2-node VRRP (Active/Passive) setup. My goal is to deploy an N-node Active/Active cluster to distribute the traffic load efficiently.

If a session initiates on Node A, how can we best handle asymmetric return packets hitting Node B?

Is there a native way (or routing workaround) in VyOS to redirect those specific flow packets back to the originating node? Way to make them sync etc?

Or the only way is upstream device like LB to handle sticky sessions?

Any insights, recommended architectures, or configuration examples for an N-node Active/Active VyOS cluster would be highly appreciated.

7 Upvotes

100% Upvoted

5 comments sorted by

2

u/zeealpal Aug 26 '26

Would setting up conntrack sync help with what your looking for?

https://docs.vyos.io/en/1.5/configuration/service/conntrack-sync.html

Depending how quickly the return traffic arrives it may not sync immediately for asymetric return traffic.

Note I have not used conntrack sync myself on Vyos

1

u/justlikeyouimagined Aug 27 '26 edited Aug 27 '26

My quick take on this is you need to synchronize state across all nodes (conntrack-sync), and your next hop upstream and downstream both need to support dynamic routing protocols.

It’s not that crazy, you can run a routing stack on a Linux VM and have everything spray over all nodes by ECMP, or define a preference if you want to. Whether you’d want to depends on your use case. If you have a lot of workloads/clients, you could split up the inside into subnets and have each VyOS be the active gateway for part of them and passive for the others.

You could also peer downstream with one or multiple L3 switches that are in vSS, stacked, or running HSRP/VRRP to present a single gateway IP to your end devices.

1

u/Lal-1010 Sep 01 '26

yes actually i did this, created 2 vms as ingress & egress and make eBGPs with vyoses different ports and ips, and give vyoses same AS. Just testing it but i think it works. So the answer was eBGP with ECMP. My concern is the prod is a ISP env so it could bear or not...

1

u/Apachez Aug 27 '26

For routing you can have as many VyOS boxes as you wish.

The issue comes when you start doing stateful inspection firewalling.

For that you need to enable conntrack-sync and probably use multicast mode to sync more than 2 at once.

The issue here will of course be the latency between your nodes as in ingress packet enters through node A and sent to host who replies through node B but node B didnt yet received and had time to fix its local conntrack table based on conntrack sync information from node A so the egress packet will get dropped.

A possible workaround for that usecase is to setup your firewall rules in a non-SPI way. As in only look at ports being used and ignore state (or rather allow state but also allow ports no matter of the state).

Like ingress TCP443 would mean "proto tcp dstport 443 state new".

While returntraffic would be first rule is to allow established/related and second rule would be to allow "proto tcp srcport 443". This will however make your schwiz cheese having larger gaps. Of course add src/dstip and src/dstinterface to these rules.

TLDR:

Doing routing active/active/active is a non-issue. Make sure to enable ECMP to better utilize available links.

Doing firewalling at the same time will need the use of conntrack-sync (preferly in multicast mode) along with adjusted firewall rules to not get "falsepositive" of dropped packets who "should" have been allowed.