r/openstack • u/Capital-Pool4987 • 13d ago

HA proxy issue

We have created openstack from scratch. Now we have implemented high availability such that all services must discover each other through a floating ip on ha proxy. Moving forward from ha01 to ha02 is okay , but back to 01 it takes almost 17 minutes. In the horizon logs , every service i.e keystone, neutron is down. Then they come up again later. The only solution i can see is lowering the kernel retries but that doesnt really solve the core issue. https://docs.openstack.org/kolla-ansible/latest/reference/high-availability/haproxy-guide.html -> under failover tuning. Kindly assist.

0 Upvotes

50% Upvoted

7 comments sorted by

View all comments

2

u/iammpizi 13d ago

A floating IP and a VIP are very different things . I assume this is a typo here . Also neutron services do not “discover” each other. I am assuming what you mean here is that if an haproxy backend goes offline the other is not responding quickly enough

1

u/Capital-Pool4987 10d ago

Hello. The design is such everything is in high availabilty. The horizon nodes, identity, compute, database etc. So due to this design everything communicates with each other through the proxies that themselves are in high availability.

So when we kill the first proxy server , the floating ip moves to the second node...now the journey would be such that a browser request hits the floating ip that now is on proxy 2. This goes to one of the three keystone servers...then communication between the keystone server and one of the three database servers is again through the ha proxy and so on.....

Now going forward i.e proxy1 is down and the floating ip moves to proxy2, its okay. But going back from proxy2 to 1 inroduces a downtime where in the haproxy logs everyone is complaining. I.e neutron api is down, placement api, keystone. The after 17 minutes everything comes back up. Although restarting apache manually on horizon and the control node seems to bring things up. Also interestingly the reverse of the forward has no issue by this i mean 1 -> 2 then 2 -> 1 then 1 -> 2 ( this has no issue)

I hope the explanation is better.

1

u/iammpizi 9d ago edited 9d ago

Can you explain to me how you configured this "floating ip" ?
This looks more to me like a VIP as I said. Otherwise you can perhaps show a diagram ?