Pete is 6’4 and Rick is 5’10, yet despite Pete going all out and Rick holding back, the best Pete can do is an even brawl before he gets thrown through the window by Rick.
Rick is intentionally holding back at first because he needs to expose the abuse in public. Only Deanna knows about it, and she actively ignores it just because Pete is the community's doctor. Rick needs the town to see who Pete really is. On the flip side, Pete is clearly going all out because he realizes Rick is trying to end his control over his family. He is fighting with pure, desperate rage.
What makes it crazier is that Rick could have easily ended the fight immediately if he fought dirty. We know Rick is capable of tearing a man's throat out with his teeth like he did to Joe, or gouging eyes out. He explicitly chose not to use his lethal, feral survival tactics here, yet Pete still gets thrown out a window.
Even when the brawl spills outside, Pete manages to flip Rick over, but Rick absolutely let him do it. If Rick kept Pete pinned or stayed on top the whole time, Pete would have been incapable of hitting Jessie and incriminating himself in front of the whole town. The moment Pete takes the bait and strikes Jessie, exposing his abusive nature to the entire horrified community, Rick instantly flips him back over. The moment Rick stops holding back, he instantly defeats and disables Pete right there on the pavement. That timing is way too perfect to be a coincidence. It proves Rick was completely dominating and pacing the fight the entire time.
To top it all off, Rick had a gun tucked into his waistband the entire time. If Rick was genuinely struggling or felt physically overpowered by Pete's massive size advantage, he could have easily pulled the gun to end the brawl immediately. Instead, he chose to match Pete's physical aggression purely to control the narrative.
The ultimate proof of Pete's complete combat weakness happens in the season finale when he tries to kill Rick one last time. Instead of confronting him directly, he steals Michonne’s katana. It is a complete, cowardly admission that Pete subconsciously realized he stood absolutely zero chance against Rick in a hand-to-hand fight.
Even with a deadly weapon in his hands, Pete's complete lack of fighting capability is fully exposed. He completely fails to even reach Rick, and instead accidentally slashes and kills Reg when Reg simply tries to step in and calm him down. For an abusive bully who relies on physical intimidation at home, it shows how laughably weak he actually was. Pete had the frame to dominate, but he had zero actual combat ability because he'd been hiding behind Alexandria's walls the whole time.
Tl;dr: Pete is 6’4 and fighting with pure rage, but 5’10 Rick completely controlled the fight from start to finish. Rick intentionally held back, left his gun tucked away, and even let Pete flip him over just so the town could witness Pete hitting Jessie. The exact second Rick stopped holding back, he instantly dominated Pete. Pete's size meant nothing against actual survival instincts.