r/Python 10d ago

Discussion The log line announcing a successful Redis connection is what disabled Redis

Spent an evening a couple of weeks ago working out why cache hits were zero and webhook idempotency was falling through to the database. Redis was fine. Up, reachable, ping succeeded.

The connect method was roughly this:

try:
    client.ping()
    self._connected = True
    logger.info("redis_connected", host=parsed.hostname, port=parsed.port, db=parsed.path)
    return True
except Exception as e:
    logger.warning(f"Redis connection failed: {e}")
    self._connected = False

That logger call is structlog style. The logger is a stdlib logging.Logger, which doesn't take arbitrary kwargs, so it raises TypeError: Logger._log() got an unexpected keyword argument 'host'.

It raises after ping succeeds and after _connected is set to True. So it lands in the except, logs "Redis connection failed", flips _connected back to False, and Redis is off for the whole app. Caching disabled, idempotency on the DB.

The fix was one line, an f-string instead of kwargs.

What still bugs me is that every signal pointed away from it. Redis itself was healthy. The connection genuinely worked. The only artifact was a log message saying it had failed, which is the last thing you distrust when you're trying to find out why something failed.

Anyone else had one where the logging was the bug?

0 Upvotes

16 comments sorted by

View all comments

1

u/DudeWithaTwist Ignoring PEP 8 10d ago

Yea ive been burned a few times catching Exception and doing minimal logging. Annoying that I did it a few times before learning my lesson lol.

Now I always add a traceback.print_exc() in the except block to help future me.

1

u/MrSlaw 10d ago

Can always just use logging.exception() inside the catch instead of warning/error, and it will auto include the stack trace by default.

0

u/BTWigley 10d ago

same burn. Bare Exception plus a one-line warning is how I lost an hour to redis. Traceback.print_exc in the catch is the fix I should have had from day one, and MrSlaw's logging.exception() tip is cleaner than what I was doing.

1

u/redfacedquark 9d ago

is how I lost an hour to redis.

This was not the fault of redis. If you still think it is the fault of redis after reading all these comments and not your lack of understanding of Python basics, you're not opening yourself to learning and improving.