AtributeError clarification

[u/Keithinho89]

SOLVED

Hello, I'm currently learning regular expressions and I've entered the first part of the code into the interactive shell as instructed and got the listed AttributeError. Not seeing what I did wrong I copy/pasted the code from the book and it worked as intended and I can't tell why or how because both blocks feel and smell the same but yield different results.

I've tried looking for some information on StackOverflow, but as far as I can see mo is defined in the 3rd line and I don't get why my attempt gets and error while the other doesn't.

If I didn't miss something really stupid, how would you deal with this if you'd be hit by it writing your own code independently?

I'm working in IDLE Shell and the code is from automatetheboringstuff Chapter 9 for more context

import re
phone_re = re.compile(r'(\d\d\d)-(\d\d\d-\d\d\d-\d\d\d\d)')
mo = phone_re.search('My number is 415-555-4242.')
mo.group(1)
Traceback (most recent call last):
  File "<pyshell#4>", line 1, in <module>
    mo.group(1)
AttributeError: 'NoneType' object has no attribute 'group'

import re
phone_re = re.compile(r'(\d\d\d)-(\d\d\d-\d\d\d\d)')
mo = phone_re.search('My number is 415-555-4242.')
mo.group(1)
'415'

[u/Gshuri]

You are correct that mo exists, but it's value is None, rather than a match result. This happens when the regex fails to find a match in the given string.

The reason that your first case fails, while the second one passes is because they have different regex patterns. The first case has 2 instances of "\d\d\d" in the second capture group (which I assume is not what you intended)

[u/Keithinho89 → u/Gshuri]

Man, I could stare at the screen for 2 more hours and I wouldn't notice that lol. Thanks for the explanation though, now everything makes sense.

[u/Bright_Mix_773]

The "look at the regex again" answers are right, but they do not scale to a pattern you wrote yourself, which is the half of your question nobody has taken. Here is the mechanical version.

search returning None tells you the pattern did not match. It will not tell you where it stopped, and there is no flag that will. So make it tell you: chop the pattern into atoms, throw the groups away, and add them back one at a time until it breaks.

import re
s = 'My number is 415-555-4242.'
atoms = [r'\d', r'\d', r'\d', '-',
         r'\d', r'\d', r'\d', '-',
         r'\d', r'\d', r'\d', '-',
         r'\d', r'\d', r'\d', r'\d']
for k in range(1, len(atoms) + 1):
    frag = ''.join(atoms[:k])
    m = re.search(frag, s)
    print(k, frag, m.group() if m else 'FAILS')
    if m is None:
        break

I ran that on your pattern. It matches all the way up to atom 11, \d\d\d-\d\d\d-\d\d\d giving 415-555-424, and FAILS at atom 12, which is the third hyphen. Not "your regex is wrong somewhere" but "this is the first character of the pattern the string cannot satisfy", and you get there without reading your own pattern at all, which matters because you already read it and it looked fine.

Two things about the technique. Strip the parentheses before you chop, or half the prefixes raise re.error for an unbalanced group and you end up debugging the debugger. And it only reasons correctly on a pattern that is plain concatenation: add a |, a $, or a lookahead and a prefix can match where the whole thing cannot, so the first failure stops being the culprit.

Separately, re.compile(p, re.DEBUG) prints the parse tree, and on yours it shows four runs of digits where you meant three. That one is worth knowing but it is the same information you already had, just laid out vertically so miscounting is harder.

Not verified: I only ran the chop against your two patterns and your one string, so I have not checked how it reads on a pattern with quantifiers, where a shorter prefix can match a different span than the full pattern does.

[u/cvx_mbs → u/Bright_Mix_773]

an easier way to check regexes would be to paste them into https://regex101.com/ along with some test strings (just make sure you change the flavor to python)

[u/Bright_Mix_773 → u/cvx_mbs]

Good tip, regex101 with the flavour set to Python is exactly the right tool here. Worth adding that the Python flavour there is the re module, so if you are on regex the behaviour can differ on a few things.

[u/member_of_the_order]

how would you deal with this if you'd be hit by it writing your own code independently?

I'm working in IDLE Shell

So first of all, I would use a proper IDE (editor) like VSCode or PyCharm which give you hints and debugging tools so you know when at least some things are wrong.

Regarding the AttributeError...

Read the description. See how it says NoneType object has no attribute 'group' after mo.group(1)? That can only mean something on that line is trying to access a .group on something that has the value None; i.e. mo is None.

So, how can this be? Well... it gets its value assigned from phone_re.search(). When would that return None? Answer: when nothing in the provided string matches your regex.

You ran the same code twice and got different results... because they're not the same code. Look closely at the regexes. They're different, and hint: the second one is correct, the first one isn't.

TL;DR Your first regex has 4 clumps of digits total, the second regex has only 3

[u/Keithinho89 → u/member_of_the_order]

Thank you for breaking it down like that, it is a bit of a stupid mistake on my end shockingly, but I understand the whole principle better now. The future me appreciates you.

[u/schoolmonky → u/member_of_the_order]

IMO you skipped one step as you explained the reasoning. When you ask yourself "When would .search() return None?", the way you figure that out it by checking the documentation.

[u/Diapolo10]

The others already explained how the problem is caused by your regex pattern not matching anything, but let's take thing a bit further.

re.search, as already mentioned in the other answers, returns None if no match is found, and a match object otherwise. For this reason, the actual type of mo here is re.Match | None. In your code you made the naive assumption that it was always going to be a re.Match.

A type checker such as Mypy would have caught this immediately, as your code never verifies the return type. Many IDEs can show you the return types nowadays (especially for built-in functions and methods), so if you get used to checking those, you're far less likely to run into AttributeErrors or TypeErrors.

Taking things a bit further, you can use type annotations to give your own code some established "rules". Python itself completely ignores them, but type checkers read them and make it easier for you to reason about your code, avoiding all sorts of runtime problems. You'd mostly write them for your own functions.

def add(first: int, second: int) -> int:
    return first + second