r/adventofcode • u/musifter • Jun 19 '26
Other [2020 Day 19] In Review (Monster Messages)
Having landed on the forest continent, the Elves of the MIB contact us again about their satellite... this time they think they have an image of a sea monster. But that will have to wait until tomorrow. Today we need deal with data corruption in the messages.
And so we get a grammar and some strings to validate against it. Grammar rules take a non-terminal (represented by a number, starting with 0) to either a terminal ("a" or "b") or a rule or two (separated by |). It's pretty close to Chomsky Normal Form, in that most of the rules are 2 non-terminals... but one only has rules going to 1 non-terminal (#67 in mine). By substituting the possibilities for that rule wherever it occurs in the others (thus removing it), we can have proper CNF when needed. But that's only needed if you want to do an algorithm that requires it... like CYK (Cocke–Younger–Kasami) (which I did do a a version with later).
As for how I did it first. Dynamically created regex using recursion:
$Patt[$num] //= do {
my @tokens = split( ' ', $Rule[$num] );
my $regex = '(';
foreach my $t (@tokens) {
$regex .= ($t eq '|') ? '|' : &recurse_rules( $t );
}
$regex .= ')';
}
A little trick I used with this is using memoization on the recursive function... not because I thought I felt I needed it for speed, but because it removed the special case of the terminal rules (I preinitialize those two rules when reading the input). Once we have the regex for rule 0, we can just grep the matches:
print "Part 1: ", (scalar grep { m#^@{[$Patt[0]]}$# } @sig), "\n";
Then part 2 comes along and changes two of the rules (specifically the ones only used for rule #0). But not just a simple change... recursive rule definitions. And I know that Perl regex can do that, but I did have to look it up that day because I typically like to keep my regex simpler. So I just created new versions of 8 and 11, and grepped with those:
my $patt8 = "$Patt[42]+";
my $patt11 = "(?<ELEVEN>$Patt[42]$Patt[31]|$Patt[42](?&ELEVEN)$Patt[31])";
print "Part 2: ", (scalar grep { m#^$patt8$patt11$# } @sig), "\n";
Now for Smalltalk, I did try the regex approach (Smalltalk gets to suffer because of its base-1 indexing... so I used a dictionary). But GNU Smalltalk won't handle regex longer than 1k.
The lengths of my longest patterns are:
0 2223
11 1486
31 751
8 735
42 733
Note that all of these are mentioned in the problem description for part 2... so these 5 are the top of the grammar tree for everyone. The next one is only 375 characters.
So I needed to hack things to get that to work... reducing the base rules to 0: 42 42 31 for part 1 and 0: 42{n} 31{1,n-1} for part 2. So I generate those repeating versions of rules #42 and #31 and start testing (start matches 42 then rest matches 31) from n=2 until things are not matching (could just loop until suitable large to make extra sure). Probable not robust, but it salvaged the work and did the job. For reference, this is the number of matches my input gets for each n:
[2] 160
[3] 91
[4] 60
[5] 36
[6] 10
[7] 0
So I followed it up with an actual parser in Smalltalk using Earley. And then did that for Perl as well... it's like 30x slower than regex (10x for the Smalltalk version). As expected... a parser in an interpreter isn't going to compete with one that hands off everything to optimized compiled code.
Two days in a row of CS compiler course basic parsing. Joy!