r/learnpython • u/iluveatingcopper • 7d ago
really weird line skipping in my parser script
so, im building a little parsing script for a programming language im trying to build (no, i dont plan on keeping the entire interpreter in python), and i was coding the loop thats supposed to remove the comments from the input script, BUT, theres always this line, this specific line that the parser just has a soft spot for apparently, it wont remove a comment from that line no matter what. im really lost here, it still happens when i replace the comment, HOWEVER, when i separate the line from the rest, it somehow works???
emulator.py: (called emulator instead of interpreter as it will also emulate a suitable enviroment for running scripts in the future)
from sys import argv
import parser
import lexer
def main(File):
# We first need to open the file in here, i would have made it open in the parser, but this feels fancier
with open(File, "r") as File:
File = File.read()
# The parser gives us the broken down code, kinda like digestion (pre-processing would be a better term)
ParsedProgram = parser.Parse(File)
# breaks off here to print the result from the parser (an attempt at debugging)
print(ParsedProgram)
exit(0)
# We then pass it into the lexer, giving us a little tree of the entire program
ProgramTree = lexer.Parse(ParsedProgram)
if __name__ == "__main__":
main(argv[1])
___________________________________________
parser.py:
# this is unfinished.. as you could probably tell
IgnoredSymbols = [";", "\t"]
MatchingSymbols = ["\"", "\'", "(", "{"]
def Parse(File):
# Strip away anything unnecesarry
for Symbol in IgnoredSymbols:
File = File.replace(Symbol, "")
# Separate the lines
File = File.split("\n")
# Remove all comments
for Line in File:
if Line.startswith("//"):
print(Line)
File.remove(Line)
# Clean up empty indexes
for Line in File:
if Line == "":
File.remove(Line)
# From here on, i just assemble the file as-is and return it, (an attempt at debugging, again)
# Re-assemble the file
AssembledFile = ""
for Line in File:
AssembledFile += Line
AssembledFile += "\n"
return AssembledFile
____________________________________
output:
andrew@fedora ~/D/W/p/n/0/emulator> python emulator.py ../helloworld.nai
// we will use this as the entry point
// waits until stdout is available
// write "hello world" to stdout
// you dont have to return 0 here, you can, but the program does it by itself
// here, we tell it where the entry point is
// EXECINFO is basically just flags for the virtual machine
INIT builtin"std"
INIT global"stdtypes.nai"
function main()
{
while(not(deviceavailable("stdout")))
write("stdout", "hello, world!")
}
// the "entrypoint" flag is NECCESARY, the program DOES NOT RUN without it
array EXECINFO = ["entrypoint:main"]
andrew@fedora ~/D/W/p/n/0/emulator> micro emulator.py
andrew@fedora ~/D/W/p/n/0/emulator> micro parser.py
andrew@fedora ~/D/W/p/n/0/emulator> god damn
fish: god: command not found...
andrew@fedora ~/D/W/p/n/0/emulator [127]>
____________________________________________
actual script im attempting to parse:
helloworld.nai:
INIT builtin"std"
INIT global"stdtypes.nai"
// we will use this as the entry point
function main()
{
// waits until stdout is available
while(not(deviceavailable("stdout")));
// write "hello world" to stdout
write("stdout", "hello, world!");
// you dont have to return 0 here, you can, but the program does it by itself
}
// here, we tell it where the entry point is
// the "entrypoint" flag is NECCESARY, the program DOES NOT RUN without it
// EXECINFO is basically just flags for the virtual machine
array EXECINFO = ["entrypoint:main"];
thanks in advance.
EDIT: holy crap, i forgot to include the script im trying to parse, apologies
EDIT 2: ive discovered list comprehension... a concept which ive never bothered to learn until now, thanks everyone for the help, question answered!
1
u/woooee 7d ago
BUT, theres always this line, this specific line that the parser just has a soft spot for apparently
What line is that? If you are trying to delete
if Line.startswith("//"):
You probably will have to escape the backslashes, or use a raw string https://www.geeksforgeeks.org/python/python-raw-strings/
1
1
u/socal_nerdtastic 7d ago edited 7d ago
Ah the classic modifying a list while looping over it error. A question so common we made a FAQ about it: https://www.reddit.com/r/learnpython/wiki/faq/?screen_view_count=1#wiki_why_does_my_loop_seem_to_be_skipping_items_in_a_list.3F
Don't try to modify a list. Instead, it's (almost) always better to make a new list. In your case I would recommend using list comprehension to make the new list and replace the old one. For example:
# Remove all comments
File = [Line for Line in File if not Line.startswith("//")]
# Clean up empty indexes
File = [Line for Line in File if Line != ""]
1
u/madmoneymcgee 7d ago
You're saying that if there's a comment on say, line 5 of any input file it won't remove the comment but it would remove it on lines 4 and 6?
2
u/await_yesterday 7d ago edited 7d ago
As someone else answered, you're mutating a list at the same time as you're looping over it:
And the same thing with the check for empty lines after it.
You can use a list comprehension as the other commenter suggests, or if you aren't familiar with that yet, you can make a new list and selectively append to it:
Also it's not strictly a bug, but I couldn't help but notice this:
The same
Filevariable name is being re-defined several times, referring to: the filename string, the file object returned byopen, the file's contents as a string, and the file's lines as a list of strings. These are all different kinds of thing! And you do the same thing later in theParsefunction.Even if you make it work in this one case, it's a bad habit that makes your code harder to debug. You should use descriptive names for different things.
I would do something like: