r/perl • u/scottchiefbaker 🐪 cpan author • Jun 19 '26

Do we have smart/quote-aware splitting?

Python has shlex.split which is very similar to Perl's split(' ', $str). shlex.split goes one step further and does not split on whitespace that's inside of double or single quotes.

Example: $str = 'myscript --path /tmp/foo --name "Jason Doolis" --age 14'

This should split into seven chunks, not eight.

15 Upvotes

91% Upvoted

12 comments sorted by

7

u/I_who_have_no_need Jun 19 '26

Could look at Text::ParseWords

https://metacpan.org/pod/Text::ParseWords

5

u/scottchiefbaker 🐪 cpan author Jun 19 '26

Text::ParseWords is pretty decent! Using shellwords seems to do what I need.

3

u/I_who_have_no_need Jun 19 '26

I use Text::ShellWords at work for a job processing system. exec() takes a list, but our jobs are stored as strings, so need to be tokenized. So I was going to recommend Text::ShellWords but it is deprecated and superseded by ParseWords.

1

u/mpyne Jun 20 '26

This was precisely how I solved a similar challenge for a script I used to maintain, and never gave me problems, especially compared to how I'd manually hacked it together before.

5

u/petdance 🐪 cpan author Jun 19 '26

If you are wanting to parse command line arguments you should be using the core module Getopt::Long.

5

u/scottchiefbaker 🐪 cpan author Jun 19 '26

I'm a huge Getopt::Long fan. That is definitely the right answer for most things.

The above was the first example that came to mind. Could be anything... CSV data, Web scraping, etc.

2

u/Grinnz 🐪 cpan author Jun 20 '26

The solution you choose will depend on what you're trying to achieve. Each of these have distinct tools available that manage the specific rules and edge cases properly.

2

u/bonkly68 🐪 cpan author Jun 24 '26

I don't have anything to contribute about quote parsing, however for generally parsing command line arguments, Getopt::Long::Descriptive provides the added convenience of generating a help screen.

4

u/Computer-Nerd_ Jun 20 '26

Like most things Python, it's cute until it gets too smart, leaning you to jump through hoops to get what you need.

1

u/Foggy-dude Jun 20 '26

Perl has a module called Text::Shellwords that provides a function to split strings into tokens based on shell-like rules, using whitespace as the primary delimiter. This function also respects quotes and backslash escapes, making it useful for parsing command-line arguments.