Using argparse and cmd together

Written May 2020 - No GitHub Gists version

I'm currently creating an interactive tool in Python to do CRUD things. This article focuses on how I'm using two Python standard libraries, argparse and cmd, to enable user interaction.

Here's a very simple cmd-based tool that does two things: create and delete servers, along with some sample output from running the tool.

The important thing to note here is that arg is a single string containing all the text after the name of the command. This means that if we want to pass more than one argument to this tool, we'll need to split arg manually. This differs from sys.argv, which is already a list of strings.

Let's try to modify to handle an additional argument in the do_create function.

Great, we can now interpret two arguments. However, if we were to add more arguments, the user would need to remember the order of arguments, and it would be clumsy to unpack an array into a large number of individual local variables. Additionally, this method of parsing arguments doesn't support optional arguments cleanly. These are problems that the standard library argparse has solutions for. We can use argparse.ArgumentParser to scale the number of arguments in a maintainable way, and support optional arguments at the same time.

At this point, you might be tempted to think that we're done, and that we can just use all the features of argparse to solve all our argument-parsing problems. However, there are some issues with the above implementation. The first problem is that this solution doesn't currently support spaces in arguments.

Can fix this by wrapping my server between quotations, or by escaping the space with a backslach (\ )? No, because these are shell constructs. We are already inside a Python program, so the usual space-escaping rules do not work in our program.

One way this can be fixed is to replicate the space-escaping behaviour that shells provide. I chose not to do this because I didn't want to implement a shell feature in my CRUD. Instead, I chose to specify nargs='+' for any optional string arguments. By default, ArgumentParser parses such argument into a list of strings. However, I just want one string as the argument (remember, we're parsing a name field). I chose to accomplish this by implementing a custom Action. Once we tell our ArgumentParser to use this custom Action, we get the desired result: an argument with spaces in it.

There's another issue in 3_output_2. Notice how the program exits to shell after ArgumentParser fails to parse the unknown argument. ArgumentParser does this by calling sys.exit(). This makes sense for a program launched from a shell, but exiting to the shell doesn't make sense in our interactive tool. We want for the user to be able to run other commands within the cmd loop after an error. PEP 389 addresses this, and says we should subclass ArgumentParser and override the error method (1). Let's go ahead and do that.

The user now sees a helpful error message when they input command arguments incorrectly. The parameter description (NAME [NAME ...]) looks a bit weird, but this is the tradeoff for not having to re-implement space-escaping.

Another neat feature of cmd is that cmd programs have a default command called help, which displays help text for commands within the program. We can use the argument parser that we wrote for each function to display this help text. Let's extract the parser creation out to a function and use the parser in the help method.

If you're familiar with argparse you might be wondering why I bothered to write this code just to print some help text. By default, all ArgumentParsers have a default option --help which prints the same text without needing to call print_help manually. Let's see what happens if we try to use invoke create with the --help option.

Using --help exits the program; this should seem familiar. Indeed, the --help flag also results in a sys.exit()! We'll need to implement our own --help if we want to prevent the program from exiting. To prevent the body of our program from having to deal with argument parsing and handling, let's implement a custom Action again. This new Action will mimic the behaviour of the existing help flag, but only raise an exception, not terminate the whole program.

And with that, we have implemented a cmd-based interpreter that interprets user input using the argparse library. If I discover any other weird results of using argparse within a cmd-based interpreter, I will be sure to make a followup post.

  1. (1) I'm kind of curious if "if this turns out to be a common need." has turned out to be true. Maybe I'll write a followup post on this topic.