How to pass several list of arguments to @click.option

Question

I want to call a python script through the command line with this kind of parameter (list could be any size, eg with 3):

python test.py --option1 ["o11", "o12", "o13"] --option2 ["o21", "o22", "o23"]

using click. From the docs, it is not stated anywhere that we can use a list as parameter to @click.option

And when I try to do this:

#!/usr/bin/env python
import click

@click.command(context_settings=dict(help_option_names=['-h', '--help']))
@click.option('--option', default=[])
def do_stuff(option):

    return

# do stuff
if __name__ == '__main__':
    do_stuff()

in my test.py, by calling it from the command line:

python test.py --option ["some option", "some option 2"]

I get an error:

Error: Got unexpected extra argument (some option 2])

I can't really use variadic arguments as only 1 variadic arguments per command is allowed (http://click.pocoo.org/5/arguments/#variadic-arguments)

So if anyone can point me to the right direction (using click preferably) it would be very much appreciated.

Answer 1

You can coerce click into taking multiple list arguments, if the lists are formatted as a string literals of python lists by using a custom option class like:

Custom Class:

import click
import ast

class PythonLiteralOption(click.Option):

    def type_cast_value(self, ctx, value):
        try:
            return ast.literal_eval(value)
        except:
            raise click.BadParameter(value)

This class will use Python's Abstract Syntax Tree module to parse the parameter as a python literal.

Custom Class Usage:

To use the custom class, pass the cls parameter to @click.option() decorator like:

@click.option('--option1', cls=PythonLiteralOption, default=[])

How does this work?

This works because click is a well designed OO framework. The @click.option() decorator usually instantiates a click.Option object but allows this behavior to be over ridden with the cls parameter. So it is a relatively easy matter to inherit from click.Option in our own class and over ride the desired methods.

In this case we over ride click.Option.type_cast_value() and then call ast.literal_eval() to parse the list.

Test Code:

@click.command(context_settings=dict(help_option_names=['-h', '--help']))
@click.option('--option1', cls=PythonLiteralOption, default=[])
@click.option('--option2', cls=PythonLiteralOption, default=[])
def cli(option1, option2):
    click.echo("Option 1, type: {}  value: {}".format(
        type(option1), option1))
    click.echo("Option 2, type: {}  value: {}".format(
        type(option2), option2))

# do stuff
if __name__ == '__main__':
    import shlex
    cli(shlex.split(
        '''--option1 '["o11", "o12", "o13"]' 
        --option2 '["o21", "o22", "o23"]' '''))

Test Results:

Option 1, type: <type 'list'>  value: ['o11', 'o12', 'o13']
Option 2, type: <type 'list'>  value: ['o21', 'o22', 'o23']

Answer 2

If you don't insist on passing something that looks like a list, but simply want to pass multiple variadic arguments, you can use the multiple option.

From the click documentation

@click.command()
@click.option('--message', '-m', multiple=True)
def commit(message):
    click.echo('\n'.join(message))

$ commit -m foo -m bar
foo
bar

Answer 3

The following can be an easier hack fix:

#!/usr/bin/env python
import click
import json

@click.command(context_settings=dict(help_option_names=['-h', '--help']))
@click.option('--option', help='Whatever')
def do_stuff(option):
    try:
        option = json.loads(option)    
    except ValueError:
        pass

# do stuff
if __name__ == '__main__':
    do_stuff()

This can help you to use 'option' as a list or a str.

Answer 4

@Murphy's "easy hack" almost worked for me, but the thing is that option will be a string unless you single quote the options, so I had to do this in order to recompose the list:

#!/usr/bin/env python
import click
import json

@click.command(context_settings=dict(help_option_names=['-h', '--help']))
@click.option('--option', help='Whatever')
def do_stuff(option):
    try:
        option = json.loads(option)
        # option = str(option)  # this also works
    except ValueError:
        pass

    option = option[1:-1]  # trim '[' and ']'

    options = option.split(',')

    for i, value in enumerate(options):
        # catch integers
        try:
            int(value)
        except ValueError:
            options[i] = value
        else:
            options[i] = int(value)

    # Here use options as you need

# do stuff
if __name__ == '__main__':
    do_stuff()

You could catch some other types

To use it, enclose the list into quotes:

python test.py --option "[o11, o12, o13]"

Or you can avoid quotes by not leaving spaces:

python test.py --option [o11,o12,o13]

Answer 5

The answer of Stephen Rauch gave me some issues when working with Kubernetes arguments. This because the string formatting becomes a hassle and often new lines, single quotation marks or spaces were added before and after the array. Hence I made my own parser to improve this behaviour. This code should be self explanatory.
Note that this code does not support single or double quotation marks ' or " in the variables itself.

Custom class:

class ConvertStrToList(click.Option):
    def type_cast_value(self, ctx, value) -> List:
        try:
            value = str(value)
            assert value.count('[') == 1 and value.count(']') == 1
            list_as_str = value.replace('"', "'").split('[')[1].split(']')[0]
            list_of_items = [item.strip().strip("'") for item in list_as_str.split(',')]
            return list_of_items
        except Exception:
            raise click.BadParameter(value)

Custom class usage:

@click.option('--option1', cls=ConvertStrToList, default=[])