Hypothesis (Python): Omit argument

Question

I have a function like so (it's actually a class, but that's not relevant given Python's duck typing):

def myfunc(a=None, b=None):
    <snip>

Now I want to write a Hypothesis test which always supplies a, but only sometimes b.

I tried

from hypothesis import given, strategies as strat

@given(a=strat.booleans())
@given(b=strat.integers(min_value=1) | strat.nothing())
def test_model_properties(self, **kwargs):
    myval = myfunc(**kwargs)
    <snip>

But it seems that when it gets strat.nothing() just skips that test run (I get hypothesis.errors.FailedHealthCheck: It looks like your strategy is filtering out a lot of data. when using that as the sole strategy for b).

How can I only sometimes supply an argument with a Hypothesis test? Do I need to write two tests, one with b and one without?

jacg · Answer 1 · 2017-11-01T13:00:00.303

2

Your approach is guaranteed to fail, because, as the hypothesis docs imply

hypothesis.strategies.nothing()[source]

    This strategy never successfully draws a value and will always reject on an attempt to draw.

your attempt not to provide a value for b will always fail.

How about this:

from hypothesis.strategies import tuples, integers, booleans, one_of
B = booleans()
I = integers(min_value=0, max_value=10)
one_of(tuples(B), tuples(B, I)).example()

which, over a bunch of trials, gave me outputs such as (True,), (False, 9), (False, 4), (True, 5) and (False,).

You would, of course, use this with *args rather than **kwargs.

edited Nov 01 '17 at 13:00

answered Nov 01 '17 at 12:53

jacg

1,728
10
21

Unfortunately, because I have more than 1 "optional" argument (I simplified it for the question), keyword arguments are necessary. Also, if the function accepts `*args` or `**kwargs`, then you can't supply positional arguments to `@given` [(see `is_invalid_test` in these docs)](http://hypothesis.readthedocs.io/en/latest/_modules/hypothesis/core.html) - you'd need function parameters of `(self, args)` - note no `*`. Thanks for the suggestions though - they did put me on the right track. – Scott Stevens Nov 14 '17 at 11:57

score 1 · Accepted Answer · answered Nov 14 '17 at 11:58

jacq's answer put me on the right track - the selection of keywords needs to be its own strategy.

With the standard dictionary

std = {'a': strat.booleans()}

and the optional dictionary

opt = {
    'b': strat.integers(),
    'c': strat.integers(),
}

I can then use chained list comprehension for all the possible "optional argument combinations":

# chain.from_iterable may or may not be faster; it doesn't matter for me.
optional = [combo
            for n in range(len(opt.items()))
            for combo in itertools.combinations(opt.items(), n+1)]

That generates the key-value tuples for b, c, and (b, c).

In order to draw a set of values, we need to get one of those options, which can be done with sampled_from(optional). With the obtained tuples, we must draw from the strategies within, in addition to those in the std dictionary.

strat.sampled_from(optional).flatmap(
    lambda x: strat.fixed_dictionaries(
        {**std, **dict(x)}
    )
)

This can all be wrapped in a function, let's call it valid_values(). You can't use @given(valid_values()) if you specify *args or **kwargs in the signature of the wrapped function.

As a result, test_model_properties(self, **kwargs) becomes test_model_properties(self, kwargs) (and you can use @given(kwargs=valid_values())) - by calling the dictionary kwargs, the rest of the function remains unchanged.

Note: This will not include an empty tuple if you want the possibility of no optional parameters, but that can be appended to the optional list easily. Alternatively, have range(n+1) instead of combinations(..., n+1), hence including a length of 0.

score 1 · Answer 3 · answered Feb 01 '18 at 02:58

It looks like you want none() instead of nothing():

from hypothesis import given, strategies as strat

@given(a=strat.booleans(), b=strat.none() | strat.integers(min_value=1))
def test_model_properties(self, **kwargs):
    myval = myfunc(**kwargs)
    ...

This is simpler than generating dictionaries to use as **kwargs and a little more efficient too. Ordering of the strategies for b is also important - putting none() first ensures that the minimal example will be a=False, b=None instead of a=False, b=1.

Also note that applying @given multiple times is very inefficient compared to a single use, and actually deprecated since version 3.34.0.

Paul Rubenstein · Answer 4 · 2017-11-01T09:58:28.900

-1

How about:

def myfunc(a=None, b=None):
    if b is None:
        b=strat.nothing() 
        # Or whatever you would like b to be when you don't supply an argument
    else:
        <snip>

So you let b being the default value (in this case, None) trigger an 'if' condition inside myfunc() that sets it to something else.

edited Nov 01 '17 at 09:58

answered Nov 01 '17 at 09:55

Paul Rubenstein

312
2
12

I want to test the default values of `myfunc`, so I actually have that code in place already - I need the test to sometimes call with `b` set, and sometimes not. – Scott Stevens Nov 01 '17 at 09:57

Hypothesis (Python): Omit argument

4 Answers4