Async Step Definitions

[jeduden]

is there away to use async step definitions with pytest-bdd ?

For a step definition like:

@when('i send cucumbers ')
async def i_send_cucumbers(loop):
      pass

I see the warning:
pytest_bdd/scenario.py:137: RuntimeWarning: coroutine 'i_send_cucumbers' was never awaited

[olegpidsadnyi]

what about async then and async given? does it even make sense?

[olegpidsadnyi]

will this work for you?

async def send_cucumbers():
pass

@when('i send cucumbers ')
def i_send_cucumbers(loop):
loop.run_until_complete(send_cucumbers)

[jeduden]

@olegpidsadnyi This works, however, in my case all step definitions are async (and can call multiple times into async functions). so this would mean a lot of calls to run_until_complete.
As a work-around I have defined a decorator that schedules the function on the event loop.

A generic solution would be to have the ability to easily wrap any step definition with a function. (similar to the before and after hooks).

And then it would be great if support of async functions would be provided out of the box.

Especially, after reading the discussion about implicit and explicit eventloops in the thread here: https://groups.google.com/forum/#!msg/python-tulip/yF9C-rFpiKk/tk5oA3GLHAAJ seems to be tending towards implicit loops for python code that runs on 3.5.3+ and/or 3.6+.

So code that explicitly calls run_until_complete to run async functions will look more and more awkward.

[olegpidsadnyi]

@jeduden this means pytest-bdd has to depend on yet another library asyncio? I don't really see how a gherkin scenario can by async. If the whole point that it has not to break i think you need to extend your testing suite with some kind of support for async functions (decorator is a good idea).

I think semantically you can't really represent async process in step-by-step imperative Gherkin. Therefore it should be explicit. If you have to initiate few async messages - they should stand behind one When step that describes this exercise in a form that humans understand. Gherkin is not a programming language, it is the way to describe steps to humans which can't do async computation in their brains.

What do you think, @bubenkoff ?

[bubenkoff]

well, the async question comes down to the pytest, not specifically to pytest-bdd.
And for pytest's dependency injection, it's not realistic that it starts to support async fixture definitions anytime soon. Also for tests, while it sounds cool there seems to be a little win to have async fixtures, simply because fixtures should be fast enough, and then it should not matter much if you optimise dependency graph in a way that you run fixtures in parallel for some parts of the graph.
It worth effort though to add the 'async clause' to the documentation mentining the workaround to cut the async point on the fixture definition by waiting for async function to finish

[olegpidsadnyi]

@jeduden could you show your decorator implementation for when step that is awaiting?

[bubenkoff]

FYI: there's a helper apparently to minimize the efforts: https://pypi.python.org/pypi/pytest-asyncio

[jeduden]

Support on pytest side is fine. We are using aiohttp test-support and with pytest-asyncio 0.8.0 we can also the helpers from that package.

However, these helpers work on scenario level.

The step_definitions are one level below, and since the calling code here: https://github.com/pytest-dev/pytest-bdd/blob/master/pytest_bdd/scenario.py#L137
is not checking if step_func is async. i.e. it only supports synchronous step functions.

In order to fix this, we currently need to wrap all steps with decorators like this:

 def sync(func):
      @wraps(func)
      def synced_func(*args, **kwargs):
          loop = kwargs.get("loop")
          if not loop:
              raise Exception("Need loop fixture to make function sync")
          return loop.run_until_complete(func(*args, **kwargs))
      return synced_func

[jeduden]

@olegpidsadnyi declaring a function async doesn't mean it is not imperative. Async functions are imperative like synchronous functions. However, with await / yield from you explicitly define points where the execution of other scheduled coroutines is allowed.

In order to schedule the parallel execution of multiple asynchronous functions you would use helpers like asyncio.gather ( see https://docs.python.org/3/library/asyncio-task.html#example-parallel-execution-of-tasks )

The reason why we need to declare our step_definitions async is because we are using the aiohttp client inside to perform networking calls as part of our step definitions.
Furthermore, the rest of the code base is completely async, hence we think it is only natural that also step_defintions are written with async.

[bubenkoff]

As i see from the pytest-asyncio code, it just awaits for every 'async' fixture, so there's no real parallelism possible between the fixtures
And how do you mean pytest upports async fixtures in a 'proper' way? Can't really see that

[bubenkoff]

But i do see your point about the need of the wrapper everywhere - it sucks.
Looks like the only way to avoid that is to depend on pytest-asyncio and use it's helpers directly in the pytest-bdd

[jeduden]

@bubenkoff even that is not possible since there is no hook that allows me to insert a custom decorator on the functions.

even if pytest-bdd doesn't support async step definitions out of the box. A hook for wrapping a step definitions would help to reduce duplication.

[bubenkoff]

but I meant to change pytest-bdd itself to support that automatically

[bubenkoff]

are you up for making a PR which will add pytest-asyncio as a dependency and automatically use it to resolve async step definitions, if they are async?

[jeduden]

@bubenkoff In general, I am happy to make a PR. One question regarding the pytest-asyncio dependency. You mention it because we need to have a library to provide the loop fixture, right ?