How to run native asyncio tasks

Use the asyncio runner when task functions and their asynchronous subtasks are defined with async def. The runner executes them on the caller’s event loop and limits the number of hosts in flight with num_workers.

Select the runner

The configuration selects the runner by its asyncio entry-point name. Paths in this example are relative to the notebook working directory, docs/howto/.

import asyncio
import functools

from nornir import InitNornir
from nornir.core.exceptions import (
    AsyncTaskOnSyncRunError,
    RunnerNotAsyncError,
    RunnerNotSyncError,
    SyncTaskOnAsyncRunError,
)
from nornir.core.task import MultiResult, Result, Task
from nornir.plugins.runners import AsyncioRunner, SerialRunner

nr = InitNornir(config_file="asyncio_runner/config.yaml")
print(f"Configured runner: {type(nr.runner).__name__}")
print(f"Configured workers: {nr.runner.num_workers}")
print(f"Inventory hosts: {', '.join(nr.inventory.hosts)}")

Configured runner: AsyncioRunner
Configured workers: 2
Inventory hosts: edge-1, edge-2

with_runner returns a Nornir copy, so an application can select or tune the runner in Python without changing the configuration.

single_worker_nr = nr.with_runner(AsyncioRunner(num_workers=1))
print(f"Programmatic runner: {type(single_worker_nr.runner).__name__}")
print(f"Programmatic workers: {single_worker_nr.runner.num_workers}")

Programmatic runner: AsyncioRunner
Programmatic workers: 1

Run async tasks and subtasks

Await asynchronous subtasks with task.arun. A partial or callable object does not necessarily have __name__, so pass name explicitly to give its result and any diagnostic a useful name.

def sync_child(task: Task) -> Result:
    return Result(host=task.host, result="sync child ran inline")


async def async_child(task: Task) -> Result:
    await asyncio.sleep(0)
    task.run(sync_child)
    return Result(host=task.host, result="async child completed")


async def labeled_child(task: Task, label: str) -> Result:
    await asyncio.sleep(0)
    return Result(host=task.host, result=label)


class AsyncCallable:
    async def __call__(self, task: Task) -> Result:
        await asyncio.sleep(0)
        return Result(host=task.host, result="callable object completed")


async def async_workflow(task: Task) -> Result:
    await task.arun(async_child)
    partial = functools.partial(labeled_child, label="partial completed")
    await task.arun(partial, name="explicit partial")
    await task.arun(AsyncCallable(), name="explicit callable")
    return Result(host=task.host, result="workflow completed")

Use await nr.arun(...) for top-level dispatch. async with nr performs asynchronous connection cleanup for both good and failed hosts when the block exits. This example opens no connections, but follows the same lifecycle an application with connections should use.

async with nr:
    results = await nr.arun(task=async_workflow)


def print_result_tree(results: MultiResult, indent: str = "  ") -> None:
    for result in results:
        if isinstance(result, MultiResult):
            print(f"{indent}{result.name}:")
            print_result_tree(result, indent + "  ")
        else:
            print(f"{indent}{result.name}: {result.result}")


for host_name, host_results in results.items():
    print(f"{host_name}:")
    print_result_tree(host_results)

edge-1:
  async_workflow: workflow completed
  async_child:
    async_child: async child completed
    sync_child: sync child ran inline
  explicit partial: partial completed
  explicit callable: callable object completed
edge-2:
  async_workflow: workflow completed
  async_child:
    async_child: async child completed
    sync_child: sync child ran inline
  explicit partial: partial completed
  explicit callable: callable object completed

task.run(sync_child) above remains supported inside an async task and executes the synchronous child inline. Because inline synchronous children run on the caller’s event-loop thread, they must return promptly; blocking work there stalls every coroutine on that loop.

Processor hooks are also synchronous and run on the caller’s event-loop thread. Existing processors continue to work, but a hook that performs blocking I/O or long-running computation blocks async host tasks. Keep hooks non-blocking or arrange for the processor itself to hand work off appropriately.

Diagnose execution-mode mismatches

Nornir rejects task and runner combinations that would otherwise create an un-awaited coroutine or invoke an unsupported runner method. These are named exceptions, and the following messages are produced by the actual calls.

def synchronous_task(task: Task) -> Result:
    return Result(host=task.host, result="synchronous")


async def show_mismatch_errors() -> None:
    checks = []

    try:
        await nr.arun(synchronous_task)
    except SyncTaskOnAsyncRunError as error:
        checks.append(error)

    try:
        nr.run(async_workflow)
    except AsyncTaskOnSyncRunError as error:
        checks.append(error)

    try:
        nr.run(synchronous_task)
    except RunnerNotSyncError as error:
        checks.append(error)

    serial_nr = nr.with_runner(SerialRunner())
    try:
        await serial_nr.arun(async_workflow)
    except RunnerNotAsyncError as error:
        checks.append(error)

    for error in checks:
        print(f"{type(error).__name__}: {error}")


await show_mismatch_errors()

SyncTaskOnAsyncRunError: Task 'synchronous_task' is synchronous and cannot run asynchronously. Define it with 'async def' or use a synchronous runner. See https://github.com/nornir-automation/nornir/issues/1120.
AsyncTaskOnSyncRunError: Task 'async_workflow' is asynchronous and cannot run synchronously. Use 'await nr.arun(...)' or 'await task.arun(...)', or define a synchronous task.
RunnerNotSyncError: Runner 'AsyncioRunner' does not support synchronous operations. Use 'await nr.arun(...)', 'await nr.aclose_connections()', or 'async with nr'.
RunnerNotAsyncError: Runner 'SerialRunner' does not support asynchronous operations. Use 'nr.run(...)', 'nr.close_connections()', or 'with nr', or select an asyncio runner.