Processors

Processors are plugins that can execute code on certain events. For more information on those events check the class documentation.

You can see processors as an alternative way of dealing with the results of a task, however, it has a few advantages:

  1. Due to its event-based nature, you can process the events asynchronously. This means that you will be processing the result of a host exactly once the host is completed without the need to wait for the rest of the hosts to complete.

  2. Because you are tapping into events code is more concise and easier to understand.

Let’s see how processors work with a few examples. Let’s start by loading the nornir object and some libraries we will need:

[1]:
from typing import Dict

from nornir import InitNornir

nr = InitNornir(config_file="config.yaml")

Now let’s write a processor that will print on screen some information about the execution of the task:

[2]:
# note that these imports are only needed if you are annotating your code with types
from typing import Dict

from nornir.core import Nornir
from nornir.core.inventory import Host
from nornir.core.task import AggregatedResult, MultiResult, Result, Task

class PrintResult:
    def task_started(self, task: Task) -> None:
        print(f">>> starting: {task.name}")

    def task_completed(self, task: Task, result: AggregatedResult) -> None:
        print(f">>> completed: {task.name}")

    def task_instance_started(self, task: Task, host: Host) -> None:
        pass

    def task_instance_completed(
        self, task: Task, host: Host, result: MultiResult
    ) -> None:
        print(f"  - {host.name}: - {result.result}")

    def subtask_instance_started(self, task: Task, host: Host) -> None:
        pass  # to keep example short and sweet we ignore subtasks

    def subtask_instance_completed(
        self, task: Task, host: Host, result: MultiResult
    ) -> None:
        pass  # to keep example short and sweet we ignore subtasks

Now we are going to write another processor that will save some information about the task in a dictionary:

[3]:
class SaveResultToDict:
    def __init__(self, data: Dict[str, None]) -> None:
        self.data = data

    def task_started(self, task: Task) -> None:
        self.data[task.name] = {}
        self.data[task.name]["started"] = True

    def task_completed(self, task: Task, result: AggregatedResult) -> None:
        self.data[task.name]["completed"] = True

    def task_instance_started(self, task: Task, host: Host) -> None:
        self.data[task.name][host.name] = {"started": True}

    def task_instance_completed(
        self, task: Task, host: Host, result: MultiResult
    ) -> None:
        self.data[task.name][host.name] = {
            "completed": True,
            "result": result.result,
        }

    def subtask_instance_started(self, task: Task, host: Host) -> None:
        pass  # to keep example short and sweet we ignore subtasks

    def subtask_instance_completed(
        self, task: Task, host: Host, result: MultiResult
    ) -> None:
        pass  # to keep example short and sweet we ignore subtasks

Finally, to test the processors we are going to use a very simple task that will just greet us on behalf of each device:

[4]:
def greeter(task: Task, greet: str) -> Result:
    return Result(host=task.host, result=f"{greet}! my name is {task.host.name}")

Hopefully everything is clear so far, let’s now put it to use:

[5]:
# NBVAL_IGNORE_OUTPUT

data = {}  # this is the dictionary where SaveResultToDict will store the information

# similary to .filter, with_processors returns a copy of the nornir object but with
# the processors assigned to it. Let's now use the method to assign both processors
nr_with_processors = nr.with_processors([SaveResultToDict(data), PrintResult()])

# now we can use nr_with_processors to execute our greeter task
nr_with_processors.run(
    name="hi!",
    task=greeter,
    greet="hi",
)
nr_with_processors.run(
    name="bye!",
    task=greeter,
    greet="bye",
)
>>> starting: hi!
  - host1.cmh: - hi! my name is host1.cmh
  - host2.cmh: - hi! my name is host2.cmh
  - spine00.cmh: - hi! my name is spine00.cmh
  - spine01.cmh: - hi! my name is spine01.cmh
  - leaf00.cmh: - hi! my name is leaf00.cmh
  - leaf01.cmh: - hi! my name is leaf01.cmh
  - host1.bma: - hi! my name is host1.bma
  - host2.bma: - hi! my name is host2.bma
  - spine00.bma: - hi! my name is spine00.bma
  - spine01.bma: - hi! my name is spine01.bma
  - leaf00.bma: - hi! my name is leaf00.bma  - leaf01.bma: - hi! my name is leaf01.bma

>>> completed: hi!
>>> starting: bye!
  - host1.cmh: - bye! my name is host1.cmh
  - host2.cmh: - bye! my name is host2.cmh
  - spine00.cmh: - bye! my name is spine00.cmh
  - spine01.cmh: - bye! my name is spine01.cmh
  - leaf00.cmh: - bye! my name is leaf00.cmh
  - leaf01.cmh: - bye! my name is leaf01.cmh
  - host1.bma: - bye! my name is host1.bma
  - host2.bma: - bye! my name is host2.bma
  - spine00.bma: - bye! my name is spine00.bma
  - spine01.bma: - bye! my name is spine01.bma
  - leaf00.bma: - bye! my name is leaf00.bma  - leaf01.bma: - bye! my name is leaf01.bma

>>> completed: bye!
[5]:
AggregatedResult (bye!): {'host1.cmh': MultiResult: [Result: "bye!"], 'host2.cmh': MultiResult: [Result: "bye!"], 'spine00.cmh': MultiResult: [Result: "bye!"], 'spine01.cmh': MultiResult: [Result: "bye!"], 'leaf00.cmh': MultiResult: [Result: "bye!"], 'leaf01.cmh': MultiResult: [Result: "bye!"], 'host1.bma': MultiResult: [Result: "bye!"], 'host2.bma': MultiResult: [Result: "bye!"], 'spine00.bma': MultiResult: [Result: "bye!"], 'spine01.bma': MultiResult: [Result: "bye!"], 'leaf00.bma': MultiResult: [Result: "bye!"], 'leaf01.bma': MultiResult: [Result: "bye!"]}

The first thing you probably noticed is that we got all those messages on screen printed for us. That was done by our processor PrintResult. You probably also noticed we got the AggregatedResult back but we didn’t even bother saving it into a variable as we don’t needed it here.

Now, let’s see if SaveResultToDict did something to the dictionary data:

[6]:
import json
print(json.dumps(data, indent=4))
{
    "hi!": {
        "started": true,
        "host1.cmh": {
            "completed": true,
            "result": "hi! my name is host1.cmh"
        },
        "host2.cmh": {
            "completed": true,
            "result": "hi! my name is host2.cmh"
        },
        "spine00.cmh": {
            "completed": true,
            "result": "hi! my name is spine00.cmh"
        },
        "spine01.cmh": {
            "completed": true,
            "result": "hi! my name is spine01.cmh"
        },
        "leaf00.cmh": {
            "completed": true,
            "result": "hi! my name is leaf00.cmh"
        },
        "leaf01.cmh": {
            "completed": true,
            "result": "hi! my name is leaf01.cmh"
        },
        "host1.bma": {
            "completed": true,
            "result": "hi! my name is host1.bma"
        },
        "host2.bma": {
            "completed": true,
            "result": "hi! my name is host2.bma"
        },
        "spine00.bma": {
            "completed": true,
            "result": "hi! my name is spine00.bma"
        },
        "spine01.bma": {
            "completed": true,
            "result": "hi! my name is spine01.bma"
        },
        "leaf00.bma": {
            "completed": true,
            "result": "hi! my name is leaf00.bma"
        },
        "leaf01.bma": {
            "completed": true,
            "result": "hi! my name is leaf01.bma"
        },
        "completed": true
    },
    "bye!": {
        "started": true,
        "host1.cmh": {
            "completed": true,
            "result": "bye! my name is host1.cmh"
        },
        "host2.cmh": {
            "completed": true,
            "result": "bye! my name is host2.cmh"
        },
        "spine00.cmh": {
            "completed": true,
            "result": "bye! my name is spine00.cmh"
        },
        "spine01.cmh": {
            "completed": true,
            "result": "bye! my name is spine01.cmh"
        },
        "leaf00.cmh": {
            "completed": true,
            "result": "bye! my name is leaf00.cmh"
        },
        "leaf01.cmh": {
            "completed": true,
            "result": "bye! my name is leaf01.cmh"
        },
        "host1.bma": {
            "completed": true,
            "result": "bye! my name is host1.bma"
        },
        "host2.bma": {
            "completed": true,
            "result": "bye! my name is host2.bma"
        },
        "spine00.bma": {
            "completed": true,
            "result": "bye! my name is spine00.bma"
        },
        "spine01.bma": {
            "completed": true,
            "result": "bye! my name is spine01.bma"
        },
        "leaf00.bma": {
            "completed": true,
            "result": "bye! my name is leaf00.bma"
        },
        "leaf01.bma": {
            "completed": true,
            "result": "bye! my name is leaf01.bma"
        },
        "completed": true
    }
}

As you can see, performing various actions on the results becomes quite easy thanks to the processors. You still get the result back but thanks to this plugins you may not needed them anymore.

Ideas

What other things could be done with processors?

  1. Send events to slack/IRC/logging_system

  2. Keep the user informed of what’s going on without having them to wait for the completion of all the hosts (particularly interesting if you have lots of devices)

  3. Page someone/raise an alert if a given task fails

  4. etc…