facilities tutorial developing a monitor

What is a monitor

A monitor is a non-GUI class which main action would be to do something, periodically and notify any possible "listeners".

Monitors anatomy

The base monitor class looks like this:

class Monitor():
    def __init__(self, **kwargs):
        # Where to store any data from this monitor
        self.workingDir = kwargs['workingDir']
        self.samplingInterval = kwargs.get('samplingInterval', None)
        self.monitorTime = kwargs.get('monitorTime', None)

        self._notifiers = []

        if kwargs.get('email', None) is not None:
            self._notifiers.append(kwargs['email'])

        if 'stdout' in kwargs:
            self._notifiers.append(PrintNotifier())

    def notify(self, title, message):
        for n in self._notifiers:
            if n: 
                n.notify(title, message)

    def info(self, message):
        self.notify("INFO", message)

    def initLoop(self):
        """ To be defined in subclasses. """
        pass

    def loop(self):
        self.initLoop()
        timeout = time.time() + 60. * self.monitorTime   # interval minutes from now

        while True:
            finished = self.step()
            if (time.time() > timeout) or finished:
                break
            time.sleep(self.samplingInterval)

    def step(self):
        """ To be defined in subclasses. """
        pass

    def addNotifier(self, notifier):
        self._notifiers.append(notifier)

The most important method of a monitor is

    def step(self):
        """ To be defined in subclasses. """
        pass

In this base class it is not implemented, but if you want to develop a monitor, better implement this method. The step() method will be called from the loop() in a regular manner until a self.monitorTime is reached or it is intentionally stoped.

Optionally, you can code and initLoop() method that will be called before the loops start, suitable for some initializations.

Additionally, monitors follow and observer pattern where any interested "listener/observer" could register and receive notifications from the monitor.

Developing a monitor to observe free space left in the HD where the project is placed.

Let's start coding something. Let's create a python file in our tutorial package to have and create there our new monitor class.

First iteration

1. Add a mymonitor.py file to pyworkflow/em/packages/myfacility/
2. Import monitor base class, and define your new class based on base monitor

from  pyworkflow.em.protocol.monitors import Monitor


class SpaceMonitor(Monitor):
    """
    Monitor to monitor free space on the HD where scipion project is placed
    """

3. Implement the step method. This method should find the HD where the project is placed.

Import some modules at the top of the file (import section)

import os
import collections

Add the following to the step method:

    def step(self):
        """ Using the workingdir attribute has to find the HD and then get the
        available free space."""

        usage = disk_usage(self.workingDir)

        print(usage)



# Taken from http://code.activestate.com/recipes/577972-disk-usage/
def disk_usage(path):
    _ntuple_diskusage = collections.namedtuple('usage', 'total used free')

    st = os.statvfs(path)
    free = st.f_bavail * st.f_frsize
    total = st.f_blocks * st.f_frsize
    used = (st.f_blocks - st.f_bfree) * st.f_frsize
    return _ntuple_diskusage(total, used, free)

This is adding a new method disk_usage which receives a path and uses it in the step method, printing it (temporarily).

Testing first iteration

We are going now to add some test to test our progress.

1. Add a tests folder to the test folder of the package (myfacility/tests)
2. Add an empty __init__.py to make it a module
3. Add test_monitor.py to the tests folder
4. Add the code below

from pyworkflow.tests import *
from pyworkflow.em.packages.myfacility import SpaceMonitor

# Test monitor functionality
class TestMonitor(BaseTest):

    def test_monitor(self):

        # Instantiate the monitor
        spaceMonitor = SpaceMonitor(workingDir=os.getcwd())

        spaceMonitor.step()