HOWTO - Logging

As the pipelines and tools with the MuG VRE environment run without the terminal returning to the user, it is important to have a way to communicate to the user that there is an error with the pipeline. As the code is run within a cluster, the text that is printed to screen won’t be returned to the user. Within the Tool API a logging interface has been implemented.

Levels of Logging

When there is an issue it can be passed back to the VRE. These are tracked and passed back to the VRE as the application finishes. There is the option to raise errors in 1 of 6 states:

  • INFO
    Confirmation that Tool execution is working as expected.
  • DEBUG
    Detailed information, typically of interest only when diagnosing problems.
  • WARNING
    An indication that something unexpected happened, but that the Tool can continue working successfully.
  • ERROR
    A more serious problem has occurred, and the Tool will not be able to perform some function.
  • FATAL
    A serious error, indicating that the Tool may be unable to continue running.
  • PROGRESS
    Provide the VRE with information about Tool execution progress, in the form of a percentage (0-100)

Using Logging

The code is present within the Tools API, so adding it into a tool or pipeline requires minimal effort. Improting the loggin functions requires the following code:

1
from utils import logger

To add elements to the log can be implemented by:

1
logger.info("Processing Text")

This logging has been implemented within the mg-process-test repository within the process_test.py and within the testTool.py scripts. There is no logging within the @task as from this it is possible to return an actual object that can then be checked by the run() function to determine the correct error to return to the main pipeline.