- Development Checklist
- HOWTO - Tools
- HOWTO - Pipelines
- HOWTO - Documentation
- HOWTO - Licensing
- HOWTO - Configuration Files
- HOWTO - Logging
- HOWTO - Testing Your Code
The following is a walk through of developing a tool and pipeline wrapper to include new functionality within the MuG VRE. There are several stages covering the Tool development, using the tool within a pipeline and defining the configuration files required so that the final product can be smoothly integrated into the MuG VRE.
Common Coding Standards¶
When it comes to developing the code all the code should stick to a common standard. This has been defined within the Coding Standards documentation as well as how to set up the licenses correctly so that your package can be integrated.
Adding a new function¶
All of the examples in the following sections describe code that has been incorporated into a functional pipeline and tool within a demonstration VRE Tool that is ready for deployment within the VRE. The code can be found in GitHub repository mg-process-test.
In the test process there are example pipelines, tools, documentation, setup scripts unit tests and config files. This repository can be forked and used as the base for developing new pipelines and tools.
The following documents will help guide with the creation of all the components required for creating a tool ready to be integrated into the VRE. To help with the development and Development Checklist has been created to provide a generic guide and checklist to help make sure that nothing has been forgotten.
- Wrapping a Tool
- This section guides you through how to wrap an external tool, or create a tool that utilises the pyCOMPSs framework and should be capable of running within the MuG VRE environment.
- Creating a Pipeline
- Once you have created a tool you can now incorporate one or multiple tools into a pipeline. This will handle the passing of variables from the VRE to the tool and the tracking of outputs ready for handing back to the VRE. This document will also help in creating test input metadata and file location JSON files that are required to run the pipeline.
- This provides a overview of the documentation requirements as described by the MuG Coding Standards.
- Takes you through adding logging to your pipelines and tools to return messages to the user via the MuG VRE.
- Testing Your Code
- A important part of making sure that a pipeline or tool is ready for integration is ensuring that the code has been tested. This covers testing the code is functional and that it is capable of running within the infrastructure used by the VRE.
- Licensung Your Code
- The Apache 2.0 license is required for pipelines and tools to to be integrated into the MuG VRE.
- VRE Configuration
- This takes you through creating JSON configuration files for your tool. This should define all the inputs, outputs and any arguments that are required by the pipelines and tools.
Integrating a new tool into the VRE¶
The next step is the integration of the pipeline/tool into the MuG VRE. The Configuration should provide guide to the initial JSON files. The full JSON specification is located in this GoogleDoc. The details the requirements for correctly creating the Tool description JSON file and the requirements for parameters needed for an application.