- Dec 30, 2013: Initial release
Download report_progress, use
% python report_progress
chmod +x report_progress.
Note: Python module matplotlib is required to execute this script.
variant tools records the progress of the execution of pipelines in log files. This script parses the log files, extract the starting end ending time of the execution of each step of the pipeline, and plots them in a resulting pdf file.
% report_progress -h
usage: report_progress [-h] [--output OUTPUT] logfiles [logfiles ...] This script reads one or more local or remote log files produces by variant tools, extract starting times of steps of pipelines, and generate graphical reports positional arguments: logfiles One or more log files. If address:path is specified, log files will be copied to a temporary directory using command scp. No password can be specified so public key authentication is needed if a password is needed. optional arguments: -h, --help show this help message and exit --output OUTPUT Name of the output file (in pdf format). progress.pdf will be used if no file is specified.
This command accept a list of log files, which can be local or remote. In the latter case, command
scp will be used to copy the files to a local temporary directory.
% python report_progress /path/to/*.log % python report_progress 'username@server:path/to/*bam.log'
Without specifying the
--output parameter, the figures will be saved in a PDF file named
progress.pdf, with figures similar to
- Green lines represent running times of successful steps.
- Red lines represent running times of failed steps.
- Purple lines represent on-going steps, which is the last started job in the log file. Because a job might be killed during the last step, the running time might not be accurate.
- Duration for the execution of successful steps are reported (for non-trivial steps that lasts more than 2 minutes).