Programs

Main CLI

These are the core programs of BPReveal. Each one takes a JSON configuration file.

interpretFlat: Generates shap scores of the same type as BPNet. Hypothetical contributions for each base are written to a modisco-compatible h5.
interpretPisa: Runs an all-to-all shap analysis on the given bed regions or fasta sequences.
makePredictions: Takes a trained model (solo, combined, residual, or even transformation models work) and predicts over the given regions.
motifScan: Scan the genome for patterns of contribution scores that match motifs identified by modiscolite.
motifSeqletCutoffs: Loads the output from modiscolite and calculates cutoff values to use during motif scanning.
prepareBed: Given a set of regions and data tracks, reject regions that have too few (or too many) reads, or that have unmapped bases in the genome.
prepareTrainingData: Takes in bed and bigwig files and a genome, and generates an hdf5-format file containing the samples used for training.
trainCombinedModel: Takes a transformation model and experimental data and builds a model to explain the residuals. Saves both a combined model and the residual model alone to disk.
trainSoloModel: Takes in a training input configuration and trains up a model to predict the given data, with no bias correction. Saves the model to disk, along with information from the training phase.
trainTransformationModel: Takes in a bias (i.e., solo) model and the actual experimental (i.e., biology + bias) data. Derives a relation to best fit the bias profile onto the experimental data. Saves a new model to disk, adding a simple layer or two to do the regression.

Utility CLI

These are little tools and utilities that help in dealing with models. These take arguments on the command line.

lengthCalc: Given the parameters of a network, like input filter width, number of layers &c., determine the input width or output width.
makeLossPlots: Once you’ve trained a model, you can run this on the history file to get plots of all of the components of the loss.
metrics: Calculates a suite of metrics about how good a model’s predictions are.
motifAddQuantiles: Takes the output from motifScan and adds quantile information for determining how good your motif matches were.
predictToBigwig: Takes the hdf5 file generated by the predict step and converts one track from it into a bigwig file.
shapToBigwig: Converts a shap hdf5 file (from interpretFlat) into a bigwig track for visualization.
shapToNumpy: Takes the interpretations from interpretFlat and converts them to numpy arrays that can be read in by modiscolite.
checkJson: Take a json file and make sure that it’s valid input for one of the BPReveal programs. Can also be used to identify which BPReveal program a json belongs to.
showTrainingProgress: Read in the log files generated by the training programs (when verbosity is INFO or DEBUG and show you how well the model’s doing in real time.
showModel: (DEPRECATED, will be removed in 6.0.0) Make a pretty picture of your model.

API

These are Python libraries that do most of the heavy lifting, and can be imported to do useful things in your code.

gaOptimize: contains tools for evolving sequences that lead to desired profiles. It implements a genetic algorithm that supports insertions and deletions.
utils: Contains general-use utilities and a high-performance tool to generate predictions for many sequences.
bedUtils: Useful functions for manipulating bed files, particularly for tiling the genome with regions.
motifUtils: Functions for dealing with motif scanning and modisco files.
logUtils: Functions used to log information. It’s basically TensorFlow’s wrapper around the logging module in the standard library.
interpretUtils: Functions for getting interpretation scores. Contains a streaming system for calculating pisa and flat importance scores.
schema: A set of JSON schemas that validate the inputs to the BPReveal programs. These are used to make sure that incorrect inputs trigger errors early, and that those errors are clearer to the user.
training: A very simple module that actually runs the training loop for trainSoloModel, trainTransformationModel, and trainCombinedModel.
jaccard: Contains wrappers around C functions that calculate the sliding Jaccard similarity used to scan for motifs.
ushuffle: A wrapper around the ushuffle library, used to perform shuffles of sequences that preserve k-mer distributions.

Tools

These are miscellaneous programs that are not part of BPReveal proper, but that I have found useful. They are not actively maintained, and tend to have subpar documentation.