There are a great number of LillyMol command line tools. Over the years some have proven more useful than others, but all have played useful roles on one or more projects. Here is an alphabetic listing of tools with a very brief description. Some of the tools have their own documentation.
Some tools are seldom used, and/or have been superseded by others. So you will see things that seem duplicative. Work is ongoing to clean deprecate obsolete tools and replace with their more modern counterparts.
Crude 3D alignment tool. Uses a substructure query to identify the atoms that will be placed at the origin, along the X axis and on the Y axis. Mostly used to aid visualisation.
Fingerprint generator. Logical extension of the idea of atom pairs.
Find the average of a column of numeric values, but has a lot of flexibility - including weighted averages.
average -d w_natoms -d w_rotbond rand.w
Convert continuous values into discrete ranges. Note that the column(s) being processed are replaced by their bucketised values
bucketise -v -b 10 -j -P xlogp rand.w
Builds A BerkeleyDB database with the smiles, or smiles variants as the key, and the name as the value. See in_database.
Obscure. For a large file, generate the byte offsets where various fractions of the file start. Useful for having multiple processes process a large file without splitting it.
Generates a fingerprint based on chirality. Initial results are disappointing.
Infrastructure. Maintain mappings between text and numeric class labels.
Identify groups of structures that are identical under various smiles assumptions. common_names.
Join descriptor files. Any number of files can be processed and there is no assumption about sorting.
Enforce a correlation structure between columns of data. Unrelated to Cheminformatics, was developed to support Clinical Trial simulations, where for example, we might have a distribution of patient weights and patient heights, and we need to enforce a correlation structure beween those values.
Distance betwen features fingerprints. Can do both topological and 3D distances. The default script uses LillyMol pharmacaphoric features and computes bond distances, and/or geometric distances between feastures.
Concatenate descriptor files. Knows about header records, so if the second+ files are in different order, it can properly handle them.
Filter a descriptor file based on values in the columns. Specifically set up to work with descriptor files. Any number of filtering conditions can be used, and it can recognise columns by name.
descriptor_file_filter -e 'w_natoms<40 && w_xlogp<6' -v /rand.w
Select a subset of the rows from a descriptor file.
Sorts descriptor files - helpful because it knows about header records and column names.
Convert numeric value(s) in a descriptor file to fingerprints. The input file must consist entirely of positive numbers. If a binary fingerprint is being produced, all non zero values set the bit. If a sparse counted fingerprint is being generated, counts are used.
Convert a descriptor file to svm-lite form. Needs to be combined with an activity file when writing a training set. When writing a test set, it checks that the features present are consistent with what was present when the training set was processed.
Useful for debugging - columnar data can be hard to see outside a proper data analysis environment.
Complex. Converts a descriptor file to fingerprint form.
Very useful but very complex molecular fragmentation. dicer
Obscure. Sort a file based on the difference between two columns. Mostly useful for dealing with observed vs predicted situations.
For a given feature, generate a distribution of the values encountered.
distribution -d w_natoms rand.w
note similarities with bucketise.
A new generation ec fingerprint generator. Mostly works but needs more development.
Read a substructure query and write it. Mostly useful for debugging, but can also be used for converting old style query files to new textproto forms.
Read a reaction file and write it. Mostly useful for debugging, but can also be used for converting old style reaction files to new textproto forms.
Special purpose BerkeleyDB tool for reading Lilly inventory data.
Tool for computing correlations under conditions or quite large data, where reading into R/python might be prohibitive.
Robust atom pair fingerprint generator.
Generate molecular descriptors based on E/Z bond specifications. Generally not used, because the reliability/presence of such information can be low.
Fingerprint based on E/Z bond specifications. See previous comment.
Newer generation of E/Z fingerprinting. Still unsatisfactory. E/Z data is not robustly handled in LillyMol.
Infrastructure. Scales and unscales numeric data - for use in models.
Extremely useful utilities for fetching a subset of molecules from an .sdf
file. The difference is that fetch_sdf retrieves the records in the same
order as they appear in the file containing the list of items requested,
whereas fetch_sdf_quick
returns molecules in the order they appear in the structure file.
Fetching utilities for smiles. By combining with the -X and -Y options
these become generally useful differencing tools.
Fetching utilities for TDT files.
Generally useful structure processing tool fileconv
Fingerprint just a subset of the atoms in a molecule. The subset is defined by a substructure query, and an optional radius. Useful for identifying unique reagents - to a given radius. fingerprint_substructure
Re-write a smiles with a given atom as the first atom in the smiles.
Part of the zof.sh fragment filtering script.
For a given set of matched atoms, extract the coordinates from molecules.
For a core defined by a substructure, identify the substituents attached. get_substituents
Obscure. Adds information from a descriptor file to a fingerprint file.
Filter one set of fingerprints according to their distance to another set. Hard to use, but useful in certain circumstances.
Generate a distance matrix from a set of fingerprints.
Generate a binary distance matrix file from fingerprints.
Fingerprints from Erg reduced graph representations. Only used within gfp_make where Erg fingerprints sometimes do well in models.
Reduce all counted fingerprints to max count 1.
Generate histogram of distances in a collection.
Report extra diversity added as new molecules are added to a set. Useful for tracking the progress of a discovery/optimisation program.
Leader, or sphere exclusion. Extremely useful tools that implement sphere exclusion. This is generally the preferred method for dealing with the problem of having a set or ranked molecules from which a small subset must be selected. Sort by desirability and run leader. gfp_leader
Tool designed for comparing a small set of molecules, needles, against a larger set, haystack.
gfp_lnearneighbours_standard -p needles.gfp -n 100 haystack.gfp.gz > needles.nn
nn2csv needles.nn > needles.nn.csv
An attempt at using the gfp framework for an approximate MCS solution. Needs work, preferred means of tackling the approximate MCS problem is with dicer.
An often powerful modeling method. Make sure that more sophisticated models actually perform better than this.
Generally use gfp_lnearneighbours instead. This version loads the haystack into memory and then reads the needles one at a time. Turns out this is the slow way of doing things. gfp_nearneighbours.
Nearest neighbours within a single file - no needles/haystack distinction.
For given pairs of molecules, return the distances. Useful if you only need information about specific pairs from a larger set, although you may be better off extracting the fingerprints of interest from the larger set first. Or extract the subset of interest and run that alone.
Extremely useful tool for examining the tendency of bits to differntially appear across classes, or with different activity values.
A simple clustering tool.
Convert sparse fingerprints to fixed form.
Max/min picker. Useful for maximum diversity selections. gfp_spread
A bucket constrained version of spread. Molecules are assigned to buckets and as molecules are selected, selections are forced to uniformly sample from the buckets. gfp_spread
Can do similarity calculations using smiles. Computes only the default set of fingerprints internally.
Infrastructure. Used for scoring svmfp models.
Convert gfp fingerprint forms to descriptor file format. Always
use gfp_to_descriptors_multiple. gfp_to_descriptors_multiple.
Convert gfp to svm-lite format.
Implements the Ghose-Crippen fragment additivity models.
Identifies large likely hydrophobic sections in molecules.
Look for text in certain column(s) only.
Compute a fingerprint of how a molecule interacts with a fixed grid.
Compute a tanomito overlap between two molecular grids.
Descriptor generator for sections of molecules likely to be hydrophobic.
Can identify and enumerate chiral centres. Also useful for identifying and unmarked and invalid chirality.
An important tool for maintaining a database where the key is a unique smiles. Databases are built with buildsmidb_bdb. in_database
Special purpose tools, only relevant inside Lilly.
Early implementation of Isolation Forest. The version in sklearn is better today.
Command line utilities for BerkeleyDB databases
A version of cut. Understands descriptor files, and writes columns in the order specified iwcut
Part of Lilly Medchem Rules.
Descriptor generator. Generates 250+ mostly interpretable molecular descriptors. iwdescr
Generates EC fingerprints, the -EC to gfp_make.
The synethetic feasibility databases [synthetic feasibility(Molecule_Tools/synthetic_precedent.md).
An innovative approach to measuring the interaction of a ligand with a protein.
Generates linear fingerprints -FPIW
Generates T-shaped paths -PATHD. Needs work.
Split utility. Now Linux split is almost as good, but the naming is better here and it knows about descriptor files.
Generate statistics for the problem of model prediction - regression only.
Compare descriptor files.
Infrastructure. Some tools need to process features that do not cross zero.
CATS pharmacaphore fetures, also fingerprint generator -CATS
Descriptor generator.
Descriptor generator.
Descriptor generator.
Descriptor generator.
Descriptor generator. Molecular connectivity indices.
Descriptor generator.
Descriptor generator. Ideas from publications from Peter Jurs publications.
Identify and remove correlated features
New version linear fingerprint generator. Still needs work.
Given linkers, two connected fragments, generated by dicer or other tools, replace interiour parts of a molecule linker_replacement.
Identify molecules that have a long max-bond_separator compared to the other dimesions.
Descriptor and fingerprint generator.
A descriptor generator make_descriptors.
Given a combinatorial library scheme, make specific molecules from among those possible.
Convert Marvin fingerprint output to gfp form.
Post process output from Marvin pKa calculations.
Implementation of Abbot paper. The shell wrapper in contrib/bin is set up to read the reactions from the paper.
Across a set of train/test splits, examine the predicted files and identify those molecules that are always predicted poorly.
Convert a multi-fragment molecule to separate molecules.
Given multiple files containing model predictions, produce an averge prediction.
Convert a molecule to a substructure query file. A great deal of control is available control how atoms are translated mol2qry
Generate various molecular abstractions molecular abstraction
Generate a grid around one or more molecules molecular_grid.
Concatentate a file of molecules into one molecule - mostly useful for testing.
Convert the molecule to scaffold form - molecular_abstraction also does this.
Apply multiple reactions to a molecule.
Useful for identifying smaller pharmacaphore equivalents. Driven by external reaction files that specify transformations.
Extracts subsets of the atoms in a molecule based on a query.
Sorts files of molecules. Can be very efficient. Also useful for splitting a set of molecules into non-overlapping sets msort.
Enables cycling through directories in an interactive session. cd $(nextdir)
Convert a nearest neighbour file to csv form. Intended as an easier to use nplotnn.
Jarvis Patric clustering and other capabilities.
Merge multiple nearest neighbour files.
Mergest nearest neighbour files.
Infrastructure. Predictions from nearest neighbours, knn variant.
Simple clustering
Normalise columns in a tabular file.
Identify and remove constant or nearly constnt columns.
Converts nearest neighbour files to smiles form. Very useful, but can be complex.
Write a smiles where the isotopic label is one of several molecular properties. numbered_smiles.
A difference utility where a threshold for numeric difference can be specified.
A fragment based model that allows atoms to be matched by multiple fragments.
Multiple individual nearest neighbour searches have been run, and those files
need to be made into a form suitable for the -A option of gfp_spread.
Can generate queries for pharmacaphore type searches in 2D. pharmacophore_2d.
Generates statistics for predictions for a classification model.
Several smiles variants.
Toplogical polar surface area
Pubchem Fingerprints
Command line version for computing QED QED.
Consumes output from atom contribution methods like QupKake and Epok for pKa computations. QupKake.
Given a scaffold and substituent points, identify substituents.
Rand Fingerprint generator - useful for testing
Useful for testing
Permutes molecules in chemically sensible ways random_molecular_permutations.
Randomly select records from a file. Preserves the order of the records in
the file, and scalable - unlink shuf file | head -n 1000.
Generate different smiles for a molecule - useful for testing.
Generate Reduced Graphs.
Remove substituents and label the core atoms.
Remove atoms identified by a query.
Infrastructure. Part of reaction informatics.
Yet another R group utilitiy rgroup. This is the most supported of the R group identification tools.
Infrastructure Ring replacement utility ring_replacement.
Generates a fingerprint based on the rings in a molecule.
Generates a fingerprint based on the size of rings.
Generates a ring substitution fingerprint.
Remove rings and rings that might be part of a fused system.
Remove outliers from tabular data.
Fingerprint for rotatable bonds
Compute and filter rotatable bonds rotatable_bonds.
Rule of five implementation
Running Average
Infrastructure. Reaction informatics
Substructure search reagents or products. Unsophisticated...
Using the SAFE smiles representation, generate de-novo molecules SAFE.
Generates exhaustive combinations of the rings that comprise a molecular scaffold scaffolds.
Knows how to shuffle some multi-record entities.
Reads a pool of molecules and randomly permutes the sidechains between them. sidechain_switcheroo.
Infrastructure: builds a rings database smi2rings.
String mutations on smiles. Mostly generates bad smiles, but sometimes interesting. More efficient to use random_molecular_permutations.
Identify and filter on sp3 character. Can now do the same within iwdescr.
Compute Spearman rank for tabular data.
Infrastructure. Used when forming train/test splits in svmfp_calibrate.
Implements CREM-like functionality substituent_identification.
Identify the fraction of a molecule implied by matched atoms.
Geometric superimposition based on matched query atoms.
Infrastructure. Useful for examining the relationship between model error and a training set. Now available in gfp_svmfp_score
Infrastructure. Convert from svm-lite form to gfp.
Not very good tautomer generation. Problem is too hard, do not use. Use the chemical standardisation functionality instead.
Counts lines and columns. The quickest way to ensure that your tabular file is actually tabular.
Utilities for dealing with TDT files - those generated by gfp_make and consumed by the various gfp_* tools.
Infrastructure: computes the MPR fingerprint component
Infrastructure. Used with svmfp_calibrate
Implements an idea around dependent fragments - only try one if another has already matched.
Related to Topological Torsion calculations.
Parts of Lilly Medchem Rules
Reaction enumeration trxn
Descriptor generation - based on 3D structure and projections onto the axes.
Tester for smiles canonicalisation. Will fail, but failures are interesting to observe.
Tester for chemical standardisation. Run across a collection like Chembl, expect to see a couple of hundred failures. This is an extremely difficult problem.
Substructure searching.
Geneate summary reports from tsubstructure.
Tester for symmetry, obsolete.
Scan a file looking for unbalanced quotes - spans records
Discard duplicate molecules unique_molecules
Discard duplicate items in a given column of a tabular file.
Infrastructure. Verloop descriptors.
In a descriptor file, identify missing values.
Implementation of XLogp logP estimator.
Compares two structures via their coordintes.