Merge branch 'priorityassignment' into 'master'

Resolve "PriorityAssignment contains broken code"

Closes #116

See merge request mass-spectrometry/corems!66

Author: corilo

.gitignore

@@ -3,6 +3,7 @@ test_env/
 venv310/
 venvwsl/
 tmp_data/ 
+tmp_docs/
 .pyc  
 .pyo
 *.hdf5

README.md

@@ -1,6 +1,6 @@
-![CoreMS Logo](docs/CoreMS.COLOR.png)  
+![CoreMS Logo](docs/CoreMS.COLOR_small.png)  
 
-<div align="center">
+<div align="left">
 
 <br>
 <br>
@@ -25,9 +25,9 @@
   - [Installation](#corems-installation)  
   - [Thermo Raw File on Mac and Linux](#thermo-raw-file-access)  
 - Execution:     
-  - [Jupyter Notebook and Docker containers](#molecular-database-and-jupyter-notebook-containers)  
+  - [Jupyter Notebook and Docker containers](#docker-stack)  
   - [Simple Example](#simple-script-example)  
-  - [Python Examples](examples/examples)
+  - [Python Examples](examples/scripts)
   - [Jupyter Notebook Examples](examples/notebooks)  
 
 
@@ -41,21 +41,21 @@
 
 Data handling and software development for modern mass spectrometry (MS) is an interdisciplinary endeavor requiring skills in computational science and a deep understanding of MS. To enable scientific software development to keep pace with fast improvements in MS technology, we have developed a Python software framework named CoreMS. The goal of the framework is to provide a fundamental, high-level basis for working with all mass spectrometry data types, allowing custom workflows for data signal processing, annotation, and curation. The data structures were designed with an intuitive, mass spectrometric hierarchical structure, thus allowing organized and easy access to the data and calculations. Moreover, CoreMS supports direct access for almost all vendors’ data formats, allowing for the centralization and automation of all data processing workflows from the raw signal to data annotation and curation.
 
-- reproducible pipeline
+CoreMS aims to provide 
 - logical mass spectrometric data structure
 - self-containing data and metadata storage
 - modern molecular formulae assignment algorithms
 - dynamic molecular search space database search and generator
 
 ## Current Version
 
-### `2.5.3.beta`
+ `2.5.3.beta`
 
 ## Main Developers/Contact 
 - [Yuri. E. Corilo](mailto:corilo@pnnl.gov)  
 - [William Kew](mailto:william.kew@pnnl.gov)
 
-
+## Data formats
 ### Data input formats
 
 - Bruker Solarix (CompassXtract)
@@ -68,10 +68,7 @@ Data handling and software development for modern mass spectrometry (MS) is an i
 - CoreMS exported processed mass list files(excel, .csv, .txt, pandas dataframe as .pkl)
 - CoreMS self-containing Hierarchical Data Format (.hdf5)
 - Pandas Dataframe
-
-- Support for Could Storage using s3path.S3path  
-    see examples of usage here:
-      - [S3 Support](tests/s3_test.py)
+- Support for cloud Storage using s3path.S3path(see examples of usage here: [S3 Support](tests/s3_test.py))
 
 ### Data output formats
 
@@ -85,54 +82,33 @@ Data handling and software development for modern mass spectrometry (MS) is an i
 
 - LC-MS
 - GC-MS
-- IMS-MS (`TODO`)
-- LC-IMS-MS (`TODO`)
-- Collections (`TODO`)
 - Transient
 - Mass Spectra
 - Mass Spectrum
 - Mass Spectral Peak
 - Molecular Formula
-- Molecular Structure (`TODO`)
+
+### In progress data structures
+- IMS-MS
+- LC-IMS-MS 
+- Collections 
+- Molecular Structure
 
 ---
 ## Available features
 
-### FT-MS Signal Processing
+### FT-MS Signal Processing, Calibration, and Molecular Formula Search and Assignment
 
 - Apodization, Zerofilling, and Magnitude mode FT
 - Manual and automatic noise threshold calculation
 - Peak picking using apex quadratic fitting
 - Experimental resolving power calculation
-
-### GC-MS Signal Processing
-
-- Baseline detection, subtraction, smoothing 
-- m/z based Chromatogram Peak Deconvolution,
-- Manual and automatic noise threshold calculation
-- First and second derivatives peak picking methods
-- Peak Area Calculation
-
-### GC-MS Calibration
-
-- Retention Index Calibration
-
-### GC-MS Compound Identification
-
-- Automatic local (SQLite) or external (MongoDB or PostgreSQL) database check, generation, and search
-- Automatic molecular match algorithm with all spectral similarity methods 
-
-### FT-MS Calibration
-
 - Frequency and m/z domain calibration functions:
 - LedFord equation [ref]
 - Linear equation
 - Quadratic equation
 - Automatic search most abundant **Ox** homologue series
 - Step fit ('walking calibration") based on the LedFord equation [ref]
-
-### FT-MS Molecular formulae search and assignment
-
 - Automatic local (SQLite) or external (PostgreSQL) database check, generation, and search
 - Automatic molecular formulae assignments algorithm for ESI(-) MS for natural organic matter analysis
 - Automatic fine isotopic structure calculation and search for all isotopes
@@ -141,7 +117,18 @@ Data handling and software development for modern mass spectrometry (MS) is an i
 - Kendrick classification
 - Heteroatoms classification and visualization
 
-### High Resolution Mass spectrum simulations
+### GC-MS Signal Processing, Calibration, and Compound Identification
+
+- Baseline detection, subtraction, smoothing 
+- m/z based Chromatogram Peak Deconvolution,
+- Manual and automatic noise threshold calculation
+- First and second derivatives peak picking methods
+- Peak Area Calculation
+- Retention Index Calibration
+- Automatic local (SQLite) or external (MongoDB or PostgreSQL) database check, generation, and search
+- Automatic molecular match algorithm with all spectral similarity methods 
+
+### High Resolution Mass Spectrum Simulations
 
 - Peak shape (Lorentz,  Gaussian, Voigt, and pseudo-Voigt)
 - Peak fitting for peak shape definition
@@ -150,7 +137,7 @@ Data handling and software development for modern mass spectrometry (MS) is an i
 - Calculated ICR Resolving Power based on magnetic field (B), and transient time(T)
 
 ---
-## CoreMS Installation 
+## Installation 
 
 ```bash
 pip install corems
@@ -164,16 +151,10 @@ To use Postgresql the easiest way is to build a docker container:
 docker-compose up -d
 ```
 
--  Change the url_database on MSParameters.molecular_search.url_database to:
+-  Change the url_database on MSParameters.molecular_search.url_database to: "postgresql+psycopg2://coremsappdb:coremsapppnnl@localhost:5432/coremsapp"
+-  Set the url_database env variable COREMS_DATABASE_URL to: "postgresql+psycopg2://coremsappdb:coremsapppnnl@localhost:5432/coremsapp"
 
-    "postgresql+psycopg2://coremsappdb:coremsapppnnl@localhost:5432/coremsapp"
-
--  Set the url_database env variable COREMS_DATABASE_URL to:
-
-    "postgresql+psycopg2://coremsappdb:coremsapppnnl@localhost:5432/coremsapp"
-
----
-## Thermo Raw File Access:
+### Thermo Raw File Access:
 
 To be able to open thermo file a installation of pythonnet is needed:
 - Windows: 
@@ -188,11 +169,11 @@ To be able to open thermo file a installation of pythonnet is needed:
     ```  
 
 ---
-### Another option is to run the docker stack that will start the CoreMS containers:  
+## Docker stack 
 
----
+Another option to use CoreMS is to run the docker stack that will start the CoreMS containers
 
-## Molecular Database and Jupyter Notebook Containers
+### Molecular Database and Jupyter Notebook Docker Containers
 
 A docker container containing:
 - A custom python distribution will all dependencies installed
@@ -245,7 +226,7 @@ If you don't have docker installed, the easiest way is to [install docker for de
 ___
 ## Simple Script Example
 
-More examples can be found under the directory docs/example, docs/notebooks
+More examples can be found under the directory examples/scripts, examples/notebooks
 
 - Basic functionality example
 
@@ -257,75 +238,66 @@ from matplotlib import pyplot
 
 file_path= 'tests/tests_data/ftms/ESI_NEG_SRFA.d'
 
-#Bruker Solarix class reader
+# Instatiate the Bruker Solarix reader with the filepath
 bruker_reader = ReadBrukerSolarix(file_path)
 
-#access the transient object
+# Use the reader to instatiate a transient object
 bruker_transient_obj = bruker_reader.get_transient()
 
-#calculates the transient duration time
+# Calculate the transient duration time
 T =  bruker_transient_obj.transient_time
 
-#access the mass spectrum object
+# Use the transient object to instatitate a mass spectrum object
 mass_spectrum_obj = bruker_transient_obj.get_mass_spectrum(plot_result=False, auto_process=True)
 
-# - search monoisotopic molecular formulas for all mass spectral peaks
-# - calculate fine isotopic structure based on monoisotopic molecular formulas found and current dynamic range
-# - search molecular formulas of correspondent calculated isotopologues,
+# The following SearchMolecularFormulas function does the following
+# - searches monoisotopic molecular formulas for all mass spectral peaks
+# - calculates fine isotopic structure based on monoisotopic molecular formulas found and current dynamic range
+# - searches molecular formulas of correspondent calculated isotopologues
 # - settings are stored at SearchConfig.json and can be changed directly on the file or inside the framework class
 
 SearchMolecularFormulas(mass_spectrum_obj, first_hit=False).run_worker_mass_spectrum()
 
-# iterate over mass spectral peaks objs
+# Iterate over mass spectral peaks objs within the mass_spectrum_obj
 for mspeak in mass_spectrum_obj.sort_by_abundance():
 
-    # returns true if there is at least one molecular formula associated
-    # with the mass spectral peak
-    # same as mspeak.is_assigned -- > bool
+    # If there is at least one molecular formula associated, mspeak returns True
     if  mspeak:
 
-        # get the molecular formula with the highest mass accuracy
+        # Get the molecular formula with the highest mass accuracy
         molecular_formula = mspeak.molecular_formula_lowest_error
 
-        # plot mz and peak height, use mass_spectrum_obj.mz_exp to access all mz
-        # and mass_spectrum_obj.mz_exp_profile to access mz with all available datapoints
+        # Plot mz and peak height
         pyplot.plot(mspeak.mz_exp, mspeak.abundance, 'o', c='g')
 
-        #or use 
-        mspeak.plot(color="black", derivative=True, deriv_color='red')
-        # iterate over all molecular formulae associated with the ms peaks obj
+        # Iterate over all molecular formulas associated with the ms peaks obj
         for molecular_formula in mspeak:
 
-            #check if the molecular formula is a isotopologue
+            # Check if the molecular formula is a isotopologue
             if molecular_formula.is_isotopologue:
 
-                #access the molecular formula text representation
+                # Access the molecular formula text representation and print
                 print (molecular_formula.string)
 
-                #get 13C atoms count
+                # Get 13C atoms count
                 print (molecular_formula['13C'])
     else:
-        #get mz and peak height
+        # Get mz and peak height
         print(mspeak.mz_exp,mspeak.abundance)
 
-
-#exporting data
+# Save data
+## to a csv file
 mass_spectrum_obj.to_csv("filename")
-
 mass_spectrum_obj.to_hdf("filename")
-# save pandas Datarame to pickle
+# to pandas Datarame pickle
 mass_spectrum_obj.to_pandas("filename")
-# get pandas Dataframe
+
+# Extract data as a pandas Dataframe
 df = mass_spectrum_obj.to_dataframe()
 ```
 ## UML Diagrams
 
-- Direct Infusion FT-MS:   
-
-    ![FT-MS UML Diagram](docs/uml/Direct_Infusion_FTMS_Data_Model.png)
-- GC-MS:  
-
-    ![GC-MS UML Diagram](docs/uml/GC_MS_Data_Model.png)
+UML (unified modeling language) diagrams for Direct Infusion FT-MS and GC-MS classes can be found [here](docs/uml).
 
 ## Citing CoreMS