todo.txt

-- update to use junit 5
   -- also add org.junit.vintage junit-vintage-engine
   -- find compatible jacoco library, preferably a lib jar rather
      than an install for computing platform.
   -- update ant plugins for the changes:
      https://blog.jetbrains.com/idea/2020/08/migrating-from-junit-4-to-junit-5/
      https://ant.apache.org/manual/Tasks/junitlauncher.html

-- add roofline notes to vec8prod
    https://docs.nersc.gov/tools/performance/roofline/

-- guava has consistent hash!  (...load balancing w/ caveat)
   https://github.com/google/guava/blob/master/guava/src/com/google/common/hash/Hashing.java

-- consider whether can write a version of CURDecomposition using sparse matrices and
   the tensorflow operations on sparse matrices.

-- for vectorization (single instruction on mulitiple data), explore:
   - viktor:
     https://blog.jetbrains.com/kotlin/2021/03/viktor-efficient-vectorized-computations-in-kotlin/
     efficient array calculations: avoid nested arrays, 
     delegate expensive operations to JNI + SIMD, and provide built-in support 
     for arithmetics on logarithmically-stored numbers.
     optimized to work with probability arrays, intended for model-based machine learning tasks.
      - Idiomatic multidimensional array access (rows, columns, slices, views, etc.).
      - Extremely fast element-wise operations (arithmetic, exponent, logarithm, and the like)
        utilizing modern CPU cores to their full extent.
      - Really fast aggregate operations (sum, mean, standard deviation, etc.).
      - Built-in logarithmic storage support: you can convert your values to logarithms and
        work with them without having to convert them back.
      - doesn't do GPU computations
    - numpy, octave and R support vectorization
    - kotlin based multik
    - la4j library http://la4j.org/
    - intel, amd, and many other processor vendors have pages w/ linear alg and vectorization
      tips and/or apis.
      https://netlib.org/blas/faq.html
      e.g. https://www.intel.com/content/www/us/en/develop/documentation/advisor-user-guide/top/reference/recommendation-reference/vectorization-recommendations-for-c.html

-- efficient shortest paths:
   - https://www.cs.princeton.edu/courses/archive/spr06/cos423/Handouts/EPP%20shortest%20path%20algorithms.pdf
   - http://algo2.iti.kit.edu/schultes/hwy/schultes_diss.pdf
   - https://www.cs.princeton.edu/courses/archive/spr09/cos423/Lectures/reach-mit.pdf
   - jgraphT has several algorithms:
      ALT, ContractionHierarchyPrecomputation, dijkstra's many-to-many
      but not reach + shortcuts + landmarks 
      nor dynamic highway-node routing
   - nice article on broader subject:
       http://theory.stanford.edu/~amitp/GameProgramming/index.html
   - an article relevant to ALT:
       Campbell 2016, "Using Quadrilaterals to Compute the Shortest Path"
       
-- more ML libraries to explore:
   https://anywhere.epam.com/business/c-plus-plus-ml-libraries

-- amazon sage and google's kaggle

-- https://aimi.stanford.edu/shared-datasets

-- non-linear state space modeling with factor graphs
   given a series of states up to time s, and wanting to 
   determine the state at time t.

   s = t is filtering
   s < t is prediction, forecasting
   s > t is smoothing

   filtering: determine current state given past states.
      particle filtering (a.k.a. sequential monte carlo models, SMC)
      and extended kalman filtering.
      particle filtering performs approximate probabilistic inference.
         - propose distribution of particles using transition probabilities, 
           weight the particles, resample from the distribution
         - nice tutorial and python code
           https://github.com/jelfring/particle-filter-tutorial.git
         - Stanford AI CS 221 
           https://stanford-cs221.github.io/summer2023/modules/#tags=week7
   smoothing: determine a state given all states up to a later state.
      calculates a posterior probability distribution over past states
      given current and past observations (evidence).
      useful in ML learning.  Laplace smoothing helpful to avoid over-fitting.
         -  Doucet, A.; Johansen, 2008 
            "A Tutorial on Particle Filtering and Smoothing: Fifteen years later"
            https://www.stats.ox.ac.uk/~doucet/doucet_johansen_tutorialPF2011.pdf
            two-filter smoothing formula and Forward-Backward Recursions

-- browse https://github.com/GPflow/GPflow

-- explore best practices with use of java Streams, including parallel
   Brian Goetz articles are at IBM: https://developer.ibm.com/series/java-streams/

-- look into jcilk
   - does jcilk have cilksan and cilkscale?
   - also https://graphit-lang.org/ (can combine with cilk) for improving large graph processing

-- summary of cache strategy types, graph re-ordering algorithms, pull vs push (later requires
   cache line sharing between threads so needs synchronization) and performance comparisons:
   -- Esfahani, Kilpatrick, and Vandierendonck 2021, "Locality Analysis of Graph Reordering Algorithms",
        2021 IEEE Symposium on Workload Characterization.
   -- 15. Cache-Oblivious Algorithms https://youtu.be/xwE568oVQ1Y
   -- 18. Domain Specific Languages and Autotuning https://youtu.be/bd-mavr5YlA

-- explore model checking software again
-- explore https://studiolab.sagemaker.aws/

-- consider cache oblivious funnel sort

-- explore google scholar search results for
   algorithm SAS Institute Inc Chaudhuri

-- for multidimensional fast efficient KDE: the fastKDE github project
   LBL-EESA by O'Brien T.A. et al. 2014, 2016.  it uses non-uniform FFT
   and the silverman kernel method.  the code language is python.
-- explore https://www.cs.toronto.edu/~duvenaud/cookbook/

-- Bayes estimate of GEV:
   -- a conjugate prior and likelihood is nearly intractable,
      so many use sampling of the posterior by MCMC, importance, metropolis, etc.
      for an approximate inference
      - an example with details and pseudocode for the sampling:
        Choeng, R. Y. and Gabda, D., 2017, “Modelling maximum river flow by using Bayesian Markov Chain Monte Carlo”,
        prior: gaussian
        likelihood: GEV likelihood

   -- the NEVA matlab toolbox is used by
      "Non-stationary extreme value analysis in a changing climate"
       by Cheng et al. Climatic Change 2014, 127:353-369
       The default priors for the location and scale parameters are non-informative normal distributions, 
       whereas the default for the shape parameter is a normal distribution with a standard deviation of 0.3 
       as suggested in Renard et al. 2013
   -- others have used a beta distribution for the prior of the shape parameter
   -- importance sampling
   -- non-informative priors
   
-- implement approx inverse CDF:
   can approximate an inverse CDF using piecewise constant (PWC) or
   piecewise linear (PWL).
   - Inverse method from "Independent Random Sampling", 2018, Martino et al.

-- add optmization branch to Gumbel.fitGumbelUsingML
   can extend a function to use w/ LBFGS

-- explore https://www.stat.cmu.edu/~larry/all-of-statistics/=Rprograms/

-- consider implementing or a toolbox containing estimators for at least these distributions:
   Frechet,  Weibull, (X) Gumbel, (X) GEV, Rayleigh, (X) gamma distribution, inverse gaussian (wald),
   lognormal, chi-squared, non-central F

       
-- follow up browsing:
    Shamos (1976, p. 260) 
       "Geometry and Statistics: Problems at the Interface," in N ~ MDiJrections and Recent Results in Algorithms and Complexity"
    Bickel and Lehmann (1979, p. 38)
       "Descriptive Statistics for Non- parametric Models 111: Dispersion"

-- impl a GIST descriptor
-- use of CNN for images and audio.
-- use of Mel-frequency cepstrum for audio
-- impl kernel pca
-- impl  Kernel-k-means clustering
-- impl nonlinear component analysis
-- consider implementing
   Approximating a Gram Matrix for Improved Kernel-Based Learning 
   Drineas and Mahoney 
   Auer and Meir (Eds.): COLT 2005, LNAI 3559, pp. 323–337, 2005 
-- ** consider implementing
   Spectral Grouping Using the Nystrom Method
   Fowlkes, Belongie, Chung, and Malik
-- return to MMDS: TF-IDF and vectors compared to n-gram

-- implement the pseudo-determinant
   from wikipedia:
   In linear algebra and statistics, the pseudo-determinant[1] is the product of all 
   non-zero eigenvalues of a square matrix. It coincides with the regular determinant 
   when the matrix is non-singular.
   Minka, T.P. (2001). "Inferring a Gaussian Distribution"

   If A is positive semi-definite, then the singular values and eigenvalues of A coincide. 
   In this case, if the singular value decomposition (SVD) is available, then A_{+} may be 
   computed as the product of the non-zero singular values. If all singular values are zero, 
   then the pseudo-determinant is 1.
   
-- implement a fusion tree and add logic to heap wrapper for it
   from ods:
      fusion tree can store n w-bit integers in O(n) space so that the find(x) operation runs in 
      O(log{n}/log{w}) time. By using a fusion tree when log(w) > sqrt{log{n}} and 
      using a YFastTrie when log{w} le sqrt{log{n}}, one obtains an O(n) space data structure 
      that can implement the find(x) operation in O(sqrt{log{n}}) time.
-- consider dynamic graph data structures (Henzinger 2000)

-- consider implementing optimizer Adam:
   https://machinelearningmastery.com/adam-optimization-from-scratch/
   -- if haven't restored one of my downhill simplex algorithm implementations,
      can dig up the older version in two-point-correlation project history:
         src/main/java/algorithms/curves/GEVChiSquareMinimization.java
      and the non-linear conjugate gradient algorithm in
         src/main/java/algorithms/curves/NonQuadraticConjugateGradientSolver.java

-- consider changing java compile source to 1.11
   -- then update libraries

-- update the JUnit library and make any needed changes

-- in the sampling packages (e.g. algorithms.imageProcessing.features or create new), 
   add variational Bayes and expectation propagation 
   they allow Bayesian techniques to be used in large-scale applications (Blei et al., 2003).

-- consider implementing a 3D voronoi

-- implement perfect hashing one day. not currently needed.
   openjdk has an implementation of
   "A Practical Minimal Perfect Hashing Method" - Fabiano C. Botelho1,
   Yoshiharu Kohayakawa, and Nivio Ziviani, 2005 
   in PerfectHashBuilder.java 

-- consider implementing bi-directional search.
   Holte et al. 2016, "Bidirectional Search That Is Guaranteed to Meet in the Middle",
   Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16)

-- consider refactoring the code for permutations (Shuffle), combinations
   and partitions to place in one package/directory.

-- for improving the current NearestNeighbor2DLong
   could consider locality based hashing along y axis.

-- when begin to include more than 2 dimensions in clustering project,
   consider adding the vantage point tree for nearest neighbor search
   http://infolab.stanford.edu/~sergey/near.html
   http://www.pnylab.com/pny/papers/vptree/vptree/
   http://www.drmaciver.com/2011/12/4202/
   https://github.com/smreed/Java-Vantage-Tree.git

-- add unit tests where missing

-- consider importing some of code stored elsewhere, like the linear programming simplex
     method and tests in minCostFlow

-- consider moving to this project, the notes and code related to feature hashing
   in progress.
   -- related is adding compression to the bit vectors in VeryLongBitString.
      -- consider adding RoaringBitmap java implementation
      -- consider adding simple fast lite compression in bitmap encoding and decoding notes
   -- "Handling Massive N-Gram Datasets"

-- clean up the multiply and dot operations in MatrixUtil

-- consider http://members.cbio.mines-paristech.fr/~jvert/svn/bibli/local/Bengio2004Learning.pdf

-- https://adrianulbona.github.io/hmm/

-- browse 
   M. R. Henzinger, T. A. Henzinger and P. W. Kopke, "Computing simulations on finite and infinite graphs," Proceedings of IEEE 36th Annual Foundations of Computer Science, 1995, pp. 453-462, doi: 10.1109/SFCS.1995.492576.

-- implement a suffix array.
   https://github.com/kvark/dark-archon if license allows
   https://github.com/y-256/libdivsufsort/blob/wiki/SACA_Benchmarks.md
   
-- browse:
   Exactly Solving the Maximum Weight Independent Set Problem on Large Real-World Graphs
   Lamm et al. 2018, https://arxiv.org/abs/1810.10834
 
   Hopfield network

   "Narrow sieves for parameterized paths and packings", Bjorklund et al. 2010

-- low priority:
   implement the TODO in ConnectedValuesGroupFinder2
   regarding removal of merging, and adding BFS over nodes to aggregate

   - consider boundary chain codes (BCC) and differential chain codes (DCC) descriptors
   - consider Chroma subsampling
   - browse https://en.m.wikipedia.org/wiki/Fractal_compression