This package provides functions and classes to run wodan style processing functions in the Stactics AICore environment.
Use
pip install corebridgeto install corebrdige.
Wodan is a proprietary backend service that applies high performance, custom analytical processing to timeseries data in the Whysor data and dashboarding environment.
Each wodan module defines one function that operates as the entry point. The parameter annotations in this function definition are used to format data and retrieve parameters from the originating call to the wodan api. This function is called with data retrieved according to a specification and with additional parameters as annotated.
A simple function might look like:
import numpy as np
def multiply(data:np.ndarray, multiplier:float=1.0):
return data * multiplier
Wodan binds this function to a service endpoint and takes care of fetching data and parameters and converting the result for the caller.
For AICore users define a class, always named CustomModule with a
constructor __init__ and a method infer.
This package defines a baseclass to quickly construct a CustomModule
class that is able to use a wodan processor function inside the AICore
system:
import numpy as np
import corebridge
def multiply(data:np.ndarray, multiplier:float=1.0):
return data * multiplier
class CustomModule(corebridge.aicorebridge.AICoreModule):
def __init__(self, save_dir, assets_dir, *args, **kwargs):
super().__init__(multiply, save_dir, assets_dir, *args, **kwargs)
That’s it. Well, you can add parameters to __init__ that can be used
as hyperparameters in the web-interface and you could override infer
for the same reason. The baseclass takes care of converting call
parameters and data to the function specification and, calls the
function and converts the result for the caller, similar to the original
Wodan service.
You don’t have to use AICoreModule baseclas but can create your own class and use functionality within CoreBridge to convert call parameters and data to fit your needs. AICoreModule was developed to make the deployment of existing Wodan modules within AICore easier. It also helps developing functions independently of AICore while being able to deploy these functions as modules within AICore.
This library is developed with NBDev - a literate programming toolkit that supports developing code using jupyter notebooks and mix code with documentation.
Literate programming is a methodology - introduced in 1984 by Donald Knuth - that combines a programming language with a documentation language. In this approach, a program is explained in a human language (such as English) alongside code snippets. The literate source file is then processed by a preprocessor to produce both source code and formatted documentation.
This paradigm enhances program robustness, portability, and maintainability, making it a valuable tool in scientific computing and data science1
Documentation is prepared from the notebook with Quarto. Quarto too combines code with documentation but it does not extract source code into modules like nbdev.
Quarto uses Pandoc and, for pdf format, LaTeX. These must be available on your system.
Install Quarto as you see fit, there is a VSCode extension which handles this.
NBDev is available as PyPi package and is installed with
pip install nbdev
or if you are using conda
conda install -c fastai -y nbdev
If so desired you can let NBDev install Quarto with
nbdev_install_quarto
But this ask for the system admin password.
Setup a virtual environment, activate it and install the development package and dependencies with, on linux
pip install -e ‘.[dev]’
or on Windows
pip install -e .[dev]
The above pip install should also install jupyter but to use it the kernel needs to be installed with:
python -m ipykernel install --user --name=corebridge.venv
- edit
- nbdev_prepare
The latter performs - nbdev_export - nbdev_test - nbdev_clean - nbdev_readme
Before committing changes to git run - nbdev_clean
Then commit to git and for a new pip release upload to Pypi with
nbdev_pypi