(dora2) mkolopanis@enterprise [12:59:10] [~/src/dora] [master *]
-> % mamba list
__ __ __ __
/ \ / \ / \ / \
/ \/ \/ \/ \
███████████████/ /██/ /██/ /██/ /████████████████████████
/ / \ / \ / \ / \ \____
/ / \_/ \_/ \_/ \ o \__,
/ _/ \_____/ `
|/
███╗ ███╗ █████╗ ███╗ ███╗██████╗ █████╗
████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
██╔████╔██║███████║██╔████╔██║██████╔╝███████║
██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
██║ ╚═╝ ██║██║ ██║██║ ╚═╝ ██║██████╔╝██║ ██║
╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═╝╚═════╝ ╚═╝ ╚═╝
mamba (0.15.3) supported by @QuantStack
GitHub: https://github.com/mamba-org/mamba
Twitter: https://twitter.com/QuantStack
█████████████████████████████████████████████████████████████
# packages in environment at /home/mkolopanis/src/anaconda/envs/dora2:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main
_openmp_mutex 4.5 1_gnu
ca-certificates 2021.10.26 h06a4308_2
certifi 2021.10.8 py39h06a4308_0
ld_impl_linux-64 2.35.1 h7274673_9
libffi 3.3 he6710b0_2
libgcc-ng 9.3.0 h5101ec6_17
libgomp 9.3.0 h5101ec6_17
libstdcxx-ng 9.3.0 hd4cf53a_17
ncurses 6.2 he6710b0_1
openssl 1.1.1l h7f8727e_0
pip 21.2.4 py39h06a4308_0
python 3.9.7 h12debd9_1
readline 8.1 h27cfd23_0
setuptools 58.0.4 py39h06a4308_0
sqlite 3.36.0 hc218d9a_0
tk 8.6.11 h1ccaba5_0
tzdata 2021e hda174b7_0
wheel 0.37.0 pyhd3eb1b0_1
xz 5.2.5 h7b6447c_0
zlib 1.2.11 h7b6447c_3
(dora2) mkolopanis@enterprise [12:59:18] [~/src/dora] [master *]
-> % pip install .
Processing /data4/mkolopanis/src/dora
DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
Collecting matplotlib==3.3.4
Using cached matplotlib-3.3.4-cp39-cp39-manylinux1_x86_64.whl (11.5 MB)
Collecting numpy==1.19.5
Using cached numpy-1.19.5-cp39-cp39-manylinux2010_x86_64.whl (14.9 MB)
Collecting scipy==1.5.4
Using cached scipy-1.5.4-cp39-cp39-manylinux1_x86_64.whl (25.8 MB)
Collecting scikit-image==0.18.0
Using cached scikit_image-0.18.0-cp39-cp39-manylinux1_x86_64.whl (28.9 MB)
Collecting scikit-learn==0.24.2
Using cached scikit_learn-0.24.2-cp39-cp39-manylinux2010_x86_64.whl (23.8 MB)
Collecting Pillow==8.2.0
Using cached Pillow-8.2.0-cp39-cp39-manylinux1_x86_64.whl (3.0 MB)
Collecting planetaryimage==0.5.0
Using cached planetaryimage-0.5.0-py2.py3-none-any.whl (13 kB)
Collecting PyYAML==5.4.1
Using cached PyYAML-5.4.1-cp39-cp39-manylinux1_x86_64.whl (630 kB)
Collecting pandas==1.1.5
Using cached pandas-1.1.5-cp39-cp39-manylinux1_x86_64.whl (9.3 MB)
Collecting rasterio==1.2.6
Using cached rasterio-1.2.6-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (19.2 MB)
Collecting tqdm==4.62.0
Using cached tqdm-4.62.0-py2.py3-none-any.whl (76 kB)
Collecting tensorflow==2.5.1
Using cached tensorflow-2.5.1-cp39-cp39-manylinux2010_x86_64.whl (454.5 MB)
Collecting tensorflow-probability==0.13.0
Using cached tensorflow_probability-0.13.0-py2.py3-none-any.whl (5.4 MB)
Collecting tensorflow-addons==0.13.0
Using cached tensorflow_addons-0.13.0-cp39-cp39-manylinux2010_x86_64.whl (679 kB)
Collecting sklearn-som==1.1.0
Using cached sklearn_som-1.1.0-py3-none-any.whl (6.7 kB)
Collecting cycler>=0.10
Using cached cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3
Using cached pyparsing-3.0.3-py3-none-any.whl (96 kB)
Collecting kiwisolver>=1.0.1
Using cached kiwisolver-1.3.2-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)
Collecting python-dateutil>=2.1
Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting pytz>=2017.2
Using cached pytz-2021.3-py2.py3-none-any.whl (503 kB)
Collecting six
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting pvl
Using cached pvl-1.3.0-py2.py3-none-any.whl (65 kB)
Collecting click-plugins
Using cached click_plugins-1.1.1-py2.py3-none-any.whl (7.5 kB)
Requirement already satisfied: setuptools in /data4/mkolopanis/src/anaconda/envs/dora2/lib/python3.9/site-packages (from rasterio==1.2.6->dora-exp-pipeline==0.0.9) (58.0.4)
Collecting click>=4.0
Using cached click-8.0.3-py3-none-any.whl (97 kB)
Collecting snuggs>=1.4.1
Using cached snuggs-1.4.7-py3-none-any.whl (5.4 kB)
Collecting affine
Using cached affine-2.3.0-py2.py3-none-any.whl (15 kB)
Requirement already satisfied: certifi in /data4/mkolopanis/src/anaconda/envs/dora2/lib/python3.9/site-packages (from rasterio==1.2.6->dora-exp-pipeline==0.0.9) (2021.10.8)
Collecting attrs
Using cached attrs-21.2.0-py2.py3-none-any.whl (53 kB)
Collecting cligj>=0.5
Using cached cligj-0.7.2-py3-none-any.whl (7.1 kB)
Collecting networkx>=2.0
Using cached networkx-2.6.3-py3-none-any.whl (1.9 MB)
Collecting imageio>=2.3.0
Using cached imageio-2.10.1-py3-none-any.whl (3.3 MB)
Collecting tifffile>=2019.7.26
Using cached tifffile-2021.10.12-py3-none-any.whl (175 kB)
Collecting PyWavelets>=1.1.1
Using cached PyWavelets-1.1.1-cp39-cp39-manylinux1_x86_64.whl (4.3 MB)
Collecting joblib>=0.11
Using cached joblib-1.1.0-py2.py3-none-any.whl (306 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.0.0-py3-none-any.whl (14 kB)
Requirement already satisfied: wheel~=0.35 in /data4/mkolopanis/src/anaconda/envs/dora2/lib/python3.9/site-packages (from tensorflow==2.5.1->dora-exp-pipeline==0.0.9) (0.37.0)
Collecting termcolor~=1.1.0
Using cached termcolor-1.1.0-py3-none-any.whl
Collecting h5py~=3.1.0
Using cached h5py-3.1.0-cp39-cp39-manylinux1_x86_64.whl (4.4 MB)
Collecting astunparse~=1.6.3
Using cached astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting wrapt~=1.12.1
Using cached wrapt-1.12.1-cp39-cp39-linux_x86_64.whl
Collecting grpcio~=1.34.0
Using cached grpcio-1.34.1-cp39-cp39-manylinux2014_x86_64.whl (4.0 MB)
Collecting gast==0.4.0
Using cached gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting keras-nightly~=2.5.0.dev
Using cached keras_nightly-2.5.0.dev2021032900-py2.py3-none-any.whl (1.2 MB)
Collecting opt-einsum~=3.3.0
Using cached opt_einsum-3.3.0-py3-none-any.whl (65 kB)
Collecting flatbuffers~=1.12.0
Using cached flatbuffers-1.12-py2.py3-none-any.whl (15 kB)
Collecting six
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting keras-preprocessing~=1.1.2
Using cached Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
Collecting typing-extensions~=3.7.4
Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Collecting protobuf>=3.9.2
Using cached protobuf-3.19.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
Collecting absl-py~=0.10
Using cached absl_py-0.15.0-py3-none-any.whl (132 kB)
Collecting tensorflow-estimator<2.6.0,>=2.5.0
Using cached tensorflow_estimator-2.5.0-py2.py3-none-any.whl (462 kB)
Collecting google-pasta~=0.2
Using cached google_pasta-0.2.0-py3-none-any.whl (57 kB)
Collecting tensorboard~=2.5
Using cached tensorboard-2.7.0-py3-none-any.whl (5.8 MB)
Collecting typeguard>=2.7
Using cached typeguard-2.13.0-py3-none-any.whl (17 kB)
Collecting decorator
Using cached decorator-5.1.0-py3-none-any.whl (9.1 kB)
Collecting cloudpickle>=1.3
Using cached cloudpickle-2.0.0-py3-none-any.whl (25 kB)
Collecting dm-tree
Using cached dm_tree-0.1.6-cp39-cp39-manylinux_2_24_x86_64.whl (94 kB)
Collecting imageio>=2.3.0
Using cached imageio-2.9.0-py3-none-any.whl (3.3 MB)
Collecting werkzeug>=0.11.15
Using cached Werkzeug-2.0.2-py3-none-any.whl (288 kB)
Collecting tensorboard-plugin-wit>=1.6.0
Using cached tensorboard_plugin_wit-1.8.0-py3-none-any.whl (781 kB)
Collecting requests<3,>=2.21.0
Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting google-auth<3,>=1.6.3
Using cached google_auth-2.3.2-py2.py3-none-any.whl (155 kB)
Collecting tensorboard-data-server<0.7.0,>=0.6.0
Using cached tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl (4.9 MB)
Collecting markdown>=2.6.8
Using cached Markdown-3.3.4-py3-none-any.whl (97 kB)
Collecting google-auth-oauthlib<0.5,>=0.4.1
Using cached google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting cachetools<5.0,>=2.0.0
Using cached cachetools-4.2.4-py3-none-any.whl (10 kB)
Collecting requests-oauthlib>=0.7.0
Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting pyasn1<0.5.0,>=0.4.6
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.7-py3-none-any.whl (38 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.7-py2.py3-none-any.whl (138 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
Building wheels for collected packages: dora-exp-pipeline
Building wheel for dora-exp-pipeline (setup.py) ... done
Created wheel for dora-exp-pipeline: filename=dora_exp_pipeline-0.0.9-py3-none-any.whl size=45376 sha256=dd5d1c0c5abf33eeb39470e924095e487efebd5d81b4510473678f871fdd1a66
Stored in directory: /tmp/pip-ephem-wheel-cache-5x_0eamm/wheels/f6/95/41/53ba3ab1b68b2c75895f11d9ae5e4dfbecd90887f2807aab21
Successfully built dora-exp-pipeline
Installing collected packages: urllib3, pyasn1, idna, charset-normalizer, six, rsa, requests, pyasn1-modules, oauthlib, cachetools, requests-oauthlib, google-auth, werkzeug, tensorboard-plugin-wit, tensorboard-data-server, python-dateutil, pyparsing, protobuf, Pillow, numpy, markdown, kiwisolver, grpcio, google-auth-oauthlib, cycler, click, absl-py, wrapt, typing-extensions, typeguard, tifffile, threadpoolctl, termcolor, tensorflow-estimator, tensorboard, snuggs, scipy, PyWavelets, pytz, pvl, opt-einsum, networkx, matplotlib, keras-preprocessing, keras-nightly, joblib, imageio, h5py, google-pasta, gast, flatbuffers, dm-tree, decorator, cloudpickle, cligj, click-plugins, attrs, astunparse, affine, tqdm, tensorflow-probability, tensorflow-addons, tensorflow, sklearn-som, scikit-learn, scikit-image, rasterio, PyYAML, planetaryimage, pandas, dora-exp-pipeline
Successfully installed Pillow-8.2.0 PyWavelets-1.1.1 PyYAML-5.4.1 absl-py-0.15.0 affine-2.3.0 astunparse-1.6.3 attrs-21.2.0 cachetools-4.2.4 charset-normalizer-2.0.7 click-8.0.3 click-plugins-1.1.1 cligj-0.7.2 cloudpickle-2.0.0 cycler-0.10.0 decorator-5.1.0 dm-tree-0.1.6 dora-exp-pipeline-0.0.9 flatbuffers-1.12 gast-0.4.0 google-auth-2.3.2 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 grpcio-1.34.1 h5py-3.1.0 idna-3.3 imageio-2.9.0 joblib-1.1.0 keras-nightly-2.5.0.dev2021032900 keras-preprocessing-1.1.2 kiwisolver-1.3.2 markdown-3.3.4 matplotlib-3.3.4 networkx-2.6.3 numpy-1.19.5 oauthlib-3.1.1 opt-einsum-3.3.0 pandas-1.1.5 planetaryimage-0.5.0 protobuf-3.19.0 pvl-1.3.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pyparsing-3.0.3 python-dateutil-2.8.2 pytz-2021.3 rasterio-1.2.6 requests-2.26.0 requests-oauthlib-1.3.0 rsa-4.7.2 scikit-image-0.18.0 scikit-learn-0.24.2 scipy-1.5.4 six-1.15.0 sklearn-som-1.1.0 snuggs-1.4.7 tensorboard-2.7.0 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.0 tensorflow-2.5.1 tensorflow-addons-0.13.0 tensorflow-estimator-2.5.0 tensorflow-probability-0.13.0 termcolor-1.1.0 threadpoolctl-3.0.0 tifffile-2021.10.12 tqdm-4.62.0 typeguard-2.13.0 typing-extensions-3.7.4.3 urllib3-1.26.7 werkzeug-2.0.2 wrapt-1.12.1
(dora2) mkolopanis@excalibur [13:13:00] [~/src/dora] [master *]
-> % pytest
========================================== test session starts ==========================================
platform linux -- Python 3.9.7, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /data4/mkolopanis/src/dora, configfile: pytest.ini
plugins: typeguard-2.13.0
collected 4 items
test/test_earth_time_series.py . [ 25%]
test/test_earth_volcanoes_patches.py . [ 50%]
test/test_earth_volcanoes_pixels.py . [ 75%]
test/test_planetary.py F [100%]
=============================================== FAILURES ================================================
________________________________________ TestPlanetary.test_run _________________________________________
self = <test_planetary.TestPlanetary testMethod=test_run>
def test_run(self):
config_file = 'test/planetary.config'
# Run the experiment; also tests ability to parse config file
start(config_file, self.outdir)
# Check results for each algorithm
for file_base in ['demud-k=5/selections-demud.csv',
'iforest-n_trees=100-fit_single_trees=False/'
'selections-iforest.csv',
'lrx-inner_window=3-outer_window=9-bands=1/'
'selections-lrx.csv',
'negative_sampling-percent_increase=20/'
'selections-negative_sampling.csv',
'pca-k=5/selections-pca.csv',
'random/selections-random.csv'
]:
correct_file = 'test/ref-results/planetary_rover/functional' \
'/%s' % file_base
output_file = '%s/%s' % (self.outdir, file_base)
# Check results against correct output
> assert check_results(output_file, correct_file)
E AssertionError: assert False
E + where False = check_results('/tmp/dora/test/planetary/demud-k=5/selections-demud.csv', 'test/ref-results/planetary_rover/functional/demud-k=5/selections-demud.csv')
test/test_planetary.py:49: AssertionError
----------------------------------------- Captured stdout call ------------------------------------------
Loading data_to_fit
Loading data_to_score
----------------------------------------- Captured stderr call ------------------------------------------
Feature extraction: 100%|██████████| 1/1 [00:00<00:00, 37.41it/s]
Feature extraction: 100%|██████████| 1/1 [00:00<00:00, 141.73it/s]
DEMUD: 100%|██████████| 8/8 [00:00<00:00, 1239.77it/s]s]
Negative Sampling: 100%|██████████| 100/100 [00:00<00:00, 44108.78it/s]
LRX: 100%|██████████| 8/8 [00:00<00:00, 311.95it/s]6, 5.60s/it]
Outlier detection: 100%|██████████| 6/6 [00:13<00:00, 2.25s/it]
=========================================== warnings summary ============================================
../../../../home/mkolopanis/src/anaconda/envs/dora2/lib/python3.9/site-packages/pvl/collections.py:700
/home/mkolopanis/src/anaconda/envs/dora2/lib/python3.9/site-packages/pvl/collections.py:700: PendingDeprecationWarning: The pvl.collections.Units object is deprecated, and may be removed at the next major patch. Please use pvl.collections.Quantity instead.
warnings.warn(
../../../../home/mkolopanis/src/anaconda/envs/dora2/lib/python3.9/site-packages/planetaryimage/specialpixels.py:20: 12 warnings
/home/mkolopanis/src/anaconda/envs/dora2/lib/python3.9/site-packages/planetaryimage/specialpixels.py:20: DeprecationWarning: The binary mode of fromstring is deprecated, as it behaves surprisingly on unicode inputs. Use frombuffer instead
return numpy.fromstring(num, dtype=dtype)[0]
../../../../home/mkolopanis/src/anaconda/envs/dora2/lib/python3.9/site-packages/tensorflow/python/autograph/impl/api.py:22
/home/mkolopanis/src/anaconda/envs/dora2/lib/python3.9/site-packages/tensorflow/python/autograph/impl/api.py:22: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
-- Docs: https://docs.pytest.org/en/stable/warnings.html
======================================== short test summary info ========================================
FAILED test/test_planetary.py::TestPlanetary::test_run - AssertionError: assert False
========================= 1 failed, 3 passed, 14 warnings in 162.77s (0:02:42) ==========================
It appears with the tagged version of scikit-image, there is no
pyproject.tomlto require a numpy package be installed before building.As a result, a user is required to install numpy into their environment before attempting to install this package.
Even defining a minimal pyroject.toml here like below cannot alleviate the issue.
However bumping to scikit-image==0.18.0 does appear to fix the issue.
Can't guarantee tests will pass with this version since they appear to already not pass as of 5aaf81f however the three that are passing on the main branch also still pass
Blank Env Install
Env w/ Numpy
Blank Env image==0.18.0