Javascript install through `ee.install` raised `HTTP Error 404` and `ee.require` takes a long loading time

[suhendra0812]

Hi, all.
I encountered some issues when installing javascript module using ee.install. It raised HTTP Error 404: Not Found.

Downloading 'users/marcyinfeng/Algorithms:/mcd19_prior'...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\WSBARATA01\miniconda3\envs\gee\lib\site-packages\eemont\extra.py", line 80, in install
    return ee_install(module, update, quiet)
  File "C:\Users\WSBARATA01\miniconda3\envs\gee\lib\site-packages\ee_extra\JavaScript\install.py", line 200, in install
    return _install_dependencies(x=deps, update=update, installed=[])
  File "C:\Users\WSBARATA01\miniconda3\envs\gee\lib\site-packages\ee_extra\JavaScript\install.py", line 188, in _install_dependencies
    _install(dep, update, quiet)
  File "C:\Users\WSBARATA01\miniconda3\envs\gee\lib\site-packages\ee_extra\JavaScript\install.py", line 158, in _install
    with urllib.request.urlopen(_convert_path_to_ee_sources(x)) as url:
  File "C:\Users\WSBARATA01\miniconda3\envs\gee\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\WSBARATA01\miniconda3\envs\gee\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "C:\Users\WSBARATA01\miniconda3\envs\gee\lib\urllib\request.py", line 640, in http_response
    response = self.parent.error(
  File "C:\Users\WSBARATA01\miniconda3\envs\gee\lib\urllib\request.py", line 569, in error
    return self._call_chain(*args)
  File "C:\Users\WSBARATA01\miniconda3\envs\gee\lib\urllib\request.py", line 502, in _call_chain
    result = func(*args)
  File "C:\Users\WSBARATA01\miniconda3\envs\gee\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

It seems caused by the colon replacement which is insufficient for other cases. I found some javascript modules which have unusual path such as:
users/marcyinfeng/Algorithms:/mcd19_prior and users/marcyinfeng/Algorithms/:/S2_view_angle

def _convert_path_to_ee_sources(path: str) -> str:
    """Get the remote module path from the 'ee-sources' GCS bucket.

    Args:
        path: str

    Returns:
        An ee-sources module url.
    """
    if path.startswith("http"):
        eempath = path
    else:
        bpath = path.replace(":", "/") # <- insufficient replacement
        eempath = f"https://storage.googleapis.com/ee-sources/{bpath}"
    return eempath

I try to normalize the path using pathlib as follows:

def _convert_path_to_ee_sources(path: str) -> str:
    """Get the remote module path from the 'ee-sources' GCS bucket.

    Args:
        path: str

    Returns:
        An ee-sources module url.
    """
    if path.startswith("http"):
        eempath = path
    else:
        bpath = str(pathlib.Path(path.replace(":", "/"))).replace("\\", "/") # <- fix my issue, but only for Windows case
        eempath = f"https://storage.googleapis.com/ee-sources/{bpath}"
    return eempath

All dependencies were installed successfully:

The module 'users/marcyinfeng/utils:SIAC' is already installed!
Checking dependencies for users/marcyinfeng/utils:SIAC...
The module 'users/marcyinfeng/Algorithms:S2_MT_Cloud' is already installed!
Checking dependencies for users/marcyinfeng/Algorithms:S2_MT_Cloud...
The module 'users/marcyinfeng/Algorithms:NN_cloud' is already installed!
Checking dependencies for users/marcyinfeng/Algorithms:NN_cloud...
The module 'users/marcyinfeng/Algorithms:/mcd19_prior' is already installed!
Checking dependencies for users/marcyinfeng/Algorithms:/mcd19_prior...
The module 'users/marcyinfeng/Algorithms:/get_aot' is already installed!
Checking dependencies for users/marcyinfeng/Algorithms:/get_aot...
The module 'users/marcyinfeng/Algorithms/:/S2_view_angle' is already installed!
Checking dependencies for users/marcyinfeng/Algorithms/:/S2_view_angle...
The module 'users/marcyinfeng/Algorithms/:/landsat_view_angles' is already installed!
Checking dependencies for users/marcyinfeng/Algorithms/:/landsat_view_angles...
The module 'users/marcyinfeng/Algorithms/:/NN_prosail' is already installed!
Checking dependencies for users/marcyinfeng/Algorithms/:/NN_prosail...
The module 'users/marcyinfeng/Algorithms/:/get_xps' is already installed!
Checking dependencies for users/marcyinfeng/Algorithms/:/get_xps...
The module 'users/marcyinfeng/utils/:/load_NN' is already installed!
Checking dependencies for users/marcyinfeng/utils/:/load_NN...
The module 'users/gena/packages:palettes' is already installed!
Checking dependencies for users/gena/packages:palettes...
The module 'users/marcyinfeng/Algorithms/:/simu_ref' is already installed!
Checking dependencies for users/marcyinfeng/Algorithms/:/simu_ref...
The module 'users/marcyinfeng/Algorithms:mcd19_prior' is already installed!
Checking dependencies for users/marcyinfeng/Algorithms:mcd19_prior...
The module 'users/gena/packages:text' is already installed!
Checking dependencies for users/gena/packages:text...
All dependencies were successfully installed!

But it takes a long time when loading the module using ee.require. Is it depend on how many other dependent modules?

Please, fixed this issue if possible. Thank you in advance.

Best regards,
Suhendra

[csaybar]

Hi @suhendra0812 can you share with us a reproducible example? thanks :D

[suhendra0812]

Hi, @csaybar
Here is the example:

import ee
import eemont

ee.Initialize()

mod = "users/marcyinfeng/utils:SIAC"
ee.install(mod)
siac = ee.require(mod)

The example is just as simple as that. With the native script of ee.install, it raised the HTTP Error. After I edited the line in ee.install and the module was installed successfully, but it took a long loading time when executing the ee.require line.

[csaybar]

Hi @suhendra0812 sorry for the late reply, soon I will try to fix this issue and add it to our unit test to make sure it doesn't happen again. Happy New Year!

[MayerT1]

I have a similar issue: HTTPError: HTTP Error 404: Not Found.
When trying to import a JS Module. I am assuming these are the same problem.

import ee, eemont
ee.Initialize()
Rice_Extent_Mapper = "users/tjm0042/Rice_Extent_Mapper:main.js"
ee.install(Rice_Extent_Mapper)
Rice_Extent_Mapper = ee.require(Rice_Extent_Mapper)
print(Rice_Extent_Mapper)

[davemlz]

There is another error, this one using rgeeExtra:

[ckukund]

Hey @csaybar any headway on this error?

library(rgee) 
ee_initialize(drive=TRUE) 
siac <- ee$require('users/marcyinfeng/Toa2Lai:S2_Toa2Lai')

Error:

Error in py_call_impl(callable, dots$args, dots$keywords) : 
HTTPError: HTTP Error 404: Not Found

Kindly, Collins

[CvenTito]

Hi @csaybar , @davemlz ;
I still have the same error, I don't know if you managed to fix it.
Thanks

[calabres]

Hello, I have the same problem. How can I solve it?
Thank you very much

[mathiasweidinger]

Hi everyone. Also got the same issue. Any progress on fixing it? Cheers.

[thuanhavan]

I got the same issue on python