failed to index on circular redirect for https://www.snib.mx/iptconabio/archive.do?r=ecoab-host-plant

[jhpoelen]

globi.json with

{
  "format": "dwca",
  "citation": "Vandame R, Mérida J, Sagot P, Madrigal González D, Bedolla García B Y, González-Vanegas P A, Cultid-Medina C A, Barrios J M (2023). Potential host plant records recovered from ECOAB wild bee collection, Mexico. Version 1.10. Comisión nacional para el conocimiento y uso de la biodiversidad.",
  "url": "https://www.snib.mx/iptconabio/archive.do?r=ecoab-host-plant"
}

related to

Vandame R, Mérida J, Sagot P, Madrigal González D, Bedolla García B Y, González-Vanegas P A, Cultid-Medina C A, Barrios J M (2023). Potential host plant records recovered from ECOAB wild bee collection, Mexico. Version 1.10. Comisión nacional para el conocimiento y uso de la biodiversidad. Occurrence dataset https://doi.org/10.15468/xdzgbt accessed via GBIF.org on 2025-02-25.

caused -

[...]
Caused by: java.io.IOException: failed to retrieve [https://www.snib.mx/iptconabio/archive.do?r=ecoab-host-plant]
	at org.globalbioticinteractions.elton.store.CachePullThroughPrestonStore.retrieve(CachePullThroughPrestonStore.java:130)
	at org.globalbioticinteractions.cache.CacheProxy.retrieve(CacheProxy.java:33)
	at org.globalbioticinteractions.cache.CacheProxyForDataset.retrieve(CacheProxyForDataset.java:31)
	at org.globalbioticinteractions.dataset.DatasetWithCache.retrieve(DatasetWithCache.java:36)
	at org.globalbioticinteractions.dataset.DatasetProxy.retrieve(DatasetProxy.java:28)
	at org.eol.globi.data.DatasetImporterForDwCA.importStudy(DatasetImporterForDwCA.java:320)
	... 19 more
Caused by: org.apache.http.client.ClientProtocolException
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at org.eol.globi.util.ResourceServiceHTTP.getCachedRemoteInputStream(ResourceServiceHTTP.java:35)
[...]

[jhpoelen]

Firefox 135.0.1 (64-bit) doesn't appear to like the resource either. . .

[jhpoelen]

see also globalbioticinteractions/globalbioticinteractions#1049 .

[jhpoelen]

For now, I've managed to grab a DwC-A copy and embedded it into an index configuration at:

https://github.com/globalbioticinteractions/ecoab-host-plant

[jhpoelen]

Note that the issue appears to be intermittent, meaning that sometimes the download succeeds, other times a circular 301 redirect are produced.

$ curl -LI https://www.snib.mx/iptconabio/archive.do?r=ecoab-host-plant
HTTP/1.1 200 
Date: Tue, 25 Feb 2025 17:59:16 GMT
Server: Apache/2.4.29 (Ubuntu)
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, OPTIONS, HEAD
Last-Modified: Mon, 15 Apr 2024 17:06:49 GMT
Content-Disposition: filename="dwca-ecoab-host-plant-v1.10.zip"
Content-Type: application/zip;charset=ISO-8859-1
Content-Language: en-GB
Transfer-Encoding: chunked
Set-Cookie: JSESSIONID=5180449DAA8F8B41FA5D1C3DC002CFE6; Path=/iptconabio; Secure; HttpOnly
Set-Cookie: CSRFtoken=fjgWr9IXiW0k5zb7OVlpYZXmv4MWy0a9; Max-Age=900; Expires=Tue, 25-Feb-2025 18:14:16 GMT; Domain=www.snib.mx; Path=/iptconabio; Secure; HttpOnly

but moments later . . .

$ curl -LI https://www.snib.mx/iptconabio/archive.do?r=ecoab-host-plant
HTTP/1.1 301 Moved Permanently
Date: Tue, 25 Feb 2025 17:59:17 GMT
Server: Apache/2.4.29 (Ubuntu)
Location: https://www.snib.mx/iptconabio/archive.do?r=ecoab-host-plant
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 301 Moved Permanently
Date: Tue, 25 Feb 2025 17:59:17 GMT
Server: Apache/2.4.29 (Ubuntu)
Location: https://www.snib.mx/iptconabio/archive.do?r=ecoab-host-plant
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 301 Moved Permanently
Date: Tue, 25 Feb 2025 17:59:17 GMT
Server: Apache/2.4.29 (Ubuntu)
Location: https://www.snib.mx/iptconabio/archive.do?r=ecoab-host-plant
Content-Type: text/html; charset=iso-8859-1
[...]
HTTP/1.1 301 Moved Permanently
Date: Tue, 25 Feb 2025 17:59:21 GMT
Server: Apache/2.4.29 (Ubuntu)
Location: https://www.snib.mx/iptconabio/archive.do?r=ecoab-host-plant
Content-Type: text/html; charset=iso-8859-1

curl: (47) Maximum (50) redirects followed

[jhpoelen]

@mdoering is this circular redirect of IPT resources a known issue?

[mdoering]

No, but this is looks like it is some Apache setup that goes wrong. @mike-podolskiy90 ever seen this before?

[mike-podolskiy90]

I don't remember anything like that. I tried several datasets hosted at cloud.gbif.org IPTs and curl works fine for them.

[jhpoelen]

@mdoering @mike-podolskiy90 thanks for your prompt reply. Is there any way for me to bypass apache and hit the app server directly? This would help narrow down the issue.

[mdoering]

probably not, but I have no idea how this IPT is setup