Support for Mercurial repositories

[jodogne]

Dear all,

I have just created a publiccode.yml for the Orthanc project, at the following location.

When I run the validator on this file, I get:

$ docker run -i italia/publiccode-parser-go /dev/stdin < publiccode.yml
publiccode.yml:3:1: error: url: is not a valid code repository

But, the url indeed refers to a valid Mercurial source code repository.

This seems to indicate that publiccode-parser-go does not feature support for Mercurial repositories. To validate whether the url is valid for Mercurial, one could check whether the ?cmd=capabilities query string is accessible under the provided URL, while ?cmd=nope returns a 400 HTTP status code. The ?cmd=capabilities query string indeed serves as a starting point for the Mercurial binary protocol. For instance:

$ curl https://orthanc.uclouvain.be/hg/orthanc/?cmd=capabilities
batch branchmap bundle2=HG20%0Abookmarks%0Achangegroup%3D01%2C02%2C03%0Acheckheads%3Drelated%0Adelta-compression%3Dnone%2Czlib%2Czstd%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Aphases%3Dheads%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps%0Astream%3Dv2 changegroupsubset compression=zstd,zlib getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlog-compression-zstd,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
$ curl -w '%{http_code}' -o /dev/null -s https://orthanc.uclouvain.be/hg/orthanc/?cmd=nope
400

Kind Regards,
Sébastien-

[bfabio]

That would be a nice feature to have. Even better if it could be implemented without any network requests. Care to take a shot at it?

[jodogne]

I'm afraid I'm not the right person to take this on. I don't have experience with Go, and as the leader of a large-scale project (Orthanc), my available time is already quite limited. I'm not in a position to dig into a new implementation in a different stack.

[bfabio]

Sounds fair. Mercurial usage is pretty niche at this point, so this is going to be low priority. Keeping it open, if someone who needs this wants to contribute a PR.

[jodogne]

poke @barthanssens: Could someone from Fedict help here?

[barthanssens]

We're mostly a PHP / Java / NodeJS shop, but I might give it a shot myself when I have some spare time in May...

Perhaps a more general question, is publiccode parser currently limited to git ? Or does it also support e.g. SVN ?

[bfabio]

@barthanssens yes, the parser is currently limited to git for external checks (i.e. network validation that the repository URL is actually reachable and valid).

However, these checks can be disabled with the -no-network flag, so a Mercurial or SVN URL should pass validation just fine with that option (docker run -i italia/publiccode-parser-go -no-network /dev/stdin < publiccode.yml or just publiccode-parser-go -no-network publiccode.yml)

The component that would really need to support other SCMs is publiccode-crawler, which currently only knows how to interact with git repositories (actually, a subset of git forges italia/publiccode-crawler#196).

So for now, as a workaround, with -no-network (or DisableNetwork = true when used as a library).