Description
It would be very valuable to have an api by which I can query all dependencies of a group of modules provided as input.
Current approach / workaround
As this API currently does not exist, I'm sending a series of requests like this one:
my $http = HTTP::Tiny->new( agent => 'LedgerSMB-Installer/0.1' );
my $json = JSON::PP->new;
my @last_deps = qw(Carp Carp::Clan Workflow Email::Stuffer );
my $query = {
query => { match_all => {} },
_source => [ qw( release distribution status provides ), 'dependency.*' ],
filter => {
and => [
{ term => { status => 'latest' } },
{ terms => { provides => [ @last_deps ] } }
]
}
};
my $body = $json->encode( $query );
my $r = $http->request( 'POST', 'https://fastapi.metacpan.org/v1/release/_search?size=1000',
{ headers => { 'Content-Type' => 'application/json' },
content => $body });The response is then parsed for new (non-core) module dependencies, while keeping track of what modules have been provided through the provides fields. As long as new dependencies show up, iterate. For my current set of dependencies in LedgerSMB, this loop completes in 9 iterations.
The approach is a bit hand-wavy, because it ignores version ranges that may have been set for dependencies, but the results come close enough for my needs at this point.
Context
Use-case: Installer
For the LedgerSMB project, I'm writing an installer application: experience has shown that the instructions for its installation are simply too complex. Part of the complexity is in the number of distributions that need to be installed for it to work.
The installer assumes the user wants the simplest experience possible. That is, the user wants to run LedgerSMB on the system Perl; installing perlbrew or plenv will be too complex for them. That being the case, the installation procedure can leverage modules which come with the distribution to further simplify the process: distribution packages pull in their dependencies. An example of a benefits here is that Debian's libdbd-pg-perl pulls in libpq5 without the need to set up a compilation environment able to compile against libpq-dev.
Before I implemented the 9-iteration approach with the advanced ES-search query, I implemented an approach where I used the MetaCPAN API to map the modules to dists using 134 API calls to the /v1/module/<MODULE> endpoint, followed by a similar number of calls to cpandeps.grinnz.com/v1 to get the dependency tree per module.
Before I implemented the combined cpandeps+metacpan approach, I took the naive approach for the installer to just map the immediate dependencies to distribution modules, which didn't work well, because some indirect dependencies depend on external libraries and programs too (such as Workflow's XML::Simple dependency).
Use-case: Dependency reduction
For years, I've been wondering whether there were opportunities to cut down on the number of dependencies in LedgerSMB, because this seems to be an ever growing list. So far, I have never taken the time to investigate the dependency tree, but with this API, this would become a lot simpler.
Use-case: cpandeps.grinnz.com
@Grinnz implemented a no-guarantees, unpublished API for his dependency visualizer at cpandeps.grinnz.com; the code is publicly available in his GitHub account. He was kind enough to point me to it. His code shows that he is doing a lot of requests (although spread over time and cached after that) to ultimately download the dependency tree of various modules.
With this API, a guarantees-included API would become available for those who need it.