Review API documentation on guides.rubygems.org

[maras1]

Examining problems in Artifactory local repo for gems we discovered that bundle installs send wrong api calls to repository
If you execute bundle install the following wrong api cals are sent.

1. Get https://rubygems.org/versions with incorrect syntax and without gem name. The corect api call should be
   Get https://rubygems.org/api/v1/versions/rails.json
2. GET https://rubygems.org/api/v1/dependencies with incorrect syntax and without gem. The correct rest apo call should be:
   GET https://rubygems.org/api/v1/dependencies?gems=chef-vault
   below you will find artifactory apache log file with incorrect calls

10.151.80.71 - - [26/Feb/2019:08:45:15 +0100] "GET /artifactory/api/gems/rubygems/versions HTTP/1.1" 404 129 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:16 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies HTTP/1.1" 200 - "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:16 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=chef-vault HTTP/1.1" 200 3420 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:16 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=chef HTTP/1.1" 200 358751 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:16 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=addressable%2Cbundler%2Cbunny%2Cchef-config%2Cchef-zero%2Cdiff-lcs%2Cerubis%2Cffi%2Cffi-yajl%2Chighline%2Ciniparse%2Ciso8601%2Cjson%2Cmime-types%2Cmixlib-archive%2Cmixlib-authentication%2Cmixlib-cli%2Cmixlib-config%2Cmixlib-log%2Cmixlib-shellout%2Cnet-sftp%2Cnet-ssh%2Cnet-ssh-multi%2Cohai%2Cplist%2Cproxifier%2Cpry%2Crest-client%2Crspec-core%2Crspec-expectations%2Crspec-mocks%2Crspec_junit_formatter%2Cserverspec%2Cspecinfra%2Csyslog-logger%2Cuuidtools%2Cwin32-api%2Cwin32-certstore%2Cwin32-dir%2Cwin32-event%2Cwin32-eventlog%2Cwin32-mmap%2Cwin32-mutex%2Cwin32-process%2Cwin32-service%2Cwin32-taskscheduler%2Cwindows-api%2Cwindows-pr%2Cwmi-lite%2Cyajl-ruby HTTP/1.1" 200 406822 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:16 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=amqp%2Cextlib%2Cmoneta%2Cpuma%2Crdp-ruby-wmi%2Cruby-hmac%2Cruby-openid%2Cstomp%2Csystemu%2Ctreetop HTTP/1.1" 200 33745 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:16 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=abstract%2Camq-protocol%2Carchive-tar-minitar%2Cbcrypt_pbkdf%2Cbones%2Cbuilder%2Ccoderay%2Cechoe%2Cfuzzyurl%2Chashie%2Choe%2Chttp-accept%2Chttp-cookie%2Cipaddress%2Cjruby-pageant%2Claunchy%2Clibyajl2%2Cmethod_source%2Cmime-types-data%2Cmulti_json%2Cneedle%2Cnet-dhcp%2Cnet-scp%2Cnet-ssh-gateway%2Cnet-telnet%2Cnetrc%2Cnokogiri%2Cptools%2Cpublic_suffix%2Crack%2Crake%2Crbnacl%2Crbnacl-libsodium%2Crcov%2Crdoc%2Crspec%2Crspec-its%2Crspec-support%2Cruby_parser%2Csfl%2Cslop%2Csolve%2Cspoon%2Ctermios%2Ctest-unit%2Cthin%2Ctomlrb%2Cwin32-ipc%2Cwin32console%2Cyard HTTP/1.1" 200 156684 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:17 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=amq-client%2Cenglish%2Ceventmachine%2Cfacets%2Cffi-win32-extensions%2Cpolyglot%2Cruby-yadis%2Cstructured_warnings HTTP/1.1" 200 21917 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:17 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=ParseTree%2CRubyInline%2Callison%2Cbones-extras%2Cbones-git%2Ccamping%2Cconfiguration%2Ccucumber%2Cdaemons%2Cdep_selector%2Cdomain_name%2Ceventmachine-le%2Cfcgi%2Cgemcutter%2Chttp_parser.rb%2Clanguage%2Clittle-plugger%2Cloquacious%2Cmemcache-client%2Cmini_portile%2Cmini_portile2%2Cminitar%2Cminitar-cli%2Cminitest%2Cmolinillo%2Cmongrel%2Cpcap%2Cpkg-config%2Cpower_assert%2Cpreforker%2Cracc%2Crake-compiler%2Crexical%2Crubyforge%2Csemverse%2Csexp_processor%2Cspicycode-rcov%2Csqlite3%2Csyntax%2Ctenderlove-frex%2Ctest-spec%2Cweakling%2Cwin32-file HTTP/1.1" 200 98731 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:17 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=SexpProcessor%2CZenTest%2Cactiverecord%2Cactivesupport%2Cbones-rcov%2Cbones-rspec%2Cbones-rubyforge%2Cbones-zentest%2Ccgi_multipart_eof_fix%2Ccucumber-core%2Ccucumber-expressions%2Ccucumber-formatter-dots%2Ccucumber-wire%2Cdep-selector-libgecode%2Cevent-bus%2Cfastthread%2Cflexmock%2Cgem_plugin%2Cgherkin%2Cgherkin3%2Cgit%2Cjson_pure%2Cmab%2Cmarkaby%2Cmetaid%2Cmocha%2Cmulti_test%2Cpattern-match%2Cpowerbar%2Cterm-ansicolor%2Cunf%2Cwin32-file-stat HTTP/1.1" 200 183851 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:17 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=activemodel%2Cactiverecord-deprecated_finders%2Cansi%2Carel%2Cbackports%2Cc21e%2Cconcurrent-ruby%2Ccucumber-messages%2Ccucumber-tag_expressions%2Ci18n%2Cmetaclass%2Cos%2Cspruz%2Cthread_safe%2Ctins%2Ctrollop%2Ctzinfo%2Cunf_ext%2Czeitwerk HTTP/1.1" 200 73439 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:17 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=atomic%2Cbcrypt-ruby%2Ccurses%2Cfunctional-ruby%2Cgoogle-protobuf%2Cref HTTP/1.1" 200 19545 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:17 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=bcrypt HTTP/1.1" 200 1921 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"
10.151.80.71 - - [26/Feb/2019:08:45:18 +0100] "GET /artifactory/api/gems/rubygems/api/v1/dependencies?gems=bundler HTTP/1.1" 200 14018 "-" "bundler/1.16.1 rubygems/2.7.6 ruby/2.5.1 (x86_64-pc-linux-gnu) command/install options/no_install,wro2096v.centrala.bzwbk 0861e96b13b2dd66"

[segiddins]

These are not incorrect calls, a gem repository should either return the appropriate response (see RubyGems.org for the canonical example), or a 404

[maras1]

I rely on the information indicated in https://guides.rubygems.org/rubygems-org-api/

1. The following methods for the version resource are described there:
   GET - /api/v1/versions/[GEM NAME].(json|yaml)
   GET - /api/v1/versions/[GEM NAME]/latest.json
   I do not see any Get / versions method there.
2. The following methods for the dependencies resource are described there:
   GET - /api/v1/dependencies?gems=[COMMA DELIMITED GEM NAMES]
   IMHO sending rest api call without specifying a gem name is incorrect.

[segiddins]

The compact index API might be undocumented on that page

[deivid-rodriguez]

Updated the title to reflect what would need to be done here.

[decoyjoe]

The Compact Index API, as implemented in rubygems/compact_index, is still undocumented on https://guides.rubygems.org/rubygems-org-api/

Now that the Dependency API is going away 2023-05-10, the Compact Index API should be documented.

[coldnebo]

I'm not sure I understand the discussion here. We are running the bundler 2.4.13 and it is generating these calls and it takes 10 minutes to install or even check deps. This is a major performance impact. Are you saying Artifactory is wrong, or bundler is wrong? The fallback isn't falling back to the compact api properly. Is there a workaround?

$ bundle install --verbose
Running `bundle install --verbose` with bundler 2.4.13
Found changes from the lockfile, re-resolving dependencies because the dependencies in your gemfile changed
HTTP GET [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/versions)
HTTP 404 Not Found [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/versions)
Bundler::Fetcher::FallbackError: Net::HTTPNotFound: [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/versions)
/DONOTDELETE/shared/rvm/gems/ruby-3.1.2@app/gems/bundler-2.4.12/lib/bundler/fetcher/downloader.rb:45:in `fetch'
  /DONOTDELETE/shared/rvm/gems/ruby-3.1.2@app/gems/bundler-2.4.12/lib/bundler/fetcher/compact_index.rb:124:in `call'
...
HTTP GET [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/api/v1/dependencies)
HTTP 404 Not Found [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/api/v1/dependencies)
...
Fetching source index from [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/)
HTTP GET [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/quick/Marshal.4.8/amq-protocol-0.8.3.gemspec.rz)
HTTP 200 OK [http://repository/artifactory/api/gems/gems-repo...](http://repository/artifactory/api/gems/gems-repos/quick/Marshal.4.8/amq-protocol-0.8.3.gemspec.rz)

[coldnebo]

I'm moving my issue to #337 since I'm not sure it's related to the doc issue.

[coldnebo]

This is the issue that Artifactory opened with bundler. Their side of this issue is: https://jfrog.atlassian.net/browse/RTFACT-13635