How to install a Drupal Composer based project when packages.drupal.org is down

Submitted by Frederic Marand on
PHP Tips and tricks
Drupal tips and tricks

The problem: packages.drupal.org broken

I was starting my weekly work for the Drupal GraphQL module by the customary composer update --prefer-source -vvv command, ready to watch composer spit out some hundred lines sipping my coffee, but this time something turned out to be wrong:

  [...snip...]
Downloading https://packages.drupal.org/8/drupal/provider-2016-4%24a30289dd8394e5271bd77777bb14b361c5938656f1cddad7fae1c00d5d6ba9c6.json

  [Composer\Downloader\TransportException]
  The "https://packages.drupal.org/8/drupal/provider-2016-4%24a30289dd8394e5271bd77777bb14b361c5938656f1cddad7fae1c00d5d6ba9c6.json" file could not be downloaded (HTTP/1.1 404 Not Found)
  [...snip...]

OK, so packages.drupal.org is down for now. How can we work around this ?

The diagnostic

By default, most of my Drupal 8 project (aren't yours ?) are either based on drupal-composer/drupal-project for actual projects, or a slightly modified drupal/drupal when working on Drupal core or contrib modules like GraphQL. This was the case of the latter model, done by adding specific repositories entries to core composer.json, like:

    "repositories": {
        "fgm_graphql": {
            "type": "vcs",
            "url": "https://github.com/FGM/graphql-drupal"
        },
        "drupal": {
            "type": "composer",
            "url": "https://packages.drupal.org/8"
        }
    },

The explicit vcs entry on top is to enable me to work on my custom fork of the module (from which I send pull requests), no problem here. The issue is with the bottom one, used to download the other package in this project, specifically devel.

When working with such composer repositories, what Composer does is fetch a packages.json path below the url parameter, in this case https://packages.drupal.org/8/packages.json. That files is a list of data providers, basically one more level of metadata about available repositories and distributions (releases), looking like this:

{
   "notify-batch":"\/8\/downloads",
   "providers-url":"\/8\/%package%$%hash%.json",
   "search":"\/8\/search.json?s=%query%",
   "provider-includes":{
      "drupal\/provider-2017-1$%hash%.json":{
         "sha256":"67abb1f57bb826754ae5f4f877f92d2b1b58bfd45a56f4de883f2424be6cf8d5"
      },
      "drupal\/provider-2016-4$%hash%.json":{
         "sha256":"a30289dd8394e5271bd77777bb14b361c5938656f1cddad7fae1c00d5d6ba9c6"
      },
     [ ...snip ...]
      }
   }
}

Composer will then download from each of these providers in turn, hence the URL displayed as returning a 404 at the top of this story. Sure enough, manual checking of the URL returned a 404, even with a cache-buster query added.

Sure enough, the issue was already mentioned on IRC on #drupal : what could be done at this point without being able to touch packages.drupal.org itself ?

The solution: skip one layer

Actually, the answer is already present in the existing repositories clause: vcs-type repositories do not need a "directory"-type service like Packagist or packages.drupal.org, because in essence what these directories do is provide a way to locate sources and dists. But vcs provide the same service, with the limitation that they have to be listed for each project. So let us skip the composer directory and list devel's repository directly:

    "repositories": {
        "fgm_graphql": {
            "type": "vcs",
            "url": "https://github.com/FGM/graphql-drupal"
        },
        "devel": {
            "type": "vcs",
            "url": "https://git.drupal.org/project/devel.git"
        }
    },

Now, a composer require drupal/devel --prefer-source -vvv works normally, no longer needing to parse the broken directory:

Reading ./composer.json
Loading config file /Users/fgm/.composer/auth.json
Loading config file ./composer.json
[...snip...]
Executing command (/Users/fgm/.composer/cache/vcs/https---git.drupal.org-project-devel.git/): git show-ref --tags --dereference
Executing command (/Users/fgm/.composer/cache/vcs/https---git.drupal.org-project-devel.git/): git branch --no-color --no-abbrev -v
Executing command (/Users/fgm/.composer/cache/vcs/https---git.drupal.org-project-devel.git/): git branch --no-color
Executing command (/Users/fgm/.composer/cache/vcs/https---git.drupal.org-project-devel.git/): git show '8.x-1.x':'composer.json'
Executing command (/Users/fgm/.composer/cache/vcs/https---git.drupal.org-project-devel.git/): git log -1 --format=%at '8.x-1.x'
Reading composer.json of drupal/devel (5.x-0.1)
Skipped tag 5.x-0.1, invalid tag name
[...snip...]
Reading composer.json of drupal/devel (8.x-1.x)
Reading /Users/fgm/.composer/cache/repo/https---git.drupal.org-project-devel.git/70f62fd0773082a1a4305c6a7c2bccc649bc98a2 from cache
Importing branch 8.x-1.x (dev-8.x-1.x)
[...snip...]
(success)

Time to return to actual work :-)