An Appeal to All Package Managers

I work on Composer – a new PHP dependency manager – and it is now working quite well for managing PHP packages. We have a decent amount (and growing fast) of packages on Packagist. All is well. Yet, most PHP projects are websites, and they need some frontend libraries, be it JavaScript or CSS – I will use jQuery as an example that everyone can grasp easily. In some frameworks, you have plugins that bundle a copy of jQuery. Some people have also used Composer to hack together jQuery packages so that they can download it, and then they have some scripts to copy or symlink the files in the proper location. That is all very flaky, you end up with multiple copies of jQuery and if you are lucky you even get various different versions.

I have been thinking about it for a few months, and it seems like nothing exists out there. Every language out there has its own package manager, but everyone seems to be stuck with the same problem of frontend code. Obviously jQuery is used by virtually everyone. They can’t support Composer, Bundler, npm, pip and god knows what. Building a package manager for JS/CSS could work, but the community is huge and scattered and getting broad adoption is going to be very difficult.

The plan

As far as I can see, and given the way Composer works, it would be fairly easy to build a central package repository – similar to Packagist – where people can submit package definitions for frontend libs. For instance I could go and add a twitter bootstrap package, define the URL where you can download it including some placeholder for the version. Then all we need is a list of version that’s maintained. We could start by doing that by hand, or we can just point it to the git repo and it reads tags. That’s how Packagist works – except that it reads the composer.json file in the git repo to get all the metadata that in this case would be entered manually.

If we do this, we then end up with a central repository of CSS and JS packages, and we can integrate it in Composer, so that Composer packages can depend on jQuery and it just works. That would be a good start, but the great thing would be to get everyone on board. And I don’t mean everyone writing PHP. I mean everyone. The Ruby folks, the Python ones, Java, .NET, you name it. You all have package managers. All we have to do is agree on the API of the central package repository and on what metadata is needed. Then you can just add support for it in your package manager of choice, and we all benefit from the manual work put in to list packages. If it works, I’m sure some of the frontend packages will then add the metadata directly in their git/svn/.. repos so that we save on manual work. This would be a huge thing for everyone.

There are of course a few more details to settle regarding security and trust as well as exact package metadata, but I wanted to gauge the interest first, and then discuss further. I opened a frontend-packaging google group for that purpose, so if you are interested please join in. All it takes is a few open minded people and we could start one of the largest cross-language collaboration project ever. Sounds like fun!

May 31, 2012 by Jordi Boggiano in Development // Tags: , , , , 8 Comments

8 Responses to An Appeal to All Package Managers

  1. cordoval says:

    great proposal, for the time being what would be the best effective way to setup js/css on central repo to be lining up for this change in the future? with composer for instance? is there an example already up github?

  2. You might want to take a look at Gentoo Linux. I love the way they integrate all the different upstream code without actually copying it over to their server. You might find some inspiration there.

  3. gggeek says:

    In my own limited experience (with eZ Publish), bundling of jquery or yui is not a big problem: you never in fact deploy them on the webserver itself, but simply rely on google/yahoo providing them for you at runtime, with probably better download speed and uptime than you’d have anyway. The plugins then instruct the main application to insert appropriate html to load the js framework, and by themselves add the links to their own js code using it. The only problem with this is compatibility with different version numbers, eg. plugin A needs jquery 1.1, and plugin B needs jquery 1.2. But that’s what a dependency resolver should be able to check anyway

  4. Davert says:

    Have you tried volo? They have similar approach. I think you may contact it’s developers.

  5. Hi,

    Considering JS you might want to collaborate with the author of Volo. That gives you a nice way to deal with the JS deps.

    Basically you define a “package.json” (a bit like in NPM) at your GitHub repo and the tool ought to be able to take care of the rest.

  6. Dave says:

    Ok, I don’t want to come off as negative and shooting down enthusiasm, but I think you’re a couple of steps down the “solving the wrong problem” path here.

    I believe that dependency management (and hence package management) should happen at the OS level, not the language level. In other wordsstart by RPMing all the things, then this problem never arises. The last time you installed, say, phpmyadmin, how did you do it?

  7. David says:

    There is a relative new javascript package manager called bower