(This is a repost of an old blog post of mine from Teknik. This blog post was split into multiple parts to accomodate for Movim's size limitations; use hypelinks in the table of contents to navigate.)

Why don’t libraries have version numbers?

defaultxr

They do. Typically declared in the .asd file. It's up to the library's author to specify/update those version numbers of course.

Xach

Everything uses the latest of everything.

Quicklisp makes it pretty easy to use libraries of the same vintage that are known to work together.

qlot also makes it pretty easy to use a variety of upstream sources and stick to a known-working set of libraries, and reproduce that set on other machines. (I haven't personally used qlot, but it seems like a sensible idea.)

ruricolist

Quicklisp provides more assurance than you might expect. The criterion for inclusion, and maintenance, of a project in Quicklisp is that the project successfully load in SBCL alongside all the libraries in that Quicklisp distribution, and load without any errors or warnings. SBCL has extensive type inference and can catch and warn about many potential issues at compile time. And because of the pervasive use of macros in CL, successfully loading a library usually exercises a lot of code paths in its dependencies. To a surprising extent, "if it loads, it runs."

Qlot isn't something a library would use. You use it to set up the dependencies for an application. My approach (with TBRSS) is this. Obviously, every time I upgrade to a new Quicklisp dist, I run a test suite to make sure everything is working together. On the rare occasion there's a problem, I either pin the offending library to an older version (with Qlot) or I fork it and fix it, again using Qlot to pull from the fork until the fix makes it into the next Quicklisp dist. And of course I also use Qlot for dependencies that are not, for whatever reason, available in Quicklisp.

Aidenn0

They do. In fact all ASDF systems have a version number. Just very few people use them.

phoe

The question is both incorrect, in a way, and correct, in a way.

The incorrect fact: as we can see in a ASDF Manual - ASD systems each have the capability to declare a version number, and multipleASDsystems use this facility.

The correct intuition: some systems simply do not use it, with the most notable example being the most popular Quicklisp library - Alexandria. Additionally, manysystems simply do not use versioned dependencies and instead opt for using the latest released version of a given library.

This might look very different compared to public repositories like NPM. I will list the main differences that I see, which make this kind of approach viable:

• The JS/NPM community is much larger than the CL/Quicklisp community,
• The code volume in NPM is hundreds, if not thousands of times larger than in Quicklisp,
• The libraries are changed and modified much quicker and more often in the JS world compared to the CL world.

This way, there is not much space where breakage may occur the moment a change in a dependency breaks projects that depend on it. Though, I'm not the most proper person to talk about this; Zach Beane runs monthly tests on whole Quicklisp whenever he updates the Quicklisp dists, and because of that, he has most likely experienced much more in that topic than I have.

Why is adding libraries to Quicklisp a manual process that requires human review?

defaultxr

Quicklisp is mostly the work of one person, and it's still considered to be in beta from what I understand. I don't think Quicklisp is "curated" in a strict sense but it does require the libraries to actually compile and not print any warnings during compilation.

Aidenn0

Because there are few enough libraries that there hasn't been a scaling problem yet.

phalp

I think it's very good that Quicklisp is curated. Without curation, you get frivolous libraries like left-pad, and you get typo-squatting malicious packages like we heard about in npm last week. You can't assume a package is reliable just because it's in Quicklisp, but at least you can have some faith in the package list itself.

phoe

Because it was designed that way by the person responsible for all of Quicklisp - Zach Beane. The current process of adding a Quicklisp project into the repositories is a rather quick and painless one - there is no actual review of code and projects other than the two sole minimal assumptions that the project builds on SBCL x64 on Linux and has the ASDF author/name/license data correctly filled in.

Hexstream

The comment above is mostly right, but SBCL-only (or single-implementation-only) libraries are frowned upon, at least. Also, I would say "builds without errors or warnings".

Is there an alternative non-curated registry?

Aidenn0

Anybody can create a quicklisp "distribution" (which is a registry), though I'm not currently aware of any public ones.

phoe

In theory: everyone can create their own Quicklisp dist, perhaps with a backend that allows automation.

In practice: nobody I know has done that yet because of lack of demand for it. Quicklisp is simply good enough.

EDIT: Check out UltraLisp.

Hexstream

I've come across at least 2 custom Quicklisp dists recently, one was the cl21 one, I don't remember what was the other one... But yeah, extremely rare. They're partial dists with a few projects, though, not full-fledged quicklisp alternative dists.

Why aren’t ASDF dependencies automatically fetched if missing?

defaultxr

That's not what ASDF is for. ASDF is like make, Quicklisp is like your OS's package manager. Different tools for different tasks... However in the past ASDF did have functionality to download libraries that were missing (I believe it was called ASDF-Install?) but nowadays that is deprecated in favor of Quicklisp which is more complete and robust, from what I understand.

Aidenn0

They are if you are using Quicklisp. That being said, ASDF is solving a narrower problem of managing already installed dependencies. Quicklisp solves the problem of automatic fetching (which requires some sort of registry of packages).

phoe

ASDF is a local system manager and a build tool, not unlike make. Quicklisp is an online repository that automatically fetches dependencies and tells ASDF where to find them for compilation.

Why do so few libraries use GitHub? / My apologies, but I’d never even heard of GitLab prior to seeing all these libraries hosted on common-lisp.net. Why not GitHub?

defaultxr

Common Lisp has been around a long time, and many older libraries predate GitHub.

arvid

Most of them do use github. 1251 out of 1582 or 79% quicklisp projects use github. See the list here.

Aidenn0

Because so many libraries are older than GitHub.

GitLab is at least somewhat well known. The really short answer is that GitLab is open source. Some projects that were hosted on common-lisp.net prior to github existing have moved to github. Others prefer to be hosted on common-lisp.net; for most of these projects SourceForge, not GitHub was the alternative to common-lisp.net when this decision was made.

kazkylheku

There are good reasons for open source project not to use GitHub.

GitHub is a company which operates a closed-source "walled garden" website with git hosting (which anyone can do with a host running GNU/Linux or BSD) plus various additional proprietary services.

If GitHub disappears, all you have is your git repos, because they are cloned all over the place; what about all the other GitHub specific data that you and your users put effort into?

Please drop the obnoxious mentality of questioning people who don't use GitHub.

It's not some axiom of the universe that "thou shalt use GitHub".

(Or even git, for that matter. Git is not the best version control tool in all requirements. One excellent technical reason for someone not to use Github is that they don't use Git.)

alcahd

Github's software is notably non-free. I find this contradictory and prefer not to use it where possible. Gitlab also has a few nice features not seen on Github such as the ability to run your own instance on a server. However, the lack of libraries on Github is mostly a symptom of age, rather than any conscious choice.

dzecniv

About github, to me the problem is to not have open registrations. Common-lisp.net uses their own gitlab, why not, but the registration is not open (one must ask) and so it drastically slows down or prevents participation.

The pb to me is using common-lisp.net where registrations are not open. This is a big pb that slows drastically participation down.

pjb

One problem is that github is github.COM and is located in the USA; gitlab on the other hand was gitorious.ORG and was located in Europe. At a time when network or github failures meant that European workers were stuck half a day without being able to push to the central repository, and considering that most CL libraries are freedom software, and not privative COMmercial software, it would seem more logical to gitorious.ORG than github.COM.

On the other hand, nowadays, gitorious has been bought by gitlab.COM, but there's framagit.ORG which is located in France.

There are also licensing restrictions that were imposed by github with some self generated FUD. (It seems they're cleared it up, but still, the point remains that if you use only one such repository, you're at the mercy of the US enterprise and their shareholder).

phoe

Why should they? As long as they are hosted in a place where they are fetchable, then they are reachable and downloadable by Quicklisp. Additionally, as long as they are hosted on some tool that allows GUI cooperation, such as Gitlab, Gogs or Gitea, then it is possible to create and submit pull requests to them in a convenient way.

It might not be fully related here, but the RedMonk graph says we're above the curve when it comes to GitHub repos.