(This is a repost of an old blog post of mine from Teknik. This blog post was split into multiple parts to accomodate for Movim's size limitations; use hypelinks in the table of contents to navigate.)

Why hasn’t the open source community standardized on a single implementation, i.e. SBCL?

Baggers

In short, because people have different needs. Folks who want to embed in C applications will benefit from ECL, people who work in a Java stack can benefit from ABCL people, etc..

Aidenn0

The same reason that the Open Source community standardized on a single Linux distribution (or even Unix kernel BSD, Illumos, Linux &c.)

alcahd

The community has mostly settled on SBCL, many projects (Stumwm, for example), have dropped support for less used implementations. And for good reason, SBCL is significantly faster than anything else. However, other implementations have their special use cases and can target non-x86 architectures.

Hexstream

It largely has. SBCL is by far the most popular and best-supported Common Lisp implementation right now, and most newbies should use it by default unless they have very specific needs.

Xach

I used to think [that the community has mostly settled on SBCL], but the more experience I get, the less I think it is true. Every active CL has a number of users that cannot or would not switch to SBCL. They'd be more likely to stop using CL.

KDallas_Multipass

Me too. At the moment one thing I got used to with CCL was not having to know the size my heap would grow to before I started the image. I didn't even know that was a problem until I started trying sbcl with code that by design would consume a bunch of memory. I remember noticing that landing in the debugger in sbcl seemed nicer and richer than it would be when I was in ccl, and I'm aware of anecdotes about how sbcl might be a slower compiler for compiling faster code, and that it provides a lot of feedback that you can use to guide performance improvements. I appreciate that there are multiple mature implementations. I worry about the messages about how the windows implementation of sbcl is considered "unstable". I know if I had found sbcl first I would probably be saying different things.

phoe

Because it's much better and more varied this way. You can use one Common Lisp code and use it to reap whatever implementation's strengths you need to leverage at the moment. (Warning: this list of individual implementations' strengths is not exhaustive and is meant to be illustrative.)

• SBCL has a compiler producing very fast code, capable of e.g. using SSE CPU instructions and doing atomic compare-and-swaps.
• CCL has a fast compiler, as in, compilation is very quick there. It also has a native Cocoa bridge for OS X and has some nice debugging capabilities.
• ECL compiles from plain C and runs anywhere, including Android.
• CLISP also compiles from plain C and is an interpreter (optionally compiling to bytecode) that has very good native Unix interoperability and no compilation overhead, making it viable for scripting.
• Clasp interoperates between C++ and Common Lisp code, allowing you to marry these two completely different languages.
• Mezzano is a FREAKING OPERATING SYSTEM WRITTEN IN COMMON LISP THAT RUNS IRC, QUAKE AND DOOM.
• ABCL runs on the JVM, meaning that it runs everywhere Java does.
• mocl runs on iOS/Android and allows you to develop for mobile platforms.
• LispWorks allows you to write very portable GUI using its CAPI.
• ACL has a concurrent garbage collector, which means very small GC pauses.

Would there be any benefit to extending SBCL beyond the Common Lisp standard to better support modern language features and paradigms?

defaultxr

SBCL and most other CL implementations do include extensions. For example, threading is not described in the standard, but SBCL provides an extension to allow Lisp programs to use threads.

Baggers

As you've seen above, most implementations do extend. But it feels like the 'modern language features and paradigms' part of the question is suggesting updating the standard. One the beuties of CL (and some other members of the lisp family) is the fact you dont need to force the entire community to do something to get change. You can see projects like CL21 as an example, once again choice is key and the non-dictating nature of CL make it a good home for people with a wide range of priorities.

You can see things like grey-streams, cffi & bordeux-threads for examples of things that have become defacto standards in the community and also to projects like optima and named-readtables which provide feature to CL as libraries which in other languages would require a new version of the language.

It's also wonderful that, if you make a better way of working, you can release it as a library and let people decide if you were right :) and if they dont like it you lose nothing. That and that standards compliant code from 30 years back still works just fine as we've been gaining all these nice features.

There are some things that can't happen without buy in of every implementation. Here are a few examples.

Aidenn0

Some of this exists already. The lisp community is very diverse, so attempts to modernize the library are hard to do. There have been many attempts; see one recent discussion.

alcahd

As the for the CL standard, basically every implementation already extends it, which is part of the reason there were portability issues in the past. The standard is another product of age and omits a number of things you'd expect from a modern language.

PuercoPop

Lisp is all about extensibiliy. Case in Point: CLOS. One can extend the object system in terms of the same object system. (Modularity not so much imho).

phoe

You are confusing two things.

First thing: You do not need to extend the implementation to better support (most) modern language features and paradigms. Common Lisp already allows you to do so on any implementation.

Lisp does not have RegEx. So what do you do? You implement it in Lisp! Lisp does not have contracts. So what do you do? You implement it in Lisp! Lisp does not have pattern matching. So what do you do? You implement it in Lisp! Lisp does not have asynchronous programming capabilities like promises etc. So what do you do? You implement it in Lisp! Hell, early Lisps did not have CLOS. So what do you do? You implement it in Lisp!

Second thing: the implementations were and are being extended to support hardware and OS features that were unavailable previously. The most important examples I can think of are threads, sockets, OS interoperability and SSE CPU instructions for numeric/array crunching.

fiddlerwoaroof

as far as the "extend the standard" question goes, my sense is that the Python community has discovered that the standard library is where libraries go to die. Once a library gets incorporated into a "standard", it can no longer develop at its own pace, but only at the pace of the various implementations of the standard.

Whereas, if you implement the library in terms of other language features and maintain it, it can evolve as necessary. See http://www.leancrew.com/all-this/2012/04/where-modules-go-to-die/

Supposedly Rust people have also found this. https://news.ycombinator.com/item?id=12177545

dzecniv

did you see http://cl21.org/ ? It's a very important effort to me. It brings more functional and generic stuff, and fixes many CL oddities. It's very usable right now. But as always doc is scarce or missing. This is a problem I'm not sure the community is well aware of.

So you could have a look at https://lispcookbook.github.io/cl-cookbook/cl21.html for cl21 details. And it's a community project hosted on github, so… ;)

lispm

I personally would not use cl21.org , since it has some of the worst design of any CL library I have seen. It's ugly and misguided. The code is bad and poorly designed. One takes a huge performance and functionality hit. One can look at each function and discuss its design. Something which actually was done during the CL design, which was based on knowledge of prior dialects. You can find that the designers of Common Lisp both integrated knowledge from other dialects (Maclisp, Interlisp, Standard Lisp, ZetaLisp, ...) and tried to design a language with purpose.

The thing was written when the author did understand only 20% of Common Lisp, which is not a good base for a language redesign. But this is a common pattern, people not knowing too much of the language and with exposure to languages like Perl, Ruby and others, first want to redesign Common Lisp. Often they do it in ways which directly violates the spirit of the language and its built-in ways to change and extend the language. The only worse thing are people who actually understand Common Lisp and use it to create personalized language constructs, where each construct got a lot of thought, but them sum of all the thoughts isn't elegant and not fit for wider use. I'm not naming people, but some can probably guess who I mean. ;-)

It only supports just four implementations, though claiming to be written in portable CL...

As its page says it's 'experimental'. Experiments are fine, but it's an experiment which needs a garbage collector. Throw it away and start new. (Side note: this is not limited to Common Lisp.)

Imagine someone with exposure to Wing-Tsun and Taekwondo is trying to learn traditional Karate. Now after a year he is starting to design his own version of Karate with more effective moves, reusing tricks from Wing-Tsun and Taekwondo. But how big is the chance that this person will have understood the spirit and foundations of Karate? Alternatively one could study Karate until one has mastered it and having a Dan grade (a black belt) and then being able to redesign the moves based on actual background knowledge and first-hand experience.

The speed hit [of cl21] is acceptable for a Ruby user, because Ruby is about 103 times slower than a typical Common Lisp natively-compiled implementation and thus cl21 becomes as slow as Ruby. Works fine as a scripting language though.

The author does not understand how actual implementations extend the language in ways that makes sense and what consequences this has for certain language constructs.

Redesigning (Common) Lisp is a popular sport and some invested a lot of time: Dylan, ISLisp, Eulisp, Arc, Clojure, CLtL3, CL21, ...

The author wrote a lot of software in a very short time, but he didn't took the time to learn the language on a deeper level, which would be required for a language (re)design. That works for him and some others, but please don't assume that everyone likes the design, the implementation or even its goals. The CL21 implementation and design has never been discussed on a wider level.

Promoting an experimental design as the base for general purpose libraries is possible, but don't be surprised when others point out its flaws. I'm also not a fan of the gradiose claim of a CL in the 21st century.

I couldn't even be motivated to port it to other implementations, because I would not want to use it.