(This is a repost of an old blog post of mine from Teknik.)

#ultraspec #clus #common-lisp #documentation

Foreword/Confession

tl;dr I have started the Common Lisp UltraSpec project some time ago. I am responsible for it being stuck in place.

When I started the CLUS, as a one-man project back in 2016, I have never expected that it would generate so much, how do you say it in English. Echo? Answers? Agreement? Need, hidden by years of being stuck to the CLHS? I don't exactly know, but even at the European Lisp Symposium 2017 I presented CLUS at, I got multiple offers for help and support, even financial. It was obvious that the Lisp world in 2017 needs such a project

Then real life came. I don't really want to excuse myself in this blog post since I could have organized my life better and worked on CLUS easier, but I needed to straighten out multiple things that required attention. I also needed to rest, relax and reorganize larger chunks of my life - CLUS as a project was pushed aside, in a very messy, unfinished and, lo and behold, undocumented state.

Thanks to Michael Fiano, who prodded me back into life, I have decided that it's been enough time of not accepting help from all the people who offered me their help with CLUS. It's time to make this project fully alive again.

I hereby offer this short writeup as something that describes the current state of CLUS and will, along with my further work, serve as a basis of pulling the UltraSpec out of the stagnancy that I allowed it to slip into.

TODO

Diffs

tl;dr What are the differences between CLUS text and dpANS3 text?

One of the weak points of my process of converting TeX into DokuWiki markup was - I fixed various things and tidbits along the way. This means that CLUS right now is perhaps better documentation but it is no longer consistent with the specification, as there are differences at unknown points in text compared to the original text. It had become a task to find and list these differences.

Thanks to the monumental work by Kamil Ziemian and Ricardo M. H. da Silva, the diff files have been generated and are now being reviewed, with the differences listed. Thanks to this effort, we might soon have this subtask done.

Format

tl;dr What format should CLUS use for its data?

One of the questions after my ELS talk was, is there a formal grammar for the DokuWiki format. Obviously, there's not, and the format itself is good enough for displaying things on DokuWiki, but quite bad for everything else. It is a combination of the original DokuWiki format and some ugly hacks that I have made to display it the way I wanted it to.

This format is not perfect and has its obvious flaws. I have been thinking of various formats: TeX, S-expressions, Markdown, DokuWiki, but all of them don't seem good enough for the job. Perhaps Markdown would work, with some extensions, but then we'd need means of parsing its extended form and displaying it on various places. The format should be advanced enough to handle displaying of monstrocities like Macro DEFSTRUCT - the single most complicated page of the specification.

This question is open and I will be glad to accept any sensible ideas.

Displaying engine

tl;dr How should CLUS be displayed to the end users?

DokuWiki is a wiki that I use for displaying CLUS content. It is used exclusively as a displaying engine that also has a search capability. It does the job, but is not perfect, and most notably, requires us to use DokuWiki format for the text files.

Are there any better ones around?

What next?

tl;dr What should be done after finishing the CL part of CLUS

As someone on Lisp Discord noted, the sidebar of CLUS contains a lot of topics that are not related to the CL standard - libraries, extensions, substandards, implementations, books. I originally imagined CLUS to be an effort to collect information from the whole Common Lisp universe. Obviously, we first need to finish CL in order to have a foundation, but my dream was CLUS unifying various pieces of documentation from all over the Lisp universe.

Decisions and conclusions

Live chat

We obviously need means of organizing work by means of a live chat where we can discuss matters

Courtesy of lonjil, we have linked the #clus-general channel of Lisp Discord server and #clus @ Freenode, so it is possible to discuss work both on IRC and Discord. This is where I have decided to settle for CLUS-related discussion.

You're invited.

Splitting of work

There are several areas in which CLUS needs help - outlined above in the TODO section. I do not want to organize how work should be split among willing people in this post, as it's much better discussed live, on the IRC/Discord channel linked above - and that's what I'd like to focus on during the next few days.

Follow-up

I will write more on the topic in 48 hours from now. (lonjil, prepare your killswitch.)