[Scheme-reports] Fixing libraries (was Re: Questions about cond-expand)

[Aaron W. Hsu <arcfide@x>]

Dear Working Group:

I hope to bring this discussion to the fore as I believe that we as a 
group have made some errors in the approach that we have taken to 
dealing with these libraries. I believe we have a chance at this moment 
to rectify the situation and I hope that we will do so.  

As a history of the issue as I see it, we have a library system 
that we have defined in special language requiring it to be static. 
We have also defined a set of library clauses which are static.
Additionally, we have voted to allow specific exception that allow 
for these forms to appear "at the top-level" but we are not clear 
about this, and the language is ambiguous. Moreover, this will 
likely lead to at least three different concepts of forms like 'include' 
in any reasonable Scheme implementation.

However, I argue that this is not how Scheme implementations currently 
do this, and moreover, that we have an opportunity to create a simplified 
approach to this that is at the same time richer and more unified. 
It is also backed up by existing implementation strategy and does 
not restrict the implementation strategy of Schemes which may have a 
slightly different picture than presented here, because the 
strategy taken by these Schemes is observationally equivalent to the 
behavior required by my suggestions here.

We really have two problems. Firstly, we have a library system that 
is completely outside the rest of the standard, and is unified in no 
ways with the rest of the system. This requires special language to deal 
with this fact. Secondly, we have restricted the system to such a point 
and in such language that users are unnecessarily restricted. 

I would like to introduce my suggested revisions that fix these 
problems and clean up the language and standard by first talking 
about what an existing Scheme does to accomplish this. In Chez Scheme,
The library form is just another Scheme form, but it accurately implements 
the semantics of the R6RS library form. Moreover, the export, 
import, and include forms, among others are also valid Scheme forms.
These work as might be expected as forms. The library form has its 
own special clauses for export and import (which are not the same, 
but behave the same, as the export and and import forms of the language).
In other words, Chez Scheme has provided the entire library system 
with no further language complication than was necessary to introduce and 
define the meaning of the forms.

One of the nice consequences of this is that libraries can be used 
to implement richer systems like a library system that separates 
implementation from interface, or one that stores documentation and 
so forth. This is an important principle in Scheme. User created 
extensions should not be underpriveleged. They should work and feel 
just like an implementation built-in. We should be able to implement 
our own versions of things using possibly simpler primitives provided 
by the Scheme language. These versions should play nicely, within 
reason, with each other.

We can clean up the entire mess that we have created here by changing 
a few things. By making these changes, we will be able to unify the 
library system and its parts with the rest of the language.

1. Define library as just another Scheme form that creates a 
close world. We can keep the syntax and such that we have already 
chosen, and we can also keep the requirement that these clauses 
of a library are static and not affected by the world created for 
code by the library.

Making this change immediately removes the need to think about 
the library form as a static thing separate from the rest of the 
Scheme world. However, it still makes it a top-level, static form 
that is open to analysis. 

2. Define 'include', 'import', and the other forms as regular Scheme 
forms which have their obvious semantics. This means that we can 
do useful things like (let () (import ...) ...) and the like. This 
immediately makes our language richer, and provides primitives that 
are not implementable in the R7RS Small language. More importantly 
for this discussion, not only have we made the users life better, but 
we have also simplified the standard by eliminating the need to discuss 
these forms as separate, special entities for the top-level, because 
these forms can be used directly in any REPL. This gets rid of the 
problem we have right now that has introduced ambiguity.

3. Change the language of the 'library' form to define the behavior 
of the static library clauses in terms of their language form counter
parts. This unifies the semantics of these two forms and simplifies 
things down so that a single semantics can mostly suffice for both 
the static library forms and the Scheme language forms.

4. Unify the top-level of files and REPL to have at the minimum 
the 'library' and 'import' forms. This is all that is needed to 
have an usable system. That is to say, we define the environment of 
REPLs and files meaningfully, with at least this minimum intersection.
With this minimal section, systems that load files statically can actually 
still load libraries without any special expansion, because the only 
two forms that are in the environment are import and library. This 
means that we have a semantics that, while defined in terms of language 
forms, does not preclude implementations which are equivalent, but 
slightly different in approach.

5. We can create a separate (scheme libraries) library to hold these 
forms, so that they need not pollute the user level code unless 
the user wants them. This is not necessary, but could be a nice 
separation of features.

By making these changes we simplify the language of all the library 
system to require only talking about it as a special form of the language, 
and we can unify the library system with the normal concept of a 
Scheme environment and Scheme code, because a library is just 
Scheme code. We are friendly to implementors, who already must implement 
a top-level for the REPL. We provide more power to the users, and 
make it easier for WG2 to specify extensions to the library system that 
allow for user defined library extensions to exist comfortably with 
one another. Furthermore, we match the behavior of existing implementations 
better, without being overly specific about details that might 
force one implementation strategy over another. 

The overall effect is that we have removed and addressed the 
ambiguities mentioned beforehand, and have at the same time dealt 
with what I consider a glaringly ugly bit of standards writing on 
our part. The result is simpler and cleaner.

	Yours truly,

		Aaron W. Hsu

-- 

Aaron W. Hsu | arcfide@x | http://www.sacrideo.us
Programming is just another word for the lost art of thinking.

_______________________________________________
Scheme-reports mailing list
Scheme-reports@x
http://lists.scheme-reports.org/cgi-bin/mailman/listinfo/scheme-reports