Lambda System Source Code Conditionalization for the Falcon Processor *** DRAFT *** [internal document -- smh 8Aug88] SUMMARY: This proposal puts forth the starting point for a mechanism by which a single set of Lisp system source files may be conditionalized and maintained to compile and run on the Lambda, K, and other (known) processors. BACKGROUND: While certain files implementing either low-level system functionality or system-specific facilities may contain machine-dependent code (and therefore must be maintained separately), the bulk of system files that implement the LISP language and most LISP Machine tools are primarily machine-independent in their coding. This means that they require only occasional conditionalization, when either the target processor or run-time environment is not known at coding time. The existence and behavior of the cross compiler, however, adds significantly to the complexity required for a consistent conditionalizing mechanism. Run-Time Code Conditionalization When run-time parameters such as processor type and/or loaded system options are unknown to the code writer, conditionalizations must be written into the code itself to cover multiple contigencies. One way of providing such conditionalizations is to have code that explicitly checks the status of the machine at run time. This is a feasible alternative if the condition being checked is something which cannot be determined at compile time, such as whether a floating-point accelerator exists on the machine which will run the code. However, run-time machine status has no bearing on processor type conditionalization. There is no reason for code written to run on a K processor to test whether it is running on a Lambda. This `test' is performed implicitly at compile time. The normal way of invoking compile-time conditionalization is with the read-time conditionalizers #+ and #-. These examine whether :FEATURE is present on the system's *FEATURES* list. However, this mechanism is insufficient when conditionalizing for cross compilation. [Note: For the purposes of this discussion, a cross compiler is invoked to compile code written on a SOURCE MACHINE, with the intention that its compiled output code will be executed on a TARGET MACHINE (or a simulation thereof).] The conditionalization occurs at read time on the source machine, and so the target machine never sees any code that was not strictly source machine compatible. IMPLEMENTATION: An Extension To Run-Time Code Conditionalization A proposed extension to the #+ and #- reader macros would support cross compilation for multiple target processors. The TARGET macro would understand a new keyword symbol which performs conditionalization with regard to SI:*TARGET-FEATURES*. This is best illustrated by an example: (defconstant pi #+(target lambda) ;; Use this value if running on a Lambda: 3.14285s0 #+(target falcon) ;; Use this value if running on a Falcon: 22/7 #-(target (or lambda k)) ;; Use this value if running on neither: 3.1415926s0) The new variable *TARGET-FEATURES* is created for cases in which the cross-compiler encounters a #+(target ...) or #-(target ...) construct. At these times, it sets up a lexical environment in which the *TARGET-FEATURES* variable is bound to the indicated machine's features list, thus enabling the compiler to simulate a conditionalization as if being executed on the indicated machine. The reader will continue to use the regular source machine-specific features list for all non-cross-compiler cases of code conditionaliza- tion; conceptually, for all such "regular" cases, the target machine is identical to the source machine. For completeness, the syntax #+(LOCAL ...) is also understood and forces conditionalization to source machine compatibility. It would be used in the unlikely event that some feature or property of the source machine were needed for a conditionalization occurring inside a "#-LOCAL" (read, "To be compiled for a machine other than the source machine") cross-compilation environment. The following table indicates the current initial values of the local features lists for the Lambda and Falcon systems: Machine Lambda Falcon Feature List Name *FEATURES* COMPILER:*FALCON-FEATURES* Initial Features :LAMBDA :FALCON :LEXICAL :LEXICAL :COMMONLISP :COMMONLISP :LOOP :LOOP :DEFSTRUCT :DEFSTRUCT :LISPM :LISPM :MIT :MIT :LMI :LMI :COMMON :COMMON :CHAOS :SORT :SORT :FASLOAD :FASLOAD :STRING :STRING :NEWIO :NEWIO :ROMAN :ROMAN :TRACE :TRACE :GRINDEF :GRINDEF :GRIND :GRIND Note that the initial Falcon Features list is identical to the initial Lambda Features List, except that the Lambda ":CHAOS" feature is not present in the Falcon list, and the ":LAMBDA" feature is replaced by the ":FALCON" feature. The conditionalization symbol used to ensure Lambda processor compatibility on should always be #+LAMBDA. The conditionalization symbol for the Falcon (and all related processor names, such as "K" and "Phoenix") should always be #+FALCON, as later versions of the Falcon system may use a different conditionalization symbol. Failure to conform at this stage will add to the complexity of any future switchovers. Certain functions are shared by the Lambda and Falcon compilers. An example of such code is the optimizer. Such code should use calls to ECASE to dispatch on the value of the variable COMPILER:*TARGET-COMPUTER*, which will always have either the value COMPILER::K or the value COMPILER::LAMBDA-INTERFACE. It is still open what we should use to conditionalize (for example) for the operating-system host of the Falcon processor. This could be conditionalized either at compile time or at run time. Since space is at a premium, we lean towards compile-time conditionalization; however, we have not yet addressed the issue in reality. ==== Compilation Environments ==== The regular Lambda compiler now supports a new type of structure, the COMPILER:COMPILATION-ENVIRONMENT. It is implemented as a defstruct that contains a couple internal hash tables. It is a place where definitions established during a compilation can be stored separate from the running environment of the compiling machine. For instance, suppose the Lambda and K environments require different expansions of the MULTIPLE-VALUE-BIND macro. The two definitions can be conditionalized using #+ reader macros. However, when cross compiling (e.g. compiling K code on the Lambda) one can't just redefine the Lambda's MULTIPLE-VALUE-BIND because that will cause the Lambda to lose big the next time some native code is compiled for the lambda. So when doing a file compilation, the compiler stores macro definitions and symbol properties inside a compilation-environment structure which is freshly created for that compilation, and which is discarded after the compilation. This is actually not a new mechanism. The existing Lambda compiler maintained several association lists during a file compilation which carried the same data. The new mechanism is more efficient, and permits long-term storage of definitions. The CROSS-COMPILE-FOR-K cross compiler produces two output files. The Falcon compiled code file has the standard extension "FBIN". In addition, a `K Environment' file is also created with the standard extension "FENV". It is a QFASL-format file that contains all the file-local definitions accumulated in the compilation-environment file. When it is subsequently loaded into a world the definitions will be added to the COMPILATION-ENVIRONMENT structure in COMPILER:*COMPILATION-ENVIRONMENT*. Loading a FENV file into a cross-compiling Lambda world is analogous to loading the compiled FBIN into the Falcon world, were compilation being done natively. In other words, loading a FENV file into the Lambda makes available to the cross compiler the macro and constant definitions in a previously-compiled file. To make this work, COMPILATION-ENVIRONMENT structures are actually nested. Each one has a NEXT field, which is either NIL or some other COMPILATION-ENVIRONMENT from which it inherits additional definitions. The COMPILATION-ENVIRONMENT created for a COMPILE-FILE is automatically nested inside the current binding of COMPILER:*COMPILATION-ENVIRONMENT*. This simulates the inheritance of the definitions in compiled files previously loaded into the world. This mechanism happens automatically, so developers probably won't have to worry about it very much. Nesting of environments can occur to any depth, but it is doubtful whether there will ever be any use for more than two levels. However, COMPILE-FILE and QC-FILE now accept an explicit keyword EXPLICIT-COMPILATION-ENVIRONMENT argument in case the MAKE-SYSTEM facility finds use for more control. The following are the developer-visible functions which support cross compilation. The function COMPILE-FILE-FOR-FALCON is just like COMPILE-FILE but invokes the cross compiler instead, after binding *COMPILATION-ENVIRONMENT* to the value stored in COMPILER:*FALCON-ENVIRONMENT*. The version numbers of the FBIN and FDEF files will be the same as the source file, just as is done for QFASL files. COMPILER:*FALCON-ENVIRONMENT* is a variable which holds a COMPILATION-ENVIRONMENT structure. This accumulates definitions from all the FDEF files that have ever been loaded. The function LOAD-FALCON-DEFINITIONS is just like load except that it defaults the pathname type to FDEF and binds *COMPILATION-ENVIRONMENT* to the value in COMPILER:*FALCON-ENVIRONMENT*.