Copyright @copyright{} 1992 - 1996 Ben Wing.
Copyright @copyright{} 1996, 1997 Sun Microsystems.
-Copyright @copyright{} 1994 - 1998 Free Software Foundation.
+Copyright @copyright{} 1994 - 1998, 2002, 2003 Free Software Foundation.
Copyright @copyright{} 1994, 1995 Board of Trustees, University of Illinois.
@titlepage
@title XEmacs Internals Manual
-@subtitle Version 1.3, August 1999
+@subtitle Version 1.4, March 2001
@author Ben Wing
@author Martin Buchholz
@vskip 0pt plus 1fill
@noindent
-Copyright @copyright{} 1992 - 1996 Ben Wing. @*
+Copyright @copyright{} 1992 - 1996, 2001 Ben Wing. @*
Copyright @copyright{} 1996, 1997 Sun Microsystems, Inc. @*
Copyright @copyright{} 1994 - 1998 Free Software Foundation. @*
Copyright @copyright{} 1994, 1995 Board of Trustees, University of Illinois.
@sp 2
-Version 1.3 @*
-August 1999.@*
+Version 1.4 @*
+March 2001.@*
Permission is granted to make and distribute verbatim copies of this
manual provided the copyright notice and this permission notice are
@node Top, A History of Emacs, (dir), (dir)
@ifinfo
-This Info file contains v1.0 of the XEmacs Internals Manual.
+This Info file contains v1.4 of the XEmacs Internals Manual, March 2001.
@end ifinfo
@menu
* The XEmacs Object System (Abstractly Speaking)::
* How Lisp Objects Are Represented in C::
* Rules When Writing New C Code::
+* Regression Testing XEmacs::
* A Summary of the Various XEmacs Modules::
* Allocation of Objects in XEmacs Lisp::
* Dumping::
* General Guidelines for Writing Mule-Aware Code::
* An Example of Mule-Aware Code::
+Regression Testing XEmacs
+
A Summary of the Various XEmacs Modules
* Low-Level Modules::
* Modules for Interfacing with the Operating System::
* Modules for Interfacing with X Windows::
* Modules for Internationalization::
+* Modules for Regression Testing::
Allocation of Objects in XEmacs Lisp
@node A History of Emacs, XEmacs From the Outside, Top, Top
@chapter A History of Emacs
-@cindex history of Emacs
+@cindex history of Emacs, a
+@cindex Emacs, a history of
@cindex Hackers (Steven Levy)
@cindex Levy, Steven
@cindex ITS (Incompatible Timesharing System)
* XEmacs:: The continuation of Lucid Emacs.
@end menu
-@node Through Version 18, Lucid Emacs, A History of Emacs, A History of Emacs
+@node Through Version 18
@section Through Version 18
+@cindex version 18, through
@cindex Gosling, James
@cindex Great Usenet Renaming
version 18.59 released October 31, 1992.
@end itemize
-@node Lucid Emacs, GNU Emacs 19, Through Version 18, A History of Emacs
+@node Lucid Emacs
@section Lucid Emacs
@cindex Lucid Emacs
@cindex Lucid Inc.
and Eric Benson, and the work was later taken over by Jamie Zawinski,
who became ``Mr. Lucid Emacs'' for many releases.
- A time line for Lucid Emacs/XEmacs is
+ A time line for Lucid Emacs is
@itemize @bullet
@item
@item
version 20.3 (the first stable version of XEmacs 20.x) released November 30,
1997.
+@item
version 20.4 released February 28, 1998.
+@item
+version 21.1.2 released May 14, 1999. (The version naming scheme was
+changed at this point: [a] the second version number is odd for stable
+versions, even for beta versions; [b] a third version number is added,
+replacing the "beta xxx" ending for beta versions and allowing for
+periodic maintenance releases for stable versions. Therefore, 21.0 was
+never "officially" released; similarly for 21.2, etc.)
+@item
+version 21.1.3 released June 26, 1999.
+@item
+version 21.1.4 released July 8, 1999.
+@item
+version 21.1.6 released August 14, 1999. (There was no 21.1.5.)
+@item
+version 21.1.7 released September 26, 1999.
+@item
+version 21.1.8 released November 2, 1999.
+@item
+version 21.1.9 released February 13, 2000.
+@item
+version 21.1.10 released May 7, 2000.
+@item
+version 21.1.10a released June 24, 2000.
+@item
+version 21.1.11 released July 18, 2000.
+@item
+version 21.1.12 released August 5, 2000.
+@item
+version 21.1.13 released January 7, 2001.
+@item
+version 21.1.14 released January 27, 2001.
@end itemize
-@node GNU Emacs 19, GNU Emacs 20, Lucid Emacs, A History of Emacs
+@node GNU Emacs 19
@section GNU Emacs 19
@cindex GNU Emacs 19
+@cindex Emacs 19, GNU
+@cindex version 19, GNU Emacs
@cindex FSF Emacs
About a year after the initial release of Lucid Emacs, the FSF
working on and using GNU Emacs for a long time (back as far as version
16 or 17).
-@node GNU Emacs 20, XEmacs, GNU Emacs 19, A History of Emacs
+@node GNU Emacs 20
@section GNU Emacs 20
@cindex GNU Emacs 20
+@cindex Emacs 20, GNU
+@cindex version 20, GNU Emacs
@cindex FSF Emacs
On February 2, 1997 work began on GNU Emacs to integrate Mule. The first
version 20.3 released August 19, 1998.
@end itemize
-@node XEmacs, , GNU Emacs 20, A History of Emacs
+@node XEmacs
@section XEmacs
@cindex XEmacs
A more detailed history is contained in the XEmacs About page.
+ A time line for XEmacs is
+
+@itemize @bullet
+@item
+version 19.11 (first XEmacs) released September 13, 1994.
+@item
+version 19.12 released June 23, 1995.
+@item
+version 19.13 released September 1, 1995.
+@item
+version 19.14 released June 23, 1996.
+@item
+version 20.0 released February 9, 1997.
+@item
+version 19.15 released March 28, 1997.
+@item
+version 20.1 (not released to the net) April 15, 1997.
+@item
+version 20.2 released May 16, 1997.
+@item
+version 19.16 released October 31, 1997.
+@item
+version 20.3 (the first stable version of XEmacs 20.x) released November 30,
+1997.
+@item
+version 20.4 released February 28, 1998.
+@item
+version 21.0.60 released December 10, 1998. (The version naming scheme was
+changed at this point: [a] the second version number is odd for stable
+versions, even for beta versions; [b] a third version number is added,
+replacing the "beta xxx" ending for beta versions and allowing for
+periodic maintenance releases for stable versions. Therefore, 21.0 was
+never "officially" released; similarly for 21.2, etc.)
+@item
+version 21.0.61 released January 4, 1999.
+@item
+version 21.0.63 released February 3, 1999.
+@item
+version 21.0.64 released March 1, 1999.
+@item
+version 21.0.65 released March 5, 1999.
+@item
+version 21.0.66 released March 12, 1999.
+@item
+version 21.0.67 released March 25, 1999.
+@item
+version 21.1.2 released May 14, 1999. (This is the followup to 21.0.67.
+The second version number was bumped to indicate the beginning of the
+"stable" series.)
+@item
+version 21.1.3 released June 26, 1999.
+@item
+version 21.1.4 released July 8, 1999.
+@item
+version 21.1.6 released August 14, 1999. (There was no 21.1.5.)
+@item
+version 21.1.7 released September 26, 1999.
+@item
+version 21.1.8 released November 2, 1999.
+@item
+version 21.1.9 released February 13, 2000.
+@item
+version 21.1.10 released May 7, 2000.
+@item
+version 21.1.10a released June 24, 2000.
+@item
+version 21.1.11 released July 18, 2000.
+@item
+version 21.1.12 released August 5, 2000.
+@item
+version 21.1.13 released January 7, 2001.
+@item
+version 21.1.14 released January 27, 2001.
+@item
+version 21.2.9 released February 3, 1999.
+@item
+version 21.2.10 released February 5, 1999.
+@item
+version 21.2.11 released March 1, 1999.
+@item
+version 21.2.12 released March 5, 1999.
+@item
+version 21.2.13 released March 12, 1999.
+@item
+version 21.2.14 released May 14, 1999.
+@item
+version 21.2.15 released June 4, 1999.
+@item
+version 21.2.16 released June 11, 1999.
+@item
+version 21.2.17 released June 22, 1999.
+@item
+version 21.2.18 released July 14, 1999.
+@item
+version 21.2.19 released July 30, 1999.
+@item
+version 21.2.20 released November 10, 1999.
+@item
+version 21.2.21 released November 28, 1999.
+@item
+version 21.2.22 released November 29, 1999.
+@item
+version 21.2.23 released December 7, 1999.
+@item
+version 21.2.24 released December 14, 1999.
+@item
+version 21.2.25 released December 24, 1999.
+@item
+version 21.2.26 released December 31, 1999.
+@item
+version 21.2.27 released January 18, 2000.
+@item
+version 21.2.28 released February 7, 2000.
+@item
+version 21.2.29 released February 16, 2000.
+@item
+version 21.2.30 released February 21, 2000.
+@item
+version 21.2.31 released February 23, 2000.
+@item
+version 21.2.32 released March 20, 2000.
+@item
+version 21.2.33 released May 1, 2000.
+@item
+version 21.2.34 released May 28, 2000.
+@item
+version 21.2.35 released July 19, 2000.
+@item
+version 21.2.36 released October 4, 2000.
+@item
+version 21.2.37 released November 14, 2000.
+@item
+version 21.2.38 released December 5, 2000.
+@item
+version 21.2.39 released December 31, 2000.
+@item
+version 21.2.40 released January 8, 2001.
+@item
+version 21.2.41 released January 17, 2001.
+@item
+version 21.2.42 released January 20, 2001.
+@item
+version 21.2.43 released January 26, 2001.
+@item
+version 21.2.44 released February 8, 2001.
+@item
+version 21.2.45 released February 23, 2001.
+@item
+version 21.2.46 released March 21, 2001.
+@end itemize
+
@node XEmacs From the Outside, The Lisp Language, A History of Emacs, Top
@chapter XEmacs From the Outside
+@cindex XEmacs from the outside
+@cindex outside, XEmacs from the
@cindex read-eval-print
XEmacs appears to the outside world as an editor, but it is really a
@node The Lisp Language, XEmacs From the Perspective of Building, XEmacs From the Outside, Top
@chapter The Lisp Language
+@cindex Lisp language, the
@cindex Lisp vs. C
@cindex C vs. Lisp
@cindex Lisp vs. Java
@node XEmacs From the Perspective of Building, XEmacs From the Inside, The Lisp Language, Top
@chapter XEmacs From the Perspective of Building
+@cindex XEmacs from the perspective of building
+@cindex building, XEmacs from the perspective of
The heart of XEmacs is the Lisp environment, which is written in C.
This is contained in the @file{src/} subdirectory. Underneath
@node XEmacs From the Inside, The XEmacs Object System (Abstractly Speaking), XEmacs From the Perspective of Building, Top
@chapter XEmacs From the Inside
+@cindex XEmacs from the inside
+@cindex inside, XEmacs from the
Internally, XEmacs is quite complex, and can be very confusing. To
simplify things, it can be useful to think of XEmacs as containing an
frames are displayed correctly. It is periodically told (by the event
loop) to actually ``do its job'', i.e. snoop around and see what the
current state of the environment (mostly of the currently-existing
-windows, frames, and buffers) is, and make sure that that state matches
+windows, frames, and buffers) is, and make sure that state matches
what's actually displayed. It keeps lots and lots of information around
(such as what is actually being displayed currently, and what the
environment was last time it checked) so that it can minimize the work
@node The XEmacs Object System (Abstractly Speaking), How Lisp Objects Are Represented in C, XEmacs From the Inside, Top
@chapter The XEmacs Object System (Abstractly Speaking)
+@cindex XEmacs object system (abstractly speaking), the
+@cindex object system (abstractly speaking), the XEmacs
At the heart of the Lisp interpreter is its management of objects.
XEmacs Lisp contains many built-in objects, some of which are
@node How Lisp Objects Are Represented in C, Rules When Writing New C Code, The XEmacs Object System (Abstractly Speaking), Top
@chapter How Lisp Objects Are Represented in C
+@cindex Lisp objects are represented in C, how
+@cindex objects are represented in C, how Lisp
+@cindex represented in C, how Lisp objects are
Lisp objects are represented in C using a 32-bit or 64-bit machine word
(depending on the processor; i.e. DEC Alphas use 64-bit Lisp objects and
used for the Lisp object can vary. It can be either a simple type
(@code{long} on the DEC Alpha, @code{int} on other machines) or a
structure whose fields are bit fields that line up properly (actually, a
-union of structures is used). Generally the simple integral type is
-preferable because it ensures that the compiler will actually use a
-machine word to represent the object (some compilers will use more
-general and less efficient code for unions and structs even if they can
-fit in a machine word). The union type, however, has the advantage of
-stricter type checking. If you accidentally pass an integer where a Lisp
-object is desired, you get a compile error. The choice of which type
-to use is determined by the preprocessor constant @code{USE_UNION_TYPE}
-which is defined via the @code{--use-union-type} option to
-@code{configure}.
+union of structures is used). The choice of which type to use is
+determined by the preprocessor constant @code{USE_UNION_TYPE} which is
+defined via the @code{--use-union-type} option to @code{configure}.
+
+Generally the simple integral type is preferable because it ensures that
+the compiler will actually use a machine word to represent the object
+(some compilers will use more general and less efficient code for unions
+and structs even if they can fit in a machine word). The union type,
+however, has the advantage of stricter @emph{static} type checking.
+Places where a @code{Lisp_Object} is mistakenly passed to a routine
+expecting an @code{int} (or vice-versa), or a check is written @samp{if
+(foo)} (instead of @samp{if (!NILP (foo))}, will be flagged as errors.
+None of these lead to the expected results! @code{Qnil} is not
+represented as 0 (so @samp{if (foo)} will *ALWAYS* be true for a
+@code{Lisp_Object}), and the representation of an integer as a
+@code{Lisp_Object} is not just the integer's numeric value, but usually
+2x the integer +/- 1.)
+
+There used to be a claim that the union type simplified debugging.
+There may have been a grain of truth to this pre-19.8, when there was no
+@samp{lrecord} type and all objects had a separate type appearing in the
+tag. Nowadays, however, there is no debugging gain, and in fact
+frequent debugging *@emph{loss}*, since many debuggers don't handle
+unions very well, and usually there is no way to directly specify a
+union from a debugging prompt.
+
+Furthermore, release builds should *@emph{not}* be done with union type
+because (a) you may get less efficiency, with compilers that can't
+figure out how to optimize the union into a machine word; (b) even
+worse, the union type often triggers miscompilation, especially when
+combined with Mule and error-checking. This has been the case at
+various times when using GCC and MS VC, at least with @samp{--pdump}.
+Therefore, be warned!
+
+As of 2002 4Q, miscompilation is known to happen with current versions
+of @strong{Microsoft VC++} and @strong{GCC in combination with Mule,
+pdump, and KKCC} (no error checking).
Various macros are used to convert between Lisp_Objects and the
corresponding C type. Macros of the form @code{XINT()}, @code{XCHAR()},
nothing unless the corresponding configure error checking flag was
specified.
-@node Rules When Writing New C Code, A Summary of the Various XEmacs Modules, How Lisp Objects Are Represented in C, Top
+@node Rules When Writing New C Code, Regression Testing XEmacs, How Lisp Objects Are Represented in C, Top
@chapter Rules When Writing New C Code
+@cindex writing new C code, rules when
+@cindex C code, rules when writing new
+@cindex code, rules when writing new C
The XEmacs C Code is extremely complex and intricate, and there are many
rules that are more or less consistently followed throughout the code.
situations, often in code far away from where the actual breakage is.
@menu
+* A Reader's Guide to XEmacs Coding Conventions::
* General Coding Rules::
* Writing Lisp Primitives::
+* Writing Good Comments::
* Adding Global Lisp Variables::
+* Proper Use of Unsigned Types::
* Coding for Mule::
* Techniques for XEmacs Developers::
@end menu
-@node General Coding Rules, Writing Lisp Primitives, Rules When Writing New C Code, Rules When Writing New C Code
+@node A Reader's Guide to XEmacs Coding Conventions
+@section A Reader's Guide to XEmacs Coding Conventions
+@cindex coding conventions
+@cindex reader's guide
+@cindex coding rules, naming
+
+Of course the low-level implementation language of XEmacs is C, but much
+of that uses the Lisp engine to do its work. However, because the code
+is ``inside'' of the protective containment shell around the ``reactor
+core,'' you'll see lots of complex ``plumbing'' needed to do the work
+and ``safety mechanisms,'' whose failure results in a meltdown. This
+section provides a quick overview (or review) of the various components
+of the implementation of Lisp objects.
+
+ Two typographic conventions help to identify C objects that implement
+Lisp objects. The first is that capitalized identifiers, especially
+beginning with the letters @samp{Q}, @samp{V}, @samp{F}, and @samp{S},
+for C variables and functions, and C macros with beginning with the
+letter @samp{X}, are used to implement Lisp. The second is that where
+Lisp uses the hyphen @samp{-} in symbol names, the corresponding C
+identifiers use the underscore @samp{_}. Of course, since XEmacs Lisp
+contains interfaces to many external libraries, those external names
+will follow the coding conventions their authors chose, and may overlap
+the ``XEmacs name space.'' However these cases are usually pretty
+obvious.
+
+ All Lisp objects are handled indirectly. The @code{Lisp_Object}
+type is usually a pointer to a structure, except for a very small number
+of types with immediate representations (currently characters and
+integers). However, these types cannot be directly operated on in C
+code, either, so they can also be considered indirect. Types that do
+not have an immediate representation always have a C typedef
+@code{Lisp_@var{type}} for a corresponding structure.
+@c #### mention l(c)records here?
+
+ In older code, it was common practice to pass around pointers to
+@code{Lisp_@var{type}}, but this is now deprecated in favor of using
+@code{Lisp_Object} for all function arguments and return values that are
+Lisp objects. The @code{X@var{type}} macro is used to extract the
+pointer and cast it to @code{(Lisp_@var{type} *)} for the desired type.
+
+ @strong{Convention}: macros whose names begin with @samp{X} operate on
+@code{Lisp_Object}s and do no type-checking. Many such macros are type
+extractors, but others implement Lisp operations in C (@emph{e.g.},
+@code{XCAR} implements the Lisp @code{car} function). These are unsafe,
+and must only be used where types of all data have already been checked.
+Such macros are only applied to @code{Lisp_Object}s. In internal
+implementations where the pointer has already been converted, the
+structure is operated on directly using the C @code{->} member access
+operator.
+
+ The @code{@var{type}P}, @code{CHECK_@var{type}}, and
+@code{CONCHECK_@var{type}} macros are used to test types. The first
+returns a Boolean value, and the latter signal errors. (The
+@samp{CONCHECK} variety allows execution to be CONtinued under some
+circumstances, thus the name.) Functions which expect to be passed user
+data invariably call @samp{CHECK} macros on arguments.
+
+ There are many types of specialized Lisp objects implemented in C, but
+the most pervasive type is the @dfn{symbol}. Symbols are used as
+identifiers, variables, and functions.
+
+ @strong{Convention}: Global variables whose names begin with @samp{Q}
+are constants whose value is a symbol. The name of the variable should
+be derived from the name of the symbol using the same rules as for Lisp
+primitives. Such variables allow the C code to check whether a
+particular @code{Lisp_Object} is equal to a given symbol. Symbols are
+Lisp objects, so these variables may be passed to Lisp primitives. (An
+alternative to the use of @samp{Q...} variables is to call the
+@code{intern} function at initialization in the
+@code{vars_of_@var{module}} function, which is hardly less efficient.)
+
+ @strong{Convention}: Global variables whose names begin with @samp{V}
+are variables that contain Lisp objects. The convention here is that
+all global variables of type @code{Lisp_Object} begin with @samp{V}, and
+no others do (not even integer and boolean variables that have Lisp
+equivalents). Most of the time, these variables have equivalents in
+Lisp, which are defined via the @samp{DEFVAR} family of macros, but some
+don't. Since the variable's value is a @code{Lisp_Object}, it can be
+passed to Lisp primitives.
+
+ The implementation of Lisp primitives is more complex.
+@strong{Convention}: Global variables with names beginning with @samp{S}
+contain a structure that allows the Lisp engine to identify and call a C
+function. In modern versions of XEmacs, these identifiers are almost
+always completely hidden in the @code{DEFUN} and @code{SUBR} macros, but
+you will encounter them if you look at very old versions of XEmacs or at
+GNU Emacs. @strong{Convention}: Functions with names beginning with
+@samp{F} implement Lisp primitives. Of course all their arguments and
+their return values must be Lisp_Objects. (This is hidden in the
+@code{DEFUN} macro.)
+
+
+@node General Coding Rules
@section General Coding Rules
+@cindex coding rules, general
The C code is actually written in a dialect of C called @dfn{Clean C},
meaning that it can be compiled, mostly warning-free, with either a C or
@code{LIST_LOOP_DELETE_IF} delete elements from a lisp list satisfying some
predicate.
-@node Writing Lisp Primitives, Adding Global Lisp Variables, General Coding Rules, Rules When Writing New C Code
+@node Writing Lisp Primitives
@section Writing Lisp Primitives
+@cindex writing Lisp primitives
+@cindex Lisp primitives, writing
+@cindex primitives, writing Lisp
Lisp primitives are Lisp functions implemented in C. The details of
interfacing the C function so that Lisp can call it are handled by a few
@file{lisp.h} contains the definitions for important macros and
functions.
-@node Adding Global Lisp Variables, Coding for Mule, Writing Lisp Primitives, Rules When Writing New C Code
+@node Writing Good Comments
+@section Writing Good Comments
+@cindex writing good comments
+@cindex comments, writing good
+
+Comments are a lifeline for programmers trying to understand tricky
+code. In general, the less obvious it is what you are doing, the more
+you need a comment, and the more detailed it needs to be. You should
+always be on guard when you're writing code for stuff that's tricky, and
+should constantly be putting yourself in someone else's shoes and asking
+if that person could figure out without much difficulty what's going
+on. (Assume they are a competent programmer who understands the
+essentials of how the XEmacs code is structured but doesn't know much
+about the module you're working on or any algorithms you're using.) If
+you're not sure whether they would be able to, add a comment. Always
+err on the side of more comments, rather than less.
+
+Generally, when making comments, there is no need to attribute them with
+your name or initials. This especially goes for small,
+easy-to-understand, non-opinionated ones. Also, comments indicating
+where, when, and by whom a file was changed are @emph{strongly}
+discouraged, and in general will be removed as they are discovered.
+This is exactly what @file{ChangeLogs} are there for. However, it can
+occasionally be useful to mark exactly where (but not when or by whom)
+changes are made, particularly when making small changes to a file
+imported from elsewhere. These marks help when later on a newer version
+of the file is imported and the changes need to be merged. (If
+everything were always kept in CVS, there would be no need for this.
+But in practice, this often doesn't happen, or the CVS repository is
+later on lost or unavailable to the person doing the update.)
+
+When putting in an explicit opinion in a comment, you should
+@emph{always} attribute it with your name, and optionally the date.
+This also goes for long, complex comments explaining in detail the
+workings of something -- by putting your name there, you make it
+possible for someone who has questions about how that thing works to
+determine who wrote the comment so they can write to them. Preferably,
+use your actual name and not your initials, unless your initials are
+generally recognized (e.g. @samp{jwz}). You can use only your first
+name if it's obvious who you are; otherwise, give first and last name.
+If you're not a regular contributor, you might consider putting your
+email address in -- it may be in the ChangeLog, but after awhile
+ChangeLogs have a tendency of disappearing or getting
+muddled. (E.g. your comment may get copied somewhere else or even into
+another program, and tracking down the proper ChangeLog may be very
+difficult.)
+
+If you come across an opinion that is not or no longer valid, or you
+come across any comment that no longer applies but you want to keep it
+around, enclose it in @samp{[[ } and @samp{ ]]} marks and add a comment
+afterwards explaining why the preceding comment is no longer valid. Put
+your name on this comment, as explained above.
+
+Just as comments are a lifeline to programmers, incorrect comments are
+death. If you come across an incorrect comment, @strong{immediately}
+correct it or flag it as incorrect, as described in the previous
+paragraph. Whenever you work on a section of code, @emph{always} make
+sure to update any comments to be correct -- or, at the very least, flag
+them as incorrect.
+
+To indicate a "todo" or other problem, use four pound signs --
+i.e. @samp{####}.
+
+@node Adding Global Lisp Variables
@section Adding Global Lisp Variables
+@cindex global Lisp variables, adding
+@cindex variables, adding global Lisp
Global variables whose names begin with @samp{Q} are constants whose
value is a symbol of a particular name. The name of the variable should
@code{defsymbol()} are no problem, but some linkers will complain about
multiply-defined symbols. The most insidious aspect of this is that
often the link will succeed anyway, but then the resulting executable
-will sometimes crash in obscure ways during certain operations! To
-avoid this problem, declare any symbols with common names (such as
+will sometimes crash in obscure ways during certain operations!
+
+To avoid this problem, declare any symbols with common names (such as
@code{text}) that are not obviously associated with this particular
-module in the module @file{general.c}.
+module in the file @file{general-slots.h}. The ``-slots'' suffix
+indicates that this is a file that is included multiple times in
+@file{general.c}. Redefinition of preprocessor macros allows the
+effects to be different in each context, so this is actually more
+convenient and less error-prone than doing it in your module.
Global variables whose names begin with @samp{V} are variables that
contain Lisp objects. The convention here is that all global variables
Lisp object, and you will be the one who's unhappy when you can't figure
out how your variable got overwritten.
-@node Coding for Mule, Techniques for XEmacs Developers, Adding Global Lisp Variables, Rules When Writing New C Code
+@node Proper Use of Unsigned Types
+@section Proper Use of Unsigned Types
+@cindex unsigned types, proper use of
+@cindex types, proper use of unsigned
+
+Avoid using @code{unsigned int} and @code{unsigned long} whenever
+possible. Unsigned types are viral -- any arithmetic or comparisons
+involving mixed signed and unsigned types are automatically converted to
+unsigned, which is almost certainly not what you want. Many subtle and
+hard-to-find bugs are created by careless use of unsigned types. In
+general, you should almost @emph{never} use an unsigned type to hold a
+regular quantity of any sort. The only exceptions are
+
+@enumerate
+@item
+When there's a reasonable possibility you will actually need all 32 or
+64 bits to store the quantity.
+@item
+When calling existing API's that require unsigned types. In this case,
+you should still do all manipulation using signed types, and do the
+conversion at the very threshold of the API call.
+@item
+In existing code that you don't want to modify because you don't
+maintain it.
+@item
+In bit-field structures.
+@end enumerate
+
+Other reasonable uses of @code{unsigned int} and @code{unsigned long}
+are representing non-quantities -- e.g. bit-oriented flags and such.
+
+@node Coding for Mule
@section Coding for Mule
-@cindex Coding for Mule
+@cindex coding for Mule
+@cindex Mule, coding for
Although Mule support is not compiled by default in XEmacs, many people
are using it, and we consider it crucial that new code works correctly
* An Example of Mule-Aware Code::
@end menu
-@node Character-Related Data Types, Working With Character and Byte Positions, Coding for Mule, Coding for Mule
+@node Character-Related Data Types
@subsection Character-Related Data Types
+@cindex character-related data types
+@cindex data types, character-related
First, let's review the basic character-related datatypes used by
XEmacs. Note that the separate @code{typedef}s are not mandatory in the
and Extcounts are not all that frequent in XEmacs code.
@end table
-@node Working With Character and Byte Positions, Conversion to and from External Data, Character-Related Data Types, Coding for Mule
+@node Working With Character and Byte Positions
@subsection Working With Character and Byte Positions
+@cindex character and byte positions, working with
+@cindex byte positions, working with character and
+@cindex positions, working with character and byte
Now that we have defined the basic character-related types, we can look
at the macros and functions designed for work with them and for
@end example
@end table
-@node Conversion to and from External Data, General Guidelines for Writing Mule-Aware Code, Working With Character and Byte Positions, Coding for Mule
+@node Conversion to and from External Data
@subsection Conversion to and from External Data
+@cindex conversion to and from external data
+@cindex external data, conversion to and from
When an external function, such as a C library function, returns a
@code{char} pointer, you should almost never treat it as @code{Bufbyte}.
You'll get an assertion failure if you try.
-@node General Guidelines for Writing Mule-Aware Code, An Example of Mule-Aware Code, Conversion to and from External Data, Coding for Mule
+@node General Guidelines for Writing Mule-Aware Code
@subsection General Guidelines for Writing Mule-Aware Code
+@cindex writing Mule-aware code, general guidelines for
+@cindex Mule-aware code, general guidelines for writing
+@cindex code, general guidelines for writing Mule-aware
This section contains some general guidance on how to write Mule-aware
code, as well as some pitfalls you should avoid.
passed around in internal format.
@end table
-@node An Example of Mule-Aware Code, , General Guidelines for Writing Mule-Aware Code, Coding for Mule
+@node An Example of Mule-Aware Code
@subsection An Example of Mule-Aware Code
+@cindex code, an example of Mule-aware
+@cindex Mule-aware code, an example of
As an example of Mule-aware code, we will analyze the @code{string}
function, which conses up a Lisp string from the character arguments it
understood this section of the manual and studied the examples, you can
proceed writing new Mule-aware code.
-@node Techniques for XEmacs Developers, , Coding for Mule, Rules When Writing New C Code
+@node Techniques for XEmacs Developers
@section Techniques for XEmacs Developers
+@cindex techniques for XEmacs developers
+@cindex developers, techniques for XEmacs
+@cindex Purify
+@cindex Quantify
To make a purified XEmacs, do: @code{make puremacs}.
To make a quantified XEmacs, do: @code{make quantmacs}.
temacs -batch -l loadup.el run-temacs @var{xemacs-args...}
@end example
+@cindex error checking
Before you go through the trouble, are you compiling with all
debugging and error-checking off? If not, try that first. Be warned
that while Quantify is directly responsible for quite a few
If you want to make XEmacs faster, target your favorite slow benchmark,
run a profiler like Quantify, @code{gprof}, or @code{tcov}, and figure
-out where the cycles are going. Specific projects:
+out where the cycles are going. In many cases you can localize the
+problem (because a particular new feature or even a single patch
+elicited it). Don't hesitate to use brute force techniques like a
+global counter incremented at strategic places, especially in
+combination with other performance indications (@emph{e.g.}, degree of
+buffer fragmentation into extents).
+
+Specific projects:
@itemize @bullet
@item
@item
Speed up redisplay.
@item
-Speed up syntax highlighting. Maybe moving some of the syntax
-highlighting capabilities into C would make a difference.
+Speed up syntax highlighting. It was suggested that ``maybe moving some
+of the syntax highlighting capabilities into C would make a
+difference.'' Wrong idea, I think. When processing one 400kB file a
+particular low-level routine was being called 40 @emph{million} times
+simply for @emph{one} call to @code{newline-and-indent}. Syntax
+highlighting needs to be rewritten to use a reliable, fast parser, then
+to trust the pre-parsed structure, and only do re-highlighting locally
+to a text change. Modern machines are fast enough to implement such
+parsers in Lisp; but no machine will ever be fast enough to deal with
+quadratic (or worse) algorithms!
@item
Implement tail recursion in Emacs Lisp (hard!).
@end itemize
to implement such code fragments in C is with macros. But macros in C
are known to be broken.
+@cindex macro hygiene
Macro arguments that are repeatedly evaluated may suffer from repeated
side effects or suboptimal performance.
than a statement cannot use the techniques just described to avoid
multiple evaluation.
+@cindex inline functions
In most cases where a macro has function semantics, an inline function
is a better implementation technique. Modern compiler optimizers tend
to inline functions even if they have no @code{inline} keyword, and
compiling with @code{gcc -Wmissing-declarations}. I consider issuing
this warning for inline functions a gcc bug, but the gcc maintainers disagree.
+@cindex inline functions, headers
+@cindex header files, inline functions
Every header which contains inline functions, either directly by using
@code{INLINE_HEADER} or indirectly by using @code{DECLARE_LRECORD} must
be added to @file{inline.c}'s includes to make the optimization
Before submitting a patch, please try compiling at least once with
@example
-configure --with-mule --with-union-type --error-checking=all
+configure --with-mule --use-union-type --error-checking=all
@end example
Here are things to know when you create a new source file:
@file{.c} files should @code{#include "lisp.h"} second.
@item
-Generated header files should be included using the @code{#include <...>} syntax,
-not the @code{#include "..."} syntax. The generated headers are:
+Generated header files should be included using the @samp{#include <...>}
+syntax, not the @samp{#include "..."} syntax. The generated headers are:
@file{config.h sheap-adjust.h paths.h Emacs.ad.h}
-The basic rule is that you should assume builds using @code{--srcdir}
-and the @code{#include <...>} syntax needs to be used when the
+The basic rule is that you should assume builds using @samp{--srcdir}
+and the @samp{#include <...>} syntax needs to be used when the
to-be-included generated file is in a potentially different directory
-@emph{at compile time}. The non-obvious C rule is that @code{#include "..."}
-means to search for the included file in the same directory as the
-including file, @emph{not} in the current directory.
+@emph{at compile time}. The non-obvious C rule is that
+@samp{#include "..."} means to search for the included file in the same
+directory as the including file, @emph{not} in the current directory.
+Normally this is not a problem but when building with @samp{--srcdir},
+@file{make} will search the @samp{VPATH} for you, while the C compiler
+knows nothing about it.
@item
-Header files should @emph{not} include @code{<config.h>} and
-@code{"lisp.h"}. It is the responsibility of the @file{.c} files that
+Header files should @emph{not} include @samp{<config.h>} and
+@samp{"lisp.h"}. It is the responsibility of the @file{.c} files that
use it to do so.
@end itemize
+@cindex Lisp object types, creating
+@cindex creating Lisp object types
+@cindex object types, creating Lisp
Here is a checklist of things to do when creating a new lisp object type
named @var{foo}:
add an INIT_LRECORD_IMPLEMENTATION call to @code{syms_of_@var{foo}.c}
@end enumerate
-@node A Summary of the Various XEmacs Modules, Allocation of Objects in XEmacs Lisp, Rules When Writing New C Code, Top
+
+@node Regression Testing XEmacs, A Summary of the Various XEmacs Modules, Rules When Writing New C Code, Top
+@chapter Regression Testing XEmacs
+@cindex testing, regression
+
+The source directory @file{tests/automated} contains XEmacs' automated
+test suite. The usual way of running all the tests is running
+@code{make check} from the top-level source directory.
+
+The test suite is unfinished and it's still lacking some essential
+features. It is nevertheless recommended that you run the tests to
+confirm that XEmacs behaves correctly.
+
+If you want to run a specific test case, you can do it from the
+command-line like this:
+
+@example
+$ xemacs -batch -l test-harness.elc -f batch-test-emacs TEST-FILE
+@end example
+
+If something goes wrong, you can run the test suite interactively by
+loading @file{test-harness.el} into a running XEmacs and typing
+@kbd{M-x test-emacs-test-file RET <filename> RET}. You will see a log of
+passed and failed tests, which should allow you to investigate the
+source of the error and ultimately fix the bug.
+
+Adding a new test file is trivial: just create a new file here and it
+will be run. There is no need to byte-compile any of the files in
+this directory---the test-harness will take care of any necessary
+byte-compilation.
+
+Look at the existing test cases for the examples of coding test cases.
+It all boils down to your imagination and judicious use of the macros
+@code{Assert}, @code{Check-Error}, @code{Check-Error-Message}, and
+@code{Check-Message}.
+
+Here's a simple example checking case-sensitive and case-insensitive
+comparisons from @file{case-tests.el}.
+
+@example
+(with-temp-buffer
+ (insert "Test Buffer")
+ (let ((case-fold-search t))
+ (goto-char (point-min))
+ (Assert (eq (search-forward "test buffer" nil t) 12))
+ (goto-char (point-min))
+ (Assert (eq (search-forward "Test buffer" nil t) 12))
+ (goto-char (point-min))
+ (Assert (eq (search-forward "Test Buffer" nil t) 12))
+
+ (setq case-fold-search nil)
+ (goto-char (point-min))
+ (Assert (not (search-forward "test buffer" nil t)))
+ (goto-char (point-min))
+ (Assert (not (search-forward "Test buffer" nil t)))
+ (goto-char (point-min))
+ (Assert (eq (search-forward "Test Buffer" nil t) 12))))
+@end example
+
+This example could be inserted in a file in @file{tests/automated}, and
+it would be a complete test, automatically executed when you run
+@kbd{make check} after building XEmacs. More complex tests may require
+substantial temporary scaffolding to create the environment that elicits
+the bugs, but the top-level Makefile and @file{test-harness.el} handle
+the running and collection of results from the @code{Assert},
+@code{Check-Error}, @code{Check-Error-Message}, and @code{Check-Message}
+macros.
+
+In general, you should avoid using functionality from packages in your
+tests, because you can't be sure that everyone will have the required
+package. However, if you've got a test that works, by all means add it.
+Simply wrap the test in an appropriate test, add a notice that the test
+was skipped, and update the @code{skipped-test-reasons} hashtable.
+Here's an example from @file{syntax-tests.el}:
+
+@example
+;; Test forward-comment at buffer boundaries
+(with-temp-buffer
+
+ ;; try to use exactly what you need: featurep, boundp, fboundp
+ (if (not (fboundp 'c-mode))
+
+ ;; We should provide a standard function for this boilerplate,
+ ;; probably called `Skip-Test' -- check for that API with C-h f
+ (let* ((reason "c-mode unavailable")
+ (count (gethash reason skipped-test-reasons)))
+ (puthash reason (if (null count) 1 (1+ count))
+ skipped-test-reasons)
+ (Print-Skip "comment and parse-partial-sexp tests" reason))
+
+ ;; and here's the test code
+ (c-mode)
+ (insert "// comment\n")
+ (forward-comment -2)
+ (Assert (eq (point) (point-min)))
+ (let ((point (point)))
+ (insert "/* comment */")
+ (goto-char point)
+ (forward-comment 2)
+ (Assert (eq (point) (point-max)))
+ (parse-partial-sexp point (point-max)))))
+@end example
+
+@code{Skip-Test} is intended for use with features that are normally
+present in typical configurations. For truly optional features, or
+tests that apply to one of several alternative implementations (eg, to
+GTK widgets, but not Athena, Motif, MS Windows, or Carbon), simply
+silently omit the test.
+
+
+@node A Summary of the Various XEmacs Modules, Allocation of Objects in XEmacs Lisp, Regression Testing XEmacs, Top
@chapter A Summary of the Various XEmacs Modules
+@cindex modules, a summary of the various XEmacs
This is accurate as of XEmacs 20.0.
* Modules for Interfacing with the Operating System::
* Modules for Interfacing with X Windows::
* Modules for Internationalization::
+* Modules for Regression Testing::
@end menu
-@node Low-Level Modules, Basic Lisp Modules, A Summary of the Various XEmacs Modules, A Summary of the Various XEmacs Modules
+@node Low-Level Modules
@section Low-Level Modules
+@cindex low-level modules
+@cindex modules, low-level
@example
config.h
-@node Basic Lisp Modules, Modules for Standard Editing Operations, Low-Level Modules, A Summary of the Various XEmacs Modules
+@node Basic Lisp Modules
@section Basic Lisp Modules
+@cindex Lisp modules, basic
+@cindex modules, basic Lisp
@example
lisp-disunion.h
-@node Modules for Standard Editing Operations, Editor-Level Control Flow Modules, Basic Lisp Modules, A Summary of the Various XEmacs Modules
+@node Modules for Standard Editing Operations
@section Modules for Standard Editing Operations
+@cindex modules for standard editing operations
+@cindex editing operations, modules for standard
@example
buffer.c
-@node Editor-Level Control Flow Modules, Modules for the Basic Displayable Lisp Objects, Modules for Standard Editing Operations, A Summary of the Various XEmacs Modules
+@node Editor-Level Control Flow Modules
@section Editor-Level Control Flow Modules
+@cindex control flow modules, editor-level
+@cindex modules, editor-level control flow
@example
event-Xt.c
-@node Modules for the Basic Displayable Lisp Objects, Modules for other Display-Related Lisp Objects, Editor-Level Control Flow Modules, A Summary of the Various XEmacs Modules
+@node Modules for the Basic Displayable Lisp Objects
@section Modules for the Basic Displayable Lisp Objects
+@cindex modules for the basic displayable Lisp objects
+@cindex displayable Lisp objects, modules for the basic
+@cindex Lisp objects, modules for the basic displayable
+@cindex objects, modules for the basic displayable Lisp
@example
console-msw.c
-@node Modules for other Display-Related Lisp Objects, Modules for the Redisplay Mechanism, Modules for the Basic Displayable Lisp Objects, A Summary of the Various XEmacs Modules
+@node Modules for other Display-Related Lisp Objects
@section Modules for other Display-Related Lisp Objects
+@cindex modules for other display-related Lisp objects
+@cindex display-related Lisp objects, modules for other
+@cindex Lisp objects, modules for other display-related
@example
faces.c
different colors, for easy reading. The C support is provided so that
this is fast.
+As of 21.4.10, bugs introduced at the very end of the 21.2 series in the
+``syntax properties'' code were fixed, and highlighting is acceptably
+quick again. However, presumably more improvements are possible, and
+the places to look are probably here, in the defun-traversing code, and
+in @file{syntax.c}, in the comment-traversing code.
@example
-@node Modules for the Redisplay Mechanism, Modules for Interfacing with the File System, Modules for other Display-Related Lisp Objects, A Summary of the Various XEmacs Modules
+@node Modules for the Redisplay Mechanism
@section Modules for the Redisplay Mechanism
+@cindex modules for the redisplay mechanism
+@cindex redisplay mechanism, modules for the
@example
redisplay-output.c
-@node Modules for Interfacing with the File System, Modules for Other Aspects of the Lisp Interpreter and Object System, Modules for the Redisplay Mechanism, A Summary of the Various XEmacs Modules
+@node Modules for Interfacing with the File System
@section Modules for Interfacing with the File System
+@cindex modules for interfacing with the file system
+@cindex interfacing with the file system, modules for
+@cindex file system, modules for interfacing with the
@example
lstream.c
-@node Modules for Other Aspects of the Lisp Interpreter and Object System, Modules for Interfacing with the Operating System, Modules for Interfacing with the File System, A Summary of the Various XEmacs Modules
+@node Modules for Other Aspects of the Lisp Interpreter and Object System
@section Modules for Other Aspects of the Lisp Interpreter and Object System
+@cindex modules for other aspects of the Lisp interpreter and object system
+@cindex Lisp interpreter and object system, modules for other aspects of the
+@cindex interpreter and object system, modules for other aspects of the Lisp
+@cindex object system, modules for other aspects of the Lisp interpreter and
@example
elhash.c
@code{forward-sexp}, and by @file{font-lock.c} to locate quoted strings,
comments, etc.
+@c #### Break this out into a separate node somewhere!
+Syntax codes are implemented as bitfields in an int. Bits 0-6 contain
+the syntax code itself, bit 7 is a special prefix flag used for Lisp,
+and bits 16-23 contain comment syntax flags. From the Lisp programmer's
+point of view, there are 11 flags: 2 styles X 2 characters X @{start,
+end@} flags for two-character comment delimiters, 2 style flags for
+one-character comment delimiters, and the prefix flag.
+
+Internally, however, the characters used in multi-character delimiters
+will have non-comment-character syntax classes (@emph{e.g.}, the
+@samp{/} in C's @samp{/*} comment-start delimiter has ``punctuation''
+(here meaning ``operator-like'') class in C modes). Thus in a mixed
+comment style, such as C++'s @samp{//} to end of line, is represented by
+giving @samp{/} the ``punctuation'' class and the ``style b first
+character of start sequence'' and ``style b second character of start
+sequence'' flags. The fact that class is @emph{not} punctuation allows
+the syntax scanner to recognize that this is a multi-character
+delimiter. The @samp{newline} character is given (single-character)
+``comment-end'' @emph{class} and the ``style b first character of end
+sequence'' @emph{flag}. The ``comment-end'' class allows the scanner to
+determine that no second character is needed to terminate the comment.
+
+There used to be a syntax class @samp{Sextword}. A character of
+@samp{Sextword} class is a word-constituent but a word boundary may
+exist between two such characters. Ken'ichi HANDA <handa@@etl.go.jp>
+explains the purpose of the Sextword syntax category:
+
+@quotation
+Japanese words are not separated by spaces, which makes finding word
+boundaries very difficult. Theoretically it's impossible without
+using natural language processing techniques. But, by defining
+pseudo-words as below (much simplified for letting you understand it
+easily) for Japanese, we can have a convenient forward-word function
+for Japanese.
+
+@display
+A Japanese word is a sequence of characters that consists of
+zero or more Kanji characters followed by zero or more
+Hiragana characters.
+@end display
+
+Then, the problem is that now we can't say that a sequence of
+word-constituents makes up a word. For instance, both Hiragana "A"
+and Kanji "KAN" are word-constituents but the sequence of these two
+letters can't be a single word.
+
+So, we introduced Sextword for Japanese letters.
+@end quotation
+
+There seems to have been some controversy about this category, as it has
+been removed, readded, and removed again. Currently neither GNU Emacs
+(21.3.99) nor XEmacs (21.5.17) seems to use it.
@example
-@node Modules for Interfacing with the Operating System, Modules for Interfacing with X Windows, Modules for Other Aspects of the Lisp Interpreter and Object System, A Summary of the Various XEmacs Modules
+@node Modules for Interfacing with the Operating System
@section Modules for Interfacing with the Operating System
+@cindex modules for interfacing with the operating system
+@cindex interfacing with the operating system, modules for
+@cindex operating system, modules for interfacing with the
@example
callproc.c
-@node Modules for Interfacing with X Windows, Modules for Internationalization, Modules for Interfacing with the Operating System, A Summary of the Various XEmacs Modules
+@node Modules for Interfacing with X Windows
@section Modules for Interfacing with X Windows
+@cindex modules for interfacing with X Windows
+@cindex interfacing with X Windows, modules for
+@cindex X Windows, modules for interfacing with
@example
Emacs.ad.h
-@node Modules for Internationalization, , Modules for Interfacing with X Windows, A Summary of the Various XEmacs Modules
+@node Modules for Internationalization
@section Modules for Internationalization
+@cindex modules for internationalization
+@cindex internationalization, modules for
@example
mule-canna.c
@file{mule-mcpath.c} provides some functions to allow for pathnames
containing extended characters. This code is fragmentary, obsolete, and
-completely non-working. Instead, @var{pathname-coding-system} is used
+completely non-working. Instead, @code{pathname-coding-system} is used
to specify conversions of names of files and directories. The standard
C I/O functions like @samp{open()} are wrapped so that conversion occurs
automatically.
-@file{mule.c} provides a few miscellaneous things that should probably
-be elsewhere.
+@file{mule.c} contains a few miscellaneous things. It currently seems
+to be unused and probably should be removed.
+@node Modules for Regression Testing
+@section Modules for Regression Testing
+@cindex modules for regression testing
+@cindex regression testing, modules for
+
+@example
+test-harness.el
+base64-tests.el
+byte-compiler-tests.el
+case-tests.el
+ccl-tests.el
+c-tests.el
+database-tests.el
+extent-tests.el
+hash-table-tests.el
+lisp-tests.el
+md5-tests.el
+mule-tests.el
+regexp-tests.el
+symbol-tests.el
+syntax-tests.el
+tag-tests.el
+@end example
+
+@file{test-harness.el} defines the macros @code{Assert},
+@code{Check-Error}, @code{Check-Error-Message}, and
+@code{Check-Message}. The other files are test files, testing various
+XEmacs modules.
+
+
+
@node Allocation of Objects in XEmacs Lisp, Dumping, A Summary of the Various XEmacs Modules, Top
@chapter Allocation of Objects in XEmacs Lisp
+@cindex allocation of objects in XEmacs Lisp
+@cindex objects in XEmacs Lisp, allocation of
+@cindex Lisp objects, allocation of in XEmacs
@menu
* Introduction to Allocation::
* Compiled Function::
@end menu
-@node Introduction to Allocation, Garbage Collection, Allocation of Objects in XEmacs Lisp, Allocation of Objects in XEmacs Lisp
+@node Introduction to Allocation
@section Introduction to Allocation
+@cindex allocation, introduction to
Emacs Lisp, like all Lisps, has garbage collection. This means that
the programmer never has to explicitly free (destroy) an object; it
in directly-tagged fashion.
-@node Garbage Collection, GCPROing, Introduction to Allocation, Allocation of Objects in XEmacs Lisp
+@node Garbage Collection
@section Garbage Collection
@cindex garbage collection
garbage collection (according to @code{gc-cons-threshold}).
-@node GCPROing, Garbage Collection - Step by Step, Garbage Collection, Allocation of Objects in XEmacs Lisp
+@node GCPROing
@section @code{GCPRO}ing
+@cindex @code{GCPRO}ing
+@cindex garbage collection protection
+@cindex protection, garbage collection
@code{GCPRO}ing is one of the ugliest and trickiest parts of Emacs
internals. The basic idea is that whenever garbage collection
different section of code.
@end enumerate
+If you don't understand whether to @code{GCPRO} in a particular
+instance, ask on the mailing lists. A general hint is that @code{prog1}
+is the canonical example.
+
@cindex garbage collection, conservative
@cindex conservative garbage collection
Given the extremely error-prone nature of the @code{GCPRO} scheme, and
it obviates the need for @code{GCPRO}ing, and allows garbage collection
to happen at any point at all, such as during object allocation.
-@node Garbage Collection - Step by Step, Integers and Characters, GCPROing, Allocation of Objects in XEmacs Lisp
+@node Garbage Collection - Step by Step
@section Garbage Collection - Step by Step
-@cindex garbage collection step by step
+@cindex garbage collection - step by step
@menu
* Invocation::
* sweep_bit_vectors_1::
@end menu
-@node Invocation, garbage_collect_1, Garbage Collection - Step by Step, Garbage Collection - Step by Step
+@node Invocation
@subsection Invocation
@cindex garbage collection, invocation
for example the ones raised by every @code{QUIT}-macro triggered after
pressing Ctrl-g.
-@node garbage_collect_1, mark_object, Invocation, Garbage Collection - Step by Step
+@node garbage_collect_1
@subsection @code{garbage_collect_1}
@cindex @code{garbage_collect_1}
and exit.
@end enumerate
-@node mark_object, gc_sweep, garbage_collect_1, Garbage Collection - Step by Step
+@node mark_object
@subsection @code{mark_object}
@cindex @code{mark_object}
In case another object was returned, as mentioned before, we reiterate
the whole @code{mark_object} process beginning with this next object.
-@node gc_sweep, sweep_lcrecords_1, mark_object, Garbage Collection - Step by Step
+@node gc_sweep
@subsection @code{gc_sweep}
@cindex @code{gc_sweep}
@code{xfree}) and the free list state is set to the state it had before
handling this block.
-@node sweep_lcrecords_1, compact_string_chars, gc_sweep, Garbage Collection - Step by Step
+@node sweep_lcrecords_1
@subsection @code{sweep_lcrecords_1}
@cindex @code{sweep_lcrecords_1}
@code{xfree}. During this loop, the lcrecord statistics are kept up to
date by calling @code{tick_lcrecord_stats} with the right arguments,
-@node compact_string_chars, sweep_strings, sweep_lcrecords_1, Garbage Collection - Step by Step
+@node compact_string_chars
@subsection @code{compact_string_chars}
@cindex @code{compact_string_chars}
i.e. @code{to_block}, and all remaining blocks (we know that they just
carry garbage) are explicitly @code{xfree}d.
-@node sweep_strings, sweep_bit_vectors_1, compact_string_chars, Garbage Collection - Step by Step
+@node sweep_strings
@subsection @code{sweep_strings}
@cindex @code{sweep_strings}
therefore it was @code{malloc}ed separately, we know also @code{xfree}
it explicitly.
-@node sweep_bit_vectors_1, , sweep_strings, Garbage Collection - Step by Step
+@node sweep_bit_vectors_1
@subsection @code{sweep_bit_vectors_1}
@cindex @code{sweep_bit_vectors_1}
In addition, the bookkeeping information used for garbage
collector's output purposes is updated.
-@node Integers and Characters, Allocation from Frob Blocks, Garbage Collection - Step by Step, Allocation of Objects in XEmacs Lisp
+@node Integers and Characters
@section Integers and Characters
+@cindex integers and characters
+@cindex characters, integers and
Integer and character Lisp objects are created from integers using the
macros @code{XSETINT()} and @code{XSETCHAR()} or the equivalent
are too big; i.e. you won't get the value you expected but the tag bits
will at least be correct.
-@node Allocation from Frob Blocks, lrecords, Integers and Characters, Allocation of Objects in XEmacs Lisp
+@node Allocation from Frob Blocks
@section Allocation from Frob Blocks
+@cindex allocation from frob blocks
+@cindex frob blocks, allocation from
The uninitialized memory required by a @code{Lisp_Object} of a particular type
is allocated using
none. (There are actually two versions of these macros, one of which is
more defensive but less efficient and is used for error-checking.)
-@node lrecords, Low-level allocation, Allocation from Frob Blocks, Allocation of Objects in XEmacs Lisp
+@node lrecords
@section lrecords
+@cindex lrecords
[see @file{lrecord.h}]
For an example, see the methods for window configurations and opaques.
@end enumerate
-@node Low-level allocation, Cons, lrecords, Allocation of Objects in XEmacs Lisp
+@node Low-level allocation
@section Low-level allocation
+@cindex low-level allocation
+@cindex allocation, low-level
Memory that you want to allocate directly should be allocated using
@code{xmalloc()} rather than @code{malloc()}. This implements
statistics on how much memory is allocated, so that garbage-collection
can be invoked when the threshold is reached.
-@node Cons, Vector, Low-level allocation, Allocation of Objects in XEmacs Lisp
+@node Cons
@section Cons
+@cindex cons
Conses are allocated in standard frob blocks. The only thing to
note is that conses can be explicitly freed using @code{free_cons()}
If you mess this up, you will get BADLY BURNED, and it has happened
before.
-@node Vector, Bit Vector, Cons, Allocation of Objects in XEmacs Lisp
+@node Vector
@section Vector
+@cindex vector
As mentioned above, each vector is @code{malloc()}ed individually, and
all are threaded through the variable @code{all_vectors}. Vectors are
is actually @code{malloc()}ed with the right size, however, and access
to any element through the @code{contents} array works fine.
-@node Bit Vector, Symbol, Vector, Allocation of Objects in XEmacs Lisp
+@node Bit Vector
@section Bit Vector
+@cindex bit vector
+@cindex vector, bit
Bit vectors work exactly like vectors, except for more complicated
code to access an individual bit, and except for the fact that bit
tag field in bit vector Lisp words is ``lrecord'' rather than
``vector''.)
-@node Symbol, Marker, Bit Vector, Allocation of Objects in XEmacs Lisp
+@node Symbol
@section Symbol
+@cindex symbol
Symbols are also allocated in frob blocks. Symbols in the awful
horrible obarray structure are chained through their @code{next} field.
Remember that @code{intern} looks up a symbol in an obarray, creating
one if necessary.
-@node Marker, String, Symbol, Allocation of Objects in XEmacs Lisp
+@node Marker
@section Marker
+@cindex marker
Markers are allocated in frob blocks, as usual. They are kept
in a buffer unordered, but in a doubly-linked list so that they
markers from a buffer.) Markers are removed from a buffer in
the finalize stage, in @code{ADDITIONAL_FREE_marker()}.
-@node String, Compiled Function, Marker, Allocation of Objects in XEmacs Lisp
+@node String
@section String
+@cindex string
As mentioned above, strings are a special case. A string is logically
two parts, a fixed-size object (containing the length, property list,
The string compactor recognizes this special 0xFFFFFFFF marker and
handles it correctly.
-@node Compiled Function, , String, Allocation of Objects in XEmacs Lisp
+@node Compiled Function
@section Compiled Function
+@cindex compiled function
+@cindex function, compiled
Not yet documented.
@node Dumping, Events and the Event Loop, Allocation of Objects in XEmacs Lisp, Top
@chapter Dumping
+@cindex dumping
@section What is dumping and its justification
+@cindex dumping and its justification, what is
The C code of XEmacs is just a Lisp engine with a lot of built-in
primitives useful for writing an editor. The editor itself is written
* Remaining issues::
@end menu
-@node Overview, Data descriptions, Dumping, Dumping
+@node Overview
@section Overview
+@cindex dumping overview
The portable dumping system has to:
data. Also, rebuild all the quickly rebuildable data.
@end enumerate
-@node Data descriptions, Dumping phase, Overview, Dumping
+@node Data descriptions
@section Data descriptions
+@cindex dumping data descriptions
The more complex task of the dumper is to be able to write lisp objects
(lrecords) and C structs to disk and reload them at a different address,
structures we use a struct struct_description, which includes a size
field and a pointer to an associated array of lrecord_description.
-@node Dumping phase, Reloading phase, Data descriptions, Dumping
+@node Dumping phase
@section Dumping phase
+@cindex dumping phase
Dumping is done by calling the function pdump() (in dumper.c) which is
invoked from Fdump_emacs (in emacs.c). This function performs a number
* Pointers dumping::
@end menu
-@node Object inventory, Address allocation, Dumping phase, Dumping phase
+@node Object inventory
@subsection Object inventory
+@cindex dumping object inventory
The first task is to build the list of the objects to dump. This
includes:
non-weak equivalent (without changing their type, of course). This has
not yet been a problem.
-@node Address allocation, The header, Object inventory, Dumping phase
+@node Address allocation
@subsection Address allocation
+@cindex dumping address allocation
The next step is to allocate the offsets of each of the objects in the
starting at offset 256 (this leaves room for the header and keeps the
alignments happy).
-@node The header, Data dumping, Address allocation, Dumping phase
+@node The header
@subsection The header
+@cindex dumping, the header
The next step creates the file and writes a header with a signature and
some random information in it. The @code{reloc_address} field, which
post-reload relocation, is set to 0. It then seeks to offset 256 (base
offset for the objects).
-@node Data dumping, Pointers dumping, The header, Dumping phase
+@node Data dumping
@subsection Data dumping
+@cindex data dumping
+@cindex dumping, data
The data is dumped in the same order as the addresses were allocated by
@code{pdump_dump_data()}, called from @code{pdump_scan_by_alignment()}.
are ensured that the object is always written at the offset in the file
allocated in step Address Allocation.
-@node Pointers dumping, , Data dumping, Dumping phase
+@node Pointers dumping
@subsection Pointers dumping
+@cindex pointers dumping
+@cindex dumping, pointers
A bunch of tables needed to reassign properly the global pointers are
then written. They are:
This is the end of the dumping part.
-@node Reloading phase, Remaining issues, Dumping phase, Dumping
+@node Reloading phase
@section Reloading phase
+@cindex reloading phase
+@cindex dumping, reloading phase
@subsection File loading
+@cindex dumping, file loading
The file is mmap'ed in memory (which ensures a PAGESIZE alignment, at
least 4096), or if mmap is unavailable or fails, a 256-bytes aligned
@subsection Putting back the pdump_opaques
+@cindex dumping, putting back the pdump_opaques
The memory contents are restored in the obvious and trivial way.
@subsection Putting back the pdump_root_struct_ptrs
+@cindex dumping, putting back the pdump_root_struct_ptrs
The variables pointed to by pdump_root_struct_ptrs in the dump phase are
reset to the right relocated object addresses.
@subsection Object relocation
+@cindex dumping, object relocation
All the objects are relocated using their description and their offset
by @code{pdump_reloc_one}. This step is unnecessary if the
@subsection Putting back the pdump_root_objects and pdump_weak_object_chains
+@cindex dumping, putting back the pdump_root_objects and pdump_weak_object_chains
Same as Putting back the pdump_root_struct_ptrs.
@subsection Reorganize the hash tables
+@cindex dumping, reorganize the hash tables
Since some of the hash values in the lisp hash tables are
address-dependent, their layout is now wrong. So we go through each of
them and have them resorted by calling @code{pdump_reorganize_hash_table}.
-@node Remaining issues, , Reloading phase, Dumping
+@node Remaining issues
@section Remaining issues
+@cindex dumping, remaining issues
The build process will have to start a post-dump xemacs, ask it the
loading address (which will, hopefully, be always the same between
@node Events and the Event Loop, Evaluation; Stack Frames; Bindings, Dumping, Top
@chapter Events and the Event Loop
+@cindex events and the event loop
+@cindex event loop, events and the
@menu
* Introduction to Events::
* Dispatching Events; The Command Builder::
@end menu
-@node Introduction to Events, Main Loop, Events and the Event Loop, Events and the Event Loop
+@node Introduction to Events
@section Introduction to Events
+@cindex events, introduction to
An event is an object that encapsulates information about an
interesting occurrence in the operating system. Events are
Emacs events are documented in @file{events.h}; I'll discuss them
later.
-@node Main Loop, Specifics of the Event Gathering Mechanism, Introduction to Events, Events and the Event Loop
+@node Main Loop
@section Main Loop
+@cindex main loop
+@cindex events, main loop
The @dfn{command loop} is the top-level loop that the editor is always
running. It loops endlessly, calling @code{next-event} to retrieve an
invoking @code{top_level_1()}, just like when it invokes
@code{command_loop_2()}.
-@node Specifics of the Event Gathering Mechanism, Specifics About the Emacs Event, Main Loop, Events and the Event Loop
+@node Specifics of the Event Gathering Mechanism
@section Specifics of the Event Gathering Mechanism
+@cindex event gathering mechanism, specifics of the
Here is an approximate diagram of the collection processes
at work in XEmacs, under TTY's (TTY's are simpler than X
using `dispatch-event'
@end example
-@node Specifics About the Emacs Event, The Event Stream Callback Routines, Specifics of the Event Gathering Mechanism, Events and the Event Loop
+@node Specifics About the Emacs Event
@section Specifics About the Emacs Event
+@cindex event, specifics about the Lisp object
-@node The Event Stream Callback Routines, Other Event Loop Functions, Specifics About the Emacs Event, Events and the Event Loop
+@node The Event Stream Callback Routines
@section The Event Stream Callback Routines
+@cindex event stream callback routines, the
+@cindex callback routines, the event stream
-@node Other Event Loop Functions, Converting Events, The Event Stream Callback Routines, Events and the Event Loop
+@node Other Event Loop Functions
@section Other Event Loop Functions
+@cindex event loop functions, other
@code{detect_input_pending()} and @code{input-pending-p} look for
input by calling @code{event_stream->event_pending_p} and looking in
the right kind of input method support, it is possible for (read-char)
to return a Kanji character.
-@node Converting Events, Dispatching Events; The Command Builder, Other Event Loop Functions, Events and the Event Loop
+@node Converting Events
@section Converting Events
+@cindex converting events
+@cindex events, converting
@code{character_to_event()}, @code{event_to_character()},
@code{event-to-character}, and @code{character-to-event} convert between
between character representation and the split-up event representation
(keysym plus mod keys).
-@node Dispatching Events; The Command Builder, , Converting Events, Events and the Event Loop
+@node Dispatching Events; The Command Builder
@section Dispatching Events; The Command Builder
+@cindex dispatching events; the command builder
+@cindex events; the command builder, dispatching
+@cindex command builder, dispatching events; the
Not yet documented.
@node Evaluation; Stack Frames; Bindings, Symbols and Variables, Events and the Event Loop, Top
@chapter Evaluation; Stack Frames; Bindings
+@cindex evaluation; stack frames; bindings
+@cindex stack frames; bindings, evaluation;
+@cindex bindings, evaluation; stack frames;
@menu
* Evaluation::
* Catch and Throw::
@end menu
-@node Evaluation, Dynamic Binding; The specbinding Stack; Unwind-Protects, Evaluation; Stack Frames; Bindings, Evaluation; Stack Frames; Bindings
+@node Evaluation
@section Evaluation
+@cindex evaluation
@code{Feval()} evaluates the form (a Lisp object) that is passed to
it. Note that evaluation is only non-trivial for two types of objects:
an array). @code{apply1()} uses @code{Fapply()} while the others use
@code{Ffuncall()} to do the real work.
-@node Dynamic Binding; The specbinding Stack; Unwind-Protects, Simple Special Forms, Evaluation, Evaluation; Stack Frames; Bindings
+@node Dynamic Binding; The specbinding Stack; Unwind-Protects
@section Dynamic Binding; The specbinding Stack; Unwind-Protects
+@cindex dynamic binding; the specbinding stack; unwind-protects
+@cindex binding; the specbinding stack; unwind-protects, dynamic
+@cindex specbinding stack; unwind-protects, dynamic binding; the
+@cindex unwind-protects, dynamic binding; the specbinding stack;
@example
struct specbinding
the symbol's value).
@end enumerate
-@node Simple Special Forms, Catch and Throw, Dynamic Binding; The specbinding Stack; Unwind-Protects, Evaluation; Stack Frames; Bindings
+@node Simple Special Forms
@section Simple Special Forms
+@cindex special forms, simple
@code{or}, @code{and}, @code{if}, @code{cond}, @code{progn},
@code{prog1}, @code{prog2}, @code{setq}, @code{quote}, @code{function},
compiler knows how to convert calls to these functions directly into
byte code.
-@node Catch and Throw, , Simple Special Forms, Evaluation; Stack Frames; Bindings
+@node Catch and Throw
@section Catch and Throw
+@cindex catch and throw
+@cindex throw, catch and
@example
struct catchtag
@node Symbols and Variables, Buffers and Textual Representation, Evaluation; Stack Frames; Bindings, Top
@chapter Symbols and Variables
+@cindex symbols and variables
+@cindex variables, symbols and
@menu
* Introduction to Symbols::
* Symbol Values::
@end menu
-@node Introduction to Symbols, Obarrays, Symbols and Variables, Symbols and Variables
+@node Introduction to Symbols
@section Introduction to Symbols
+@cindex symbols, introduction to
A symbol is basically just an object with four fields: a name (a
string), a value (some Lisp object), a function (some Lisp object), and
additional values with particular names, and once again the namespace is
independent of the function and variable namespaces.
-@node Obarrays, Symbol Values, Introduction to Symbols, Symbols and Variables
+@node Obarrays
@section Obarrays
+@cindex obarrays
The identity of symbols with their names is accomplished through a
structure called an obarray, which is just a poorly-implemented hash
into any obarray.) Finally, @code{mapatoms} maps over all of the symbols
in an obarray.
-@node Symbol Values, , Obarrays, Symbols and Variables
+@node Symbol Values
@section Symbol Values
+@cindex symbol values
+@cindex values, symbol
The value field of a symbol normally contains a Lisp object. However,
a symbol can be @dfn{unbound}, meaning that it logically has no value.
@node Buffers and Textual Representation, MULE Character Sets and Encodings, Symbols and Variables, Top
@chapter Buffers and Textual Representation
+@cindex buffers and textual representation
+@cindex textual representation, buffers and
@menu
* Introduction to Buffers:: A buffer holds a block of text such as a file.
* The Buffer Object:: The Lisp object corresponding to a buffer.
@end menu
-@node Introduction to Buffers, The Text in a Buffer, Buffers and Textual Representation, Buffers and Textual Representation
+@node Introduction to Buffers
@section Introduction to Buffers
+@cindex buffers, introduction to
A buffer is logically just a Lisp object that holds some text.
In this, it is like a string, but a buffer is optimized for
window. (This latter distinction is explained in detail in the section
on windows.)
-@node The Text in a Buffer, Buffer Lists, Introduction to Buffers, Buffers and Textual Representation
+@node The Text in a Buffer
@section The Text in a Buffer
+@cindex text in a buffer, the
+@cindex buffer, the text in a
The text in a buffer consists of a sequence of zero or more
characters. A @dfn{character} is an integer that logically represents
number of possible alternative representations (e.g. EUC-encoded text,
etc.).
-@node Buffer Lists, Markers and Extents, The Text in a Buffer, Buffers and Textual Representation
+@node Buffer Lists
@section Buffer Lists
+@cindex buffer lists
Recall earlier that buffers are @dfn{permanent} objects, i.e. that
they remain around until explicitly deleted. This entails that there is
a unique name from this by appending a number, and then creates the
buffer. This is basically like the symbol operation @code{gensym}.
-@node Markers and Extents, Bufbytes and Emchars, Buffer Lists, Buffers and Textual Representation
+@node Markers and Extents
@section Markers and Extents
+@cindex markers and extents
+@cindex extents, markers and
Among the things associated with a buffer are things that are
logically attached to certain buffer positions. This can be used to
(which could happen as a result of text being deleted) or the buffer is
deleted, and primitives do exist to enumerate the extents in a buffer.
-@node Bufbytes and Emchars, The Buffer Object, Markers and Extents, Buffers and Textual Representation
+@node Bufbytes and Emchars
@section Bufbytes and Emchars
+@cindex Bufbytes and Emchars
+@cindex Emchars, Bufbytes and
Not yet documented.
-@node The Buffer Object, , Bufbytes and Emchars, Buffers and Textual Representation
+@node The Buffer Object
@section The Buffer Object
+@cindex buffer object, the
+@cindex object, the buffer
Buffers contain fields not directly accessible by the Lisp programmer.
We describe them here, naming them by the names used in the C code.
@node MULE Character Sets and Encodings, The Lisp Reader and Compiler, Buffers and Textual Representation, Top
@chapter MULE Character Sets and Encodings
+@cindex Mule character sets and encodings
+@cindex character sets and encodings, Mule
+@cindex encodings, Mule character sets and
Recall that there are two primary ways that text is represented in
XEmacs. The @dfn{buffer} representation sees the text as a series of
* CCL::
@end menu
-@node Character Sets, Encodings, MULE Character Sets and Encodings, MULE Character Sets and Encodings
+@node Character Sets
@section Character Sets
+@cindex character sets
A character set (or @dfn{charset}) is an ordered set of characters. A
particular character in a charset is indexed using one or more
This is a bit ad-hoc but gets the job done.
-@node Encodings, Internal Mule Encodings, Character Sets, MULE Character Sets and Encodings
+@node Encodings
@section Encodings
+@cindex encodings, Mule
+@cindex Mule encodings
An @dfn{encoding} is a way of numerically representing characters from
one or more character sets. If an encoding only encompasses one
* JIS7::
@end menu
-@node Japanese EUC (Extended Unix Code), JIS7, Encodings, Encodings
+@node Japanese EUC (Extended Unix Code)
@subsection Japanese EUC (Extended Unix Code)
+@cindex Japanese EUC (Extended Unix Code)
+@cindex EUC (Extended Unix Code), Japanese
+@cindex Extended Unix Code, Japanese EUC
This encompasses the character sets Printing-ASCII, Japanese-JISX0201,
and Japanese-JISX0208-Kana (half-width katakana, the right half of
@end example
-@node JIS7, , Japanese EUC (Extended Unix Code), Encodings
+@node JIS7
@subsection JIS7
+@cindex JIS7
This encompasses the character sets Printing-ASCII,
Japanese-JISX0201-Roman (the left half of JISX0201; this character set
Initially, Printing-ASCII is invoked.
-@node Internal Mule Encodings, CCL, Encodings, MULE Character Sets and Encodings
+@node Internal Mule Encodings
@section Internal Mule Encodings
+@cindex internal Mule encodings
+@cindex Mule encodings, internal
+@cindex encodings, internal Mule
In XEmacs/Mule, each character set is assigned a unique number, called a
@dfn{leading byte}. This is used in the encodings of a character.
* Internal Character Encoding::
@end menu
-@node Internal String Encoding, Internal Character Encoding, Internal Mule Encodings, Internal Mule Encodings
+@node Internal String Encoding
@subsection Internal String Encoding
+@cindex internal string encoding
+@cindex string encoding, internal
+@cindex encoding, internal string
ASCII characters are encoded using their position code directly. Other
characters are encoded using their leading byte followed by their
Shift-JIS and Big5 (not yet described) satisfy only (2). (All
non-modal encodings must satisfy (2), in order to be unambiguous.)
-@node Internal Character Encoding, , Internal String Encoding, Internal Mule Encodings
+@node Internal Character Encoding
@subsection Internal Character Encoding
+@cindex internal character encoding
+@cindex character encoding, internal
+@cindex encoding, internal character
One 19-bit word represents a single character. The word is
separated into three fields:
Note that character codes 0 - 255 are the same as the ``binary encoding''
described above.
-@node CCL, , Internal Mule Encodings, MULE Character Sets and Encodings
+@node CCL
@section CCL
+@cindex CCL
@example
CCL PROGRAM SYNTAX:
@node The Lisp Reader and Compiler, Lstreams, MULE Character Sets and Encodings, Top
@chapter The Lisp Reader and Compiler
+@cindex Lisp reader and compiler, the
+@cindex reader and compiler, the Lisp
+@cindex compiler, the Lisp reader and
Not yet documented.
@node Lstreams, Consoles; Devices; Frames; Windows, The Lisp Reader and Compiler, Top
@chapter Lstreams
+@cindex lstreams
An @dfn{lstream} is an internal Lisp object that provides a generic
buffering stream implementation. Conceptually, you send data to the
* Lstream Methods:: Creating new lstream types.
@end menu
-@node Creating an Lstream, Lstream Types, Lstreams, Lstreams
+@node Creating an Lstream
@section Creating an Lstream
+@cindex lstream, creating an
Lstreams come in different types, depending on what is being interfaced
to. Although the primitive for creating new lstreams is
Open for writing, but never writes partial MULE characters.
@end table
-@node Lstream Types, Lstream Functions, Creating an Lstream, Lstreams
+@node Lstream Types
@section Lstream Types
+@cindex lstream types
+@cindex types, lstream
@table @asis
@item stdio
@item encoding
@end table
-@node Lstream Functions, Lstream Methods, Lstream Types, Lstreams
+@node Lstream Functions
@section Lstream Functions
+@cindex lstream functions
@deftypefun {Lstream *} Lstream_new (Lstream_implementation *@var{imp}, const char *@var{mode})
Allocate and return a new Lstream. This function is not really meant to
Rewind the stream to the beginning.
@end deftypefun
-@node Lstream Methods, , Lstream Functions, Lstreams
+@node Lstream Methods
@section Lstream Methods
+@cindex lstream methods
@deftypefn {Lstream Method} ssize_t reader (Lstream *@var{stream}, unsigned char *@var{data}, size_t @var{size})
Read some data from the stream's end and store it into @var{data}, which
@node Consoles; Devices; Frames; Windows, The Redisplay Mechanism, Lstreams, Top
@chapter Consoles; Devices; Frames; Windows
+@cindex consoles; devices; frames; windows
+@cindex devices; frames; windows, consoles;
+@cindex frames; windows, consoles; devices;
+@cindex windows, consoles; devices; frames;
@menu
* Introduction to Consoles; Devices; Frames; Windows::
* The Window Object::
@end menu
-@node Introduction to Consoles; Devices; Frames; Windows, Point, Consoles; Devices; Frames; Windows, Consoles; Devices; Frames; Windows
+@node Introduction to Consoles; Devices; Frames; Windows
@section Introduction to Consoles; Devices; Frames; Windows
+@cindex consoles; devices; frames; windows, introduction to
+@cindex devices; frames; windows, introduction to consoles;
+@cindex frames; windows, introduction to consoles; devices;
+@cindex windows, introduction to consoles; devices; frames;
A window-system window that you see on the screen is called a
@dfn{frame} in Emacs terminology. Each frame is subdivided into one or
within it to become the selected window. Similar relationships apply
for consoles to devices and devices to frames.
-@node Point, Window Hierarchy, Introduction to Consoles; Devices; Frames; Windows, Consoles; Devices; Frames; Windows
+@node Point
@section Point
+@cindex point
Recall that every buffer has a current insertion position, called
@dfn{point}. Now, two or more windows may be displaying the same buffer,
buffer's point instead. This is related to why @code{save-window-excursion}
does not save the selected window's value of @code{point}.
-@node Window Hierarchy, The Window Object, Point, Consoles; Devices; Frames; Windows
+@node Window Hierarchy
@section Window Hierarchy
@cindex window hierarchy
@cindex hierarchy of windows
artifact that should be fixed.)
@end enumerate
-@node The Window Object, , Window Hierarchy, Consoles; Devices; Frames; Windows
+@node The Window Object
@section The Window Object
+@cindex window object, the
+@cindex object, the window
Windows have the following accessible fields:
@node The Redisplay Mechanism, Extents, Consoles; Devices; Frames; Windows, Top
@chapter The Redisplay Mechanism
+@cindex redisplay mechanism, the
The redisplay mechanism is one of the most complicated sections of
XEmacs, especially from a conceptual standpoint. This is doubly so
* Redisplay Piece by Piece::
@end menu
-@node Critical Redisplay Sections, Line Start Cache, The Redisplay Mechanism, The Redisplay Mechanism
+@node Critical Redisplay Sections
@section Critical Redisplay Sections
+@cindex redisplay sections, critical
@cindex critical redisplay sections
Within this section, we are defenseless and assume that the
#### If a frame-size change does occur we should probably
actually be preempting redisplay.
-@node Line Start Cache, Redisplay Piece by Piece, Critical Redisplay Sections, The Redisplay Mechanism
+@node Line Start Cache
@section Line Start Cache
@cindex line start cache
In case you're wondering, the Second Golden Rule of Redisplay is not
applicable.
-@node Redisplay Piece by Piece, , Line Start Cache, The Redisplay Mechanism
+@node Redisplay Piece by Piece
@section Redisplay Piece by Piece
-@cindex Redisplay Piece by Piece
+@cindex redisplay piece by piece
As you can begin to see redisplay is complex and also not well
documented. Chuck no longer works on XEmacs so this section is my take
@node Extents, Faces, The Redisplay Mechanism, Top
@chapter Extents
+@cindex extents
@menu
* Introduction to Extents:: Extents are ranges over text, with properties.
* Extent Fragments:: Cached information useful for redisplay.
@end menu
-@node Introduction to Extents, Extent Ordering, Extents, Extents
+@node Introduction to Extents
@section Introduction to Extents
+@cindex extents, introduction to
Extents are regions over a buffer, with a start and an end position
denoting the region of the buffer included in the extent. In
however, and just ended up complexifying and buggifying all the
rest of the code.)
-@node Extent Ordering, Format of the Extent Info, Introduction to Extents, Extents
+@node Extent Ordering
@section Extent Ordering
+@cindex extent ordering
Extents are compared using memory indices. There are two orderings
for extents and both orders are kept current at all times. The normal
all occurrences of ``display order'' and ``e-order'', ``less than'' and
``greater than'', and ``extent start'' and ``extent end''.
-@node Format of the Extent Info, Zero-Length Extents, Extent Ordering, Extents
+@node Format of the Extent Info
@section Format of the Extent Info
+@cindex extent info, format of the
An extent-info structure consists of a list of the buffer or string's
extents and a @dfn{stack of extents} that lists all of the extents over
array, except for the fact that positions are integers (this should be
generalized to handle integers and linked list equally well).
-@node Zero-Length Extents, Mathematics of Extent Ordering, Format of the Extent Info, Extents
+@node Zero-Length Extents
@section Zero-Length Extents
+@cindex zero-length extents
+@cindex extents, zero-length
Extents can be zero-length, and will end up that way if their endpoints
are explicitly set that way or if their detachable property is @code{nil}
exactly like markers and that open-closed, non-detachable zero-length
extents behave like the ``point-type'' marker in Mule.
-@node Mathematics of Extent Ordering, Extent Fragments, Zero-Length Extents, Extents
+@node Mathematics of Extent Ordering
@section Mathematics of Extent Ordering
+@cindex mathematics of extent ordering
@cindex extent mathematics
-@cindex mathematics of extents
@cindex extent ordering
@cindex display order of extents
@math{S}, including @math{F}. Otherwise, @math{F2} includes @math{I}
and thus is in @math{S}, and thus @math{F2 >= F}.
-@node Extent Fragments, , Mathematics of Extent Ordering, Extents
+@node Extent Fragments
@section Extent Fragments
-@cindex extent fragment
+@cindex extent fragments
+@cindex fragments, extent
Imagine that the buffer is divided up into contiguous, non-overlapping
@dfn{runs} of text such that no extent starts or ends within a run
@node Faces, Glyphs, Extents, Top
@chapter Faces
+@cindex faces
Not yet documented.
@node Glyphs, Specifiers, Faces, Top
@chapter Glyphs
+@cindex glyphs
Glyphs are graphical elements that can be displayed in XEmacs buffers or
gutters. We use the term graphical element here in the broadest possible
image on-screen but the image need not exist at this stage, and multiple
screen images can be instantiated from a single glyph.
+@c #### find a place for this discussion
+@c The decision to make image specifiers a separate type is debatable.
+@c In fact, the design decision to create a separate image specifier
+@c type, rather than make glyphs themselves be specifiers, is
+@c debatable---the other properties of glyphs are rarely used and could
+@c conceivably have been incorporated into the glyph's instantiator.
+@c The rarely used glyph types (buffer, pointer, icon) could also have
+@c been incorporated into the instantiator.
+
Glyphs are lazily instantiated by calling one of the glyph
functions. This usually occurs within redisplay when
@code{Fglyph_height} is called. Instantiation causes an image-instance
-to be created and cached. This cache is on a device basis for all glyphs
-except glyph-widgets, and on a window basis for glyph widgets. The
+to be created and cached. This cache is on a per-device basis for all glyphs
+except widget-glyphs, and on a per-window basis for widgets-glyphs. The
caching is done by @code{image_instantiate} and is necessary because it
is generally possible to display an image-instance in multiple
domains. For instance if we create a Pixmap, we can actually display
Any action on a glyph first consults the cache before actually
instantiating a widget.
+@section Glyph Instantiation
+@cindex glyph instantiation
+@cindex instantiation, glyph
+
+Glyph instantiation is a hairy topic and requires some explanation. The
+guts of glyph instantiation is contained within
+@code{image_instantiate}. A glyph contains an image which is a
+specifier. When a glyph function - for instance @code{Fglyph_height} -
+asks for a property of the glyph that can only be determined from its
+instantiated state, then the glyph image is instantiated and an image
+instance created. The instantiation process is governed by the specifier
+code and goes through a series of steps:
+
+@itemize @bullet
+@item
+Validation. Instantiation of image instances happens dynamically - often
+within the guts of redisplay. Thus it is often not feasible to catch
+instantiator errors at instantiation time. Instead the instantiator is
+validated at the time it is added to the image specifier. This function
+is defined by @code{image_validate} and at a simple level validates
+keyword value pairs.
+@item
+Duplication. The specifier code by default takes a copy of the
+instantiator. This is reasonable for most specifiers but in the case of
+widget-glyphs can be problematic, since some of the properties in the
+instantiator - for instance callbacks - could cause infinite recursion
+in the copying process. Thus the image code defines a function -
+@code{image_copy_instantiator} - which will selectively copy values.
+This is controlled by the way that a keyword is defined either using
+@code{IIFORMAT_VALID_KEYWORD} or
+@code{IIFORMAT_VALID_NONCOPY_KEYWORD}. Note that the image caching and
+redisplay code relies on instantiator copying to ensure that current and
+new instantiators are actually different rather than referring to the
+same thing.
+@item
+Normalization. Once the instantiator has been copied it must be
+converted into a form that is viable at instantiation time. This can
+involve no changes at all, but typically involves things like converting
+file names to the actual data. This function is defined by
+@code{image_going_to_add} and @code{normalize_image_instantiator}.
+@item
+Instantiation. When an image instance is actually required for display
+it is instantiated using @code{image_instantiate}. This involves calling
+instantiate methods that are specific to the type of image being
+instantiated.
+@end itemize
+
+The final instantiation phase also involves a number of steps. In order
+to understand these we need to describe a number of concepts.
+
+An image is instantiated in a @dfn{domain}, where a domain can be any
+one of a device, frame, window or image-instance. The domain gives the
+image-instance context and identity and properties that affect the
+appearance of the image-instance may be different for the same glyph
+instantiated in different domains. An example is the face used to
+display the image-instance.
+
+Although an image is instantiated in a particular domain the
+instantiation domain is not necessarily the domain in which the
+image-instance is cached. For example a pixmap can be instantiated in a
+window be actually be cached on a per-device basis. The domain in which
+the image-instance is actually cached is called the
+@dfn{governing-domain}. A governing-domain is currently either a device
+or a window. Widget-glyphs and text-glyphs have a window as a
+governing-domain, all other image-instances have a device as the
+governing-domain. The governing domain for an image-instance is
+determined using the governing_domain image-instance method.
+
+@section Widget-Glyphs
+@cindex widget-glyphs
+
@section Widget-Glyphs in the MS-Windows Environment
+@cindex widget-glyphs in the MS-Windows environment
+@cindex MS-Windows environment, widget-glyphs in the
To Do
@section Widget-Glyphs in the X Environment
+@cindex widget-glyphs in the X environment
+@cindex X environment, widget-glyphs in the
Widget-glyphs under X make heavy use of lwlib (@pxref{Lucid Widget
Library}) for manipulating the native toolkit objects. This is primarily
so that different toolkits can be supported for widget-glyphs, just as
they are supported for features such as menubars etc.
+Lwlib is extremely poorly documented and quite hairy so here is my
+understanding of what goes on.
+
+Lwlib maintains a set of widget_instances which mirror the hierarchical
+state of Xt widgets. I think this is so that widgets can be updated and
+manipulated generically by the lwlib library. For instance
+update_one_widget_instance can cope with multiple types of widget and
+multiple types of toolkit. Each element in the widget hierarchy is updated
+from its corresponding widget_instance by walking the widget_instance
+tree recursively.
+
+This has desirable properties such as lw_modify_all_widgets which is
+called from @file{glyphs-x.c} and updates all the properties of a widget
+without having to know what the widget is or what toolkit it is from.
+Unfortunately this also has hairy properties such as making the lwlib
+code quite complex. And of course lwlib has to know at some level what
+the widget is and how to set its properties.
+
@node Specifiers, Menus, Glyphs, Top
@chapter Specifiers
+@cindex specifiers
Not yet documented.
@node Menus, Subprocesses, Specifiers, Top
@chapter Menus
+@cindex menus
A menu is set by setting the value of the variable
@code{current-menubar} (which may be buffer-local) and then calling
@node Subprocesses, Interface to the X Window System, Menus, Top
@chapter Subprocesses
+@cindex subprocesses
The fields of a process are:
@node Interface to the X Window System, Index, Subprocesses, Top
@chapter Interface to the X Window System
+@cindex X Window System, interface to the
Mostly undocumented.
* Lucid Widget Library:: An interface to various widget sets.
@end menu
-@node Lucid Widget Library, , , Interface to the X Window System
+@node Lucid Widget Library
@section Lucid Widget Library
+@cindex Lucid Widget Library
+@cindex widget library, Lucid
+@cindex library, Lucid Widget
Lwlib is extremely poorly documented and quite hairy. The author(s)
blame that on X, Xt, and Motif, with some justice, but also sufficient
* Tab Controls::
@end menu
-@node Generic Widget Interface, Scrollbars, , Lucid Widget Library
+@node Generic Widget Interface
@subsection Generic Widget Interface
+@cindex widget interface, generic
In general in any toolkit a widget may be a composite object. In Xt,
all widgets have an X window that they manage, but typically a complex
of its tree. Widget instances are further confi
-@node Scrollbars, Menubars, Generic Widget Interface, Lucid Widget Library
+@node Scrollbars
@subsection Scrollbars
+@cindex scrollbars
-@node Menubars, Checkboxes and Radio Buttons, Scrollbars, Lucid Widget Library
+@node Menubars
@subsection Menubars
+@cindex menubars
-@node Checkboxes and Radio Buttons, Progress Bars, Menubars, Lucid Widget Library
+@node Checkboxes and Radio Buttons
@subsection Checkboxes and Radio Buttons
+@cindex checkboxes and radio buttons
+@cindex radio buttons, checkboxes and
+@cindex buttons, checkboxes and radio
-@node Progress Bars, Tab Controls, Checkboxes and Radio Buttons, Lucid Widget Library
+@node Progress Bars
@subsection Progress Bars
+@cindex progress bars
+@cindex bars, progress
-@node Tab Controls, , Progress Bars, Lucid Widget Library
+@node Tab Controls
@subsection Tab Controls
+@cindex tab controls
@include index.texi
projects / chise / xemacs-chise.git.1 / blobdiff