1 This is ../info/internals.info, produced by makeinfo version 4.8 from
2 internals/internals.texi.
4 INFO-DIR-SECTION XEmacs Editor
6 * Internals: (internals). XEmacs Internals Manual.
9 Copyright (C) 1992 - 1996 Ben Wing. Copyright (C) 1996, 1997 Sun
10 Microsystems. Copyright (C) 1994 - 1998, 2002, 2003 Free Software
11 Foundation. Copyright (C) 1994, 1995 Board of Trustees, University of
14 Permission is granted to make and distribute verbatim copies of this
15 manual provided the copyright notice and this permission notice are
16 preserved on all copies.
18 Permission is granted to copy and distribute modified versions of
19 this manual under the conditions for verbatim copying, provided that the
20 entire resulting derived work is distributed under the terms of a
21 permission notice identical to this one.
23 Permission is granted to copy and distribute translations of this
24 manual into another language, under the above conditions for modified
25 versions, except that this permission notice may be stated in a
26 translation approved by the Foundation.
28 Permission is granted to copy and distribute modified versions of
29 this manual under the conditions for verbatim copying, provided also
30 that the section entitled "GNU General Public License" is included
31 exactly as in the original, and provided that the entire resulting
32 derived work is distributed under the terms of a permission notice
33 identical to this one.
35 Permission is granted to copy and distribute translations of this
36 manual into another language, under the above conditions for modified
37 versions, except that the section entitled "GNU General Public License"
38 may be included in a translation approved by the Free Software
39 Foundation instead of in the original English.
42 File: internals.info, Node: Catch and Throw, Prev: Simple Special Forms, Up: Evaluation; Stack Frames; Bindings
51 struct catchtag *next;
54 struct backtrace *backlist;
59 `catch' is a Lisp function that places a catch around a body of
60 code. A catch is a means of non-local exit from the code. When a catch
61 is created, a tag is specified, and executing a `throw' to this tag
62 will exit from the body of code caught with this tag, and its value will
63 be the value given in the call to `throw'. If there is no such call,
64 the code will be executed normally.
66 Information pertaining to a catch is held in a `struct catchtag',
67 which is placed at the head of a linked list pointed to by `catchlist'.
68 `internal_catch()' is passed a C function to call (`Fprogn()' when
69 Lisp `catch' is called) and arguments to give it, and places a catch
70 around the function. Each `struct catchtag' is held in the stack frame
71 of the `internal_catch()' instance that created the catch.
73 `internal_catch()' is fairly straightforward. It stores into the
74 `struct catchtag' the tag name and the current values of
75 `backtrace_list', `lisp_eval_depth', `gcprolist', and the offset into
76 the `specpdl' array, sets a jump point with `_setjmp()' (storing the
77 jump point into the `struct catchtag'), and calls the function.
78 Control will return to `internal_catch()' either when the function
79 exits normally or through a `_longjmp()' to this jump point. In the
80 latter case, `throw' will store the value to be returned into the
81 `struct catchtag' before jumping. When it's done, `internal_catch()'
82 removes the `struct catchtag' from the catchlist and returns the proper
85 `Fthrow()' goes up through the catchlist until it finds one with a
86 matching tag. It then calls `unbind_catch()' to restore everything to
87 what it was when the appropriate catch was set, stores the return value
88 in the `struct catchtag', and jumps (with `_longjmp()') to its jump
91 `unbind_catch()' removes all catches from the catchlist until it
92 finds the correct one. Some of the catches might have been placed for
93 error-trapping, and if so, the appropriate entries on the handlerlist
94 must be removed (see "errors"). `unbind_catch()' also restores the
95 values of `gcprolist', `backtrace_list', and `lisp_eval', and calls
96 `unbind_to()' to undo any specbindings created since the catch.
99 File: internals.info, Node: Symbols and Variables, Next: Buffers and Textual Representation, Prev: Evaluation; Stack Frames; Bindings, Up: Top
101 15 Symbols and Variables
102 ************************
106 * Introduction to Symbols::
111 File: internals.info, Node: Introduction to Symbols, Next: Obarrays, Up: Symbols and Variables
113 15.1 Introduction to Symbols
114 ============================
116 A symbol is basically just an object with four fields: a name (a
117 string), a value (some Lisp object), a function (some Lisp object), and
118 a property list (usually a list of alternating keyword/value pairs).
119 What makes symbols special is that there is usually only one symbol with
120 a given name, and the symbol is referred to by name. This makes a
121 symbol a convenient way of calling up data by name, i.e. of implementing
122 variables. (The variable's value is stored in the "value slot".)
123 Similarly, functions are referenced by name, and the definition of the
124 function is stored in a symbol's "function slot". This means that
125 there can be a distinct function and variable with the same name. The
126 property list is used as a more general mechanism of associating
127 additional values with particular names, and once again the namespace is
128 independent of the function and variable namespaces.
131 File: internals.info, Node: Obarrays, Next: Symbol Values, Prev: Introduction to Symbols, Up: Symbols and Variables
136 The identity of symbols with their names is accomplished through a
137 structure called an obarray, which is just a poorly-implemented hash
138 table mapping from strings to symbols whose name is that string. (I say
139 "poorly implemented" because an obarray appears in Lisp as a vector
140 with some hidden fields rather than as its own opaque type. This is an
141 Emacs Lisp artifact that should be fixed.)
143 Obarrays are implemented as a vector of some fixed size (which should
144 be a prime for best results), where each "bucket" of the vector
145 contains one or more symbols, threaded through a hidden `next' field in
146 the symbol. Lookup of a symbol in an obarray, and adding a symbol to
147 an obarray, is accomplished through standard hash-table techniques.
149 The standard Lisp function for working with symbols and obarrays is
150 `intern'. This looks up a symbol in an obarray given its name; if it's
151 not found, a new symbol is automatically created with the specified
152 name, added to the obarray, and returned. This is what happens when the
153 Lisp reader encounters a symbol (or more precisely, encounters the name
154 of a symbol) in some text that it is reading. There is a standard
155 obarray called `obarray' that is used for this purpose, although the
156 Lisp programmer is free to create his own obarrays and `intern' symbols
159 Note that, once a symbol is in an obarray, it stays there until
160 something is done about it, and the standard obarray `obarray' always
161 stays around, so once you use any particular variable name, a
162 corresponding symbol will stay around in `obarray' until you exit
165 Note that `obarray' itself is a variable, and as such there is a
166 symbol in `obarray' whose name is `"obarray"' and which contains
167 `obarray' as its value.
169 Note also that this call to `intern' occurs only when in the Lisp
170 reader, not when the code is executed (at which point the symbol is
171 already around, stored as such in the definition of the function).
173 You can create your own obarray using `make-vector' (this is
174 horrible but is an artifact) and intern symbols into that obarray.
175 Doing that will result in two or more symbols with the same name.
176 However, at most one of these symbols is in the standard `obarray': You
177 cannot have two symbols of the same name in any particular obarray.
178 Note that you cannot add a symbol to an obarray in any fashion other
179 than using `intern': i.e. you can't take an existing symbol and put it
180 in an existing obarray. Nor can you change the name of an existing
181 symbol. (Since obarrays are vectors, you can violate the consistency of
182 things by storing directly into the vector, but let's ignore that
185 Usually symbols are created by `intern', but if you really want, you
186 can explicitly create a symbol using `make-symbol', giving it some
187 name. The resulting symbol is not in any obarray (i.e. it is
188 "uninterned"), and you can't add it to any obarray. Therefore its
189 primary purpose is as a symbol to use in macros to avoid namespace
190 pollution. It can also be used as a carrier of information, but cons
191 cells could probably be used just as well.
193 You can also use `intern-soft' to look up a symbol but not create a
194 new one, and `unintern' to remove a symbol from an obarray. This
195 returns the removed symbol. (Remember: You can't put the symbol back
196 into any obarray.) Finally, `mapatoms' maps over all of the symbols in
200 File: internals.info, Node: Symbol Values, Prev: Obarrays, Up: Symbols and Variables
205 The value field of a symbol normally contains a Lisp object. However,
206 a symbol can be "unbound", meaning that it logically has no value.
207 This is internally indicated by storing a special Lisp object, called
208 "the unbound marker" and stored in the global variable `Qunbound'. The
209 unbound marker is of a special Lisp object type called
210 "symbol-value-magic". It is impossible for the Lisp programmer to
211 directly create or access any object of this type.
213 *You must not let any "symbol-value-magic" object escape to the Lisp
214 level.* Printing any of these objects will cause the message `INTERNAL
215 EMACS BUG' to appear as part of the print representation. (You may see
216 this normally when you call `debug_print()' from the debugger on a Lisp
217 object.) If you let one of these objects escape to the Lisp level, you
218 will violate a number of assumptions contained in the C code and make
219 the unbound marker not function right.
221 When a symbol is created, its value field (and function field) are
222 set to `Qunbound'. The Lisp programmer can restore these conditions
223 later using `makunbound' or `fmakunbound', and can query to see whether
224 the value of function fields are "bound" (i.e. have a value other than
225 `Qunbound') using `boundp' and `fboundp'. The fields are set to a
226 normal Lisp object using `set' (or `setq') and `fset'.
228 Other symbol-value-magic objects are used as special markers to
229 indicate variables that have non-normal properties. This includes any
230 variables that are tied into C variables (setting the variable magically
231 sets some global variable in the C code, and likewise for retrieving the
232 variable's value), variables that magically tie into slots in the
233 current buffer, variables that are buffer-local, etc. The
234 symbol-value-magic object is stored in the value cell in place of a
235 normal object, and the code to retrieve a symbol's value (i.e.
236 `symbol-value') knows how to do special things with them. This means
237 that you should not just fetch the value cell directly if you want a
240 The exact workings of this are rather complex and involved and are
241 well-documented in comments in `buffer.c', `symbols.c', and `lisp.h'.
244 File: internals.info, Node: Buffers and Textual Representation, Next: MULE Character Sets and Encodings, Prev: Symbols and Variables, Up: Top
246 16 Buffers and Textual Representation
247 *************************************
251 * Introduction to Buffers:: A buffer holds a block of text such as a file.
252 * The Text in a Buffer:: Representation of the text in a buffer.
253 * Buffer Lists:: Keeping track of all buffers.
254 * Markers and Extents:: Tagging locations within a buffer.
255 * Bufbytes and Emchars:: Representation of individual characters.
256 * The Buffer Object:: The Lisp object corresponding to a buffer.
259 File: internals.info, Node: Introduction to Buffers, Next: The Text in a Buffer, Up: Buffers and Textual Representation
261 16.1 Introduction to Buffers
262 ============================
264 A buffer is logically just a Lisp object that holds some text. In
265 this, it is like a string, but a buffer is optimized for frequent
266 insertion and deletion, while a string is not. Furthermore:
268 1. Buffers are "permanent" objects, i.e. once you create them, they
269 remain around, and need to be explicitly deleted before they go
272 2. Each buffer has a unique name, which is a string. Buffers are
273 normally referred to by name. In this respect, they are like
276 3. Buffers have a default insertion position, called "point".
277 Inserting text (unless you explicitly give a position) goes at
278 point, and moves point forward past the text. This is what is
279 going on when you type text into Emacs.
281 4. Buffers have lots of extra properties associated with them.
283 5. Buffers can be "displayed". What this means is that there exist a
284 number of "windows", which are objects that correspond to some
285 visible section of your display, and each window has an associated
286 buffer, and the current contents of the buffer are shown in that
287 section of the display. The redisplay mechanism (which takes care
288 of doing this) knows how to look at the text of a buffer and come
289 up with some reasonable way of displaying this. Many of the
290 properties of a buffer control how the buffer's text is displayed.
292 6. One buffer is distinguished and called the "current buffer". It is
293 stored in the variable `current_buffer'. Buffer operations operate
294 on this buffer by default. When you are typing text into a
295 buffer, the buffer you are typing into is always `current_buffer'.
296 Switching to a different window changes the current buffer. Note
297 that Lisp code can temporarily change the current buffer using
298 `set-buffer' (often enclosed in a `save-excursion' so that the
299 former current buffer gets restored when the code is finished).
300 However, calling `set-buffer' will NOT cause a permanent change in
301 the current buffer. The reason for this is that the top-level
302 event loop sets `current_buffer' to the buffer of the selected
303 window, each time it finishes executing a user command.
305 Make sure you understand the distinction between "current buffer"
306 and "buffer of the selected window", and the distinction between
307 "point" of the current buffer and "window-point" of the selected
308 window. (This latter distinction is explained in detail in the section
312 File: internals.info, Node: The Text in a Buffer, Next: Buffer Lists, Prev: Introduction to Buffers, Up: Buffers and Textual Representation
314 16.2 The Text in a Buffer
315 =========================
317 The text in a buffer consists of a sequence of zero or more characters.
318 A "character" is an integer that logically represents a letter,
319 number, space, or other unit of text. Most of the characters that you
320 will typically encounter belong to the ASCII set of characters, but
321 there are also characters for various sorts of accented letters,
322 special symbols, Chinese and Japanese ideograms (i.e. Kanji, Katakana,
323 etc.), Cyrillic and Greek letters, etc. The actual number of possible
324 characters is quite large.
326 For now, we can view a character as some non-negative integer that
327 has some shape that defines how it typically appears (e.g. as an
328 uppercase A). (The exact way in which a character appears depends on the
329 font used to display the character.) The internal type of characters in
330 the C code is an `Emchar'; this is just an `int', but using a symbolic
331 type makes the code clearer.
333 Between every character in a buffer is a "buffer position" or
334 "character position". We can speak of the character before or after a
335 particular buffer position, and when you insert a character at a
336 particular position, all characters after that position end up at new
337 positions. When we speak of the character "at" a position, we really
338 mean the character after the position. (This schizophrenia between a
339 buffer position being "between" a character and "on" a character is
342 Buffer positions are numbered starting at 1. This means that
343 position 1 is before the first character, and position 0 is not valid.
344 If there are N characters in a buffer, then buffer position N+1 is
345 after the last one, and position N+2 is not valid.
347 The internal makeup of the Emchar integer varies depending on whether
348 we have compiled with MULE support. If not, the Emchar integer is an
349 8-bit integer with possible values from 0 - 255. 0 - 127 are the
350 standard ASCII characters, while 128 - 255 are the characters from the
351 ISO-8859-1 character set. If we have compiled with MULE support, an
352 Emchar is a 19-bit integer, with the various bits having meanings
353 according to a complex scheme that will be detailed later. The
354 characters numbered 0 - 255 still have the same meanings as for the
355 non-MULE case, though.
357 Internally, the text in a buffer is represented in a fairly simple
358 fashion: as a contiguous array of bytes, with a "gap" of some size in
359 the middle. Although the gap is of some substantial size in bytes,
360 there is no text contained within it: From the perspective of the text
361 in the buffer, it does not exist. The gap logically sits at some buffer
362 position, between two characters (or possibly at the beginning or end of
363 the buffer). Insertion of text in a buffer at a particular position is
364 always accomplished by first moving the gap to that position (i.e.
365 through some block moving of text), then writing the text into the
366 beginning of the gap, thereby shrinking the gap. If the gap shrinks
367 down to nothing, a new gap is created. (What actually happens is that a
368 new gap is "created" at the end of the buffer's text, which requires
369 nothing more than changing a couple of indices; then the gap is "moved"
370 to the position where the insertion needs to take place by moving up in
371 memory all the text after that position.) Similarly, deletion occurs
372 by moving the gap to the place where the text is to be deleted, and
373 then simply expanding the gap to include the deleted text.
374 ("Expanding" and "shrinking" the gap as just described means just that
375 the internal indices that keep track of where the gap is located are
378 Note that the total amount of memory allocated for a buffer text
379 never decreases while the buffer is live. Therefore, if you load up a
380 20-megabyte file and then delete all but one character, there will be a
381 20-megabyte gap, which won't get any smaller (except by inserting
382 characters back again). Once the buffer is killed, the memory allocated
383 for the buffer text will be freed, but it will still be sitting on the
384 heap, taking up virtual memory, and will not be released back to the
385 operating system. (However, if you have compiled XEmacs with rel-alloc,
386 the situation is different. In this case, the space _will_ be released
387 back to the operating system. However, this tends to result in a
388 noticeable speed penalty.)
390 Astute readers may notice that the text in a buffer is represented as
391 an array of _bytes_, while (at least in the MULE case) an Emchar is a
392 19-bit integer, which clearly cannot fit in a byte. This means (of
393 course) that the text in a buffer uses a different representation from
394 an Emchar: specifically, the 19-bit Emchar becomes a series of one to
395 four bytes. The conversion between these two representations is complex
396 and will be described later.
398 In the non-MULE case, everything is very simple: An Emchar is an
399 8-bit value, which fits neatly into one byte.
401 If we are given a buffer position and want to retrieve the character
402 at that position, we need to follow these steps:
404 1. Pretend there's no gap, and convert the buffer position into a
405 "byte index" that indexes to the appropriate byte in the buffer's
406 stream of textual bytes. By convention, byte indices begin at 1,
407 just like buffer positions. In the non-MULE case, byte indices
408 and buffer positions are identical, since one character equals one
411 2. Convert the byte index into a "memory index", which takes the gap
412 into account. The memory index is a direct index into the block of
413 memory that stores the text of a buffer. This basically just
414 involves checking to see if the byte index is past the gap, and if
415 so, adding the size of the gap to it. By convention, memory
416 indices begin at 1, just like buffer positions and byte indices,
417 and when referring to the position that is "at" the gap, we always
418 use the memory position at the _beginning_, not at the end, of the
421 3. Fetch the appropriate bytes at the determined memory position.
423 4. Convert these bytes into an Emchar.
425 In the non-Mule case, (3) and (4) boil down to a simple one-byte
428 Note that we have defined three types of positions in a buffer:
430 1. "buffer positions" or "character positions", typedef `Bufpos'
432 2. "byte indices", typedef `Bytind'
434 3. "memory indices", typedef `Memind'
436 All three typedefs are just `int's, but defining them this way makes
437 things a lot clearer.
439 Most code works with buffer positions. In particular, all Lisp code
440 that refers to text in a buffer uses buffer positions. Lisp code does
441 not know that byte indices or memory indices exist.
443 Finally, we have a typedef for the bytes in a buffer. This is a
444 `Bufbyte', which is an unsigned char. Referring to them as Bufbytes
445 underscores the fact that we are working with a string of bytes in the
446 internal Emacs buffer representation rather than in one of a number of
447 possible alternative representations (e.g. EUC-encoded text, etc.).
450 File: internals.info, Node: Buffer Lists, Next: Markers and Extents, Prev: The Text in a Buffer, Up: Buffers and Textual Representation
455 Recall earlier that buffers are "permanent" objects, i.e. that they
456 remain around until explicitly deleted. This entails that there is a
457 list of all the buffers in existence. This list is actually an
458 assoc-list (mapping from the buffer's name to the buffer) and is stored
459 in the global variable `Vbuffer_alist'.
461 The order of the buffers in the list is important: the buffers are
462 ordered approximately from most-recently-used to least-recently-used.
463 Switching to a buffer using `switch-to-buffer', `pop-to-buffer', etc.
464 and switching windows using `other-window', etc. usually brings the
465 new current buffer to the front of the list. `switch-to-buffer',
466 `other-buffer', etc. look at the beginning of the list to find an
467 alternative buffer to suggest. You can also explicitly move a buffer
468 to the end of the list using `bury-buffer'.
470 In addition to the global ordering in `Vbuffer_alist', each frame
471 has its own ordering of the list. These lists always contain the same
472 elements as in `Vbuffer_alist' although possibly in a different order.
473 `buffer-list' normally returns the list for the selected frame. This
474 allows you to work in separate frames without things interfering with
477 The standard way to look up a buffer given a name is `get-buffer',
478 and the standard way to create a new buffer is `get-buffer-create',
479 which looks up a buffer with a given name, creating a new one if
480 necessary. These operations correspond exactly with the symbol
481 operations `intern-soft' and `intern', respectively. You can also
482 force a new buffer to be created using `generate-new-buffer', which
483 takes a name and (if necessary) makes a unique name from this by
484 appending a number, and then creates the buffer. This is basically
485 like the symbol operation `gensym'.
488 File: internals.info, Node: Markers and Extents, Next: Bufbytes and Emchars, Prev: Buffer Lists, Up: Buffers and Textual Representation
490 16.4 Markers and Extents
491 ========================
493 Among the things associated with a buffer are things that are logically
494 attached to certain buffer positions. This can be used to keep track
495 of a buffer position when text is inserted and deleted, so that it
496 remains at the same spot relative to the text around it; to assign
497 properties to particular sections of text; etc. There are two such
498 objects that are useful in this regard: they are "markers" and
501 A "marker" is simply a flag placed at a particular buffer position,
502 which is moved around as text is inserted and deleted. Markers are
503 used for all sorts of purposes, such as the `mark' that is the other
504 end of textual regions to be cut, copied, etc.
506 An "extent" is similar to two markers plus some associated
507 properties, and is used to keep track of regions in a buffer as text is
508 inserted and deleted, and to add properties (e.g. fonts) to particular
509 regions of text. The external interface of extents is explained
512 The important thing here is that markers and extents simply contain
513 buffer positions in them as integers, and every time text is inserted or
514 deleted, these positions must be updated. In order to minimize the
515 amount of shuffling that needs to be done, the positions in markers and
516 extents (there's one per marker, two per extent) are stored in Meminds.
517 This means that they only need to be moved when the text is physically
518 moved in memory; since the gap structure tries to minimize this, it also
519 minimizes the number of marker and extent indices that need to be
520 adjusted. Look in `insdel.c' for the details of how this works.
522 One other important distinction is that markers are "temporary"
523 while extents are "permanent". This means that markers disappear as
524 soon as there are no more pointers to them, and correspondingly, there
525 is no way to determine what markers are in a buffer if you are just
526 given the buffer. Extents remain in a buffer until they are detached
527 (which could happen as a result of text being deleted) or the buffer is
528 deleted, and primitives do exist to enumerate the extents in a buffer.
531 File: internals.info, Node: Bufbytes and Emchars, Next: The Buffer Object, Prev: Markers and Extents, Up: Buffers and Textual Representation
533 16.5 Bufbytes and Emchars
534 =========================
539 File: internals.info, Node: The Buffer Object, Prev: Bufbytes and Emchars, Up: Buffers and Textual Representation
541 16.6 The Buffer Object
542 ======================
544 Buffers contain fields not directly accessible by the Lisp programmer.
545 We describe them here, naming them by the names used in the C code.
546 Many are accessible indirectly in Lisp programs via Lisp primitives.
549 The buffer name is a string that names the buffer. It is
550 guaranteed to be unique. *Note Buffer Names: (lispref)Buffer
554 This field contains the time when the buffer was last saved, as an
555 integer. *Note Buffer Modification: (lispref)Buffer Modification.
558 This field contains the modification time of the visited file. It
559 is set when the file is written or read. Every time the buffer is
560 written to the file, this field is compared to the modification
561 time of the file. *Note Buffer Modification: (lispref)Buffer
565 This field contains the time when the buffer was last auto-saved.
568 This field contains the `window-start' position in the buffer as of
569 the last time the buffer was displayed in a window.
572 This field points to the buffer's undo list. *Note Undo:
576 This field contains the syntax table for the buffer. *Note Syntax
577 Tables: (lispref)Syntax Tables.
580 This field contains the conversion table for converting text to
581 lower case. *Note Case Tables: (lispref)Case Tables.
584 This field contains the conversion table for converting text to
585 upper case. *Note Case Tables: (lispref)Case Tables.
588 This field contains the conversion table for canonicalizing text
589 for case-folding search. *Note Case Tables: (lispref)Case Tables.
592 This field contains the equivalence table for case-folding search.
593 *Note Case Tables: (lispref)Case Tables.
596 This field contains the buffer's display table, or `nil' if it
597 doesn't have one. *Note Display Tables: (lispref)Display Tables.
600 This field contains the chain of all markers that currently point
601 into the buffer. Deletion of text in the buffer, and motion of
602 the buffer's gap, must check each of these markers and perhaps
603 update it. *Note Markers: (lispref)Markers.
606 This field is a flag that tells whether a backup file has been
607 made for the visited file of this buffer.
610 This field contains the mark for the buffer. The mark is a marker,
611 hence it is also included on the list `markers'. *Note The Mark:
615 This field is non-`nil' if the buffer's mark is active.
618 This field contains the association list describing the variables
619 local in this buffer, and their values, with the exception of
620 local variables that have special slots in the buffer object.
621 (Those slots are omitted from this table.) *Note Buffer-Local
622 Variables: (lispref)Buffer-Local Variables.
625 This field contains a Lisp object which controls how to display
626 the mode line for this buffer. *Note Modeline Format:
627 (lispref)Modeline Format.
630 This field holds the buffer's base buffer (if it is an indirect
634 File: internals.info, Node: MULE Character Sets and Encodings, Next: The Lisp Reader and Compiler, Prev: Buffers and Textual Representation, Up: Top
636 17 MULE Character Sets and Encodings
637 ************************************
639 Recall that there are two primary ways that text is represented in
640 XEmacs. The "buffer" representation sees the text as a series of bytes
641 (Bufbytes), with a variable number of bytes used per character. The
642 "character" representation sees the text as a series of integers
643 (Emchars), one per character. The character representation is a cleaner
644 representation from a theoretical standpoint, and is thus used in many
645 cases when lots of manipulations on a string need to be done. However,
646 the buffer representation is the standard representation used in both
647 Lisp strings and buffers, and because of this, it is the "default"
648 representation that text comes in. The reason for using this
649 representation is that it's compact and is compatible with ASCII.
655 * Internal Mule Encodings::
659 File: internals.info, Node: Character Sets, Next: Encodings, Up: MULE Character Sets and Encodings
664 A character set (or "charset") is an ordered set of characters. A
665 particular character in a charset is indexed using one or more
666 "position codes", which are non-negative integers. The number of
667 position codes needed to identify a particular character in a charset is
668 called the "dimension" of the charset. In XEmacs/Mule, all charsets
669 have dimension 1 or 2, and the size of all charsets (except for a few
670 special cases) is either 94, 96, 94 by 94, or 96 by 96. The range of
671 position codes used to index characters from any of these types of
672 character sets is as follows:
674 Charset type Position code 1 Position code 2
675 ------------------------------------------------------------
678 94x94 33 - 126 33 - 126
679 96x96 32 - 127 32 - 127
681 Note that in the above cases position codes do not start at an
682 expected value such as 0 or 1. The reason for this will become clear
685 For example, Latin-1 is a 96-character charset, and JISX0208 (the
686 Japanese national character set) is a 94x94-character charset.
688 [Note that, although the ranges above define the _valid_ position
689 codes for a charset, some of the slots in a particular charset may in
690 fact be empty. This is the case for JISX0208, for example, where (e.g.)
691 all the slots whose first position code is in the range 118 - 127 are
694 There are three charsets that do not follow the above rules. All of
695 them have one dimension, and have ranges of position codes as follows:
697 Charset name Position code 1
698 ------------------------------------
701 Composite 0 - some large number
703 (The upper bound of the position code for composite characters has
704 not yet been determined, but it will probably be at least 16,383).
706 ASCII is the union of two subsidiary character sets: Printing-ASCII
707 (the printing ASCII character set, consisting of position codes 33 -
708 126, like for a standard 94-character charset) and Control-ASCII (the
709 non-printing characters that would appear in a binary file with codes 0
712 Control-1 contains the non-printing characters that would appear in a
713 binary file with codes 128 - 159.
715 Composite contains characters that are generated by overstriking one
716 or more characters from other charsets.
718 Note that some characters in ASCII, and all characters in Control-1,
719 are "control" (non-printing) characters. These have no printed
720 representation but instead control some other function of the printing
721 (e.g. TAB or 8 moves the current character position to the next tab
722 stop). All other characters in all charsets are "graphic" (printing)
725 When a binary file is read in, the bytes in the file are assigned to
726 character sets as follows:
728 Bytes Character set Range
729 --------------------------------------------------
730 0 - 127 ASCII 0 - 127
731 128 - 159 Control-1 0 - 31
732 160 - 255 Latin-1 32 - 127
734 This is a bit ad-hoc but gets the job done.
737 File: internals.info, Node: Encodings, Next: Internal Mule Encodings, Prev: Character Sets, Up: MULE Character Sets and Encodings
742 An "encoding" is a way of numerically representing characters from one
743 or more character sets. If an encoding only encompasses one character
744 set, then the position codes for the characters in that character set
745 could be used directly. This is not possible, however, if more than
746 one character set is to be used in the encoding.
748 For example, the conversion detailed above between bytes in a binary
749 file and characters is effectively an encoding that encompasses the
750 three character sets ASCII, Control-1, and Latin-1 in a stream of 8-bit
753 Thus, an encoding can be viewed as a way of encoding characters from
754 a specified group of character sets using a stream of bytes, each of
755 which contains a fixed number of bits (but not necessarily 8, as in the
756 common usage of "byte").
758 Here are descriptions of a couple of common encodings:
762 * Japanese EUC (Extended Unix Code)::
766 File: internals.info, Node: Japanese EUC (Extended Unix Code), Next: JIS7, Up: Encodings
768 17.2.1 Japanese EUC (Extended Unix Code)
769 ----------------------------------------
771 This encompasses the character sets Printing-ASCII, Japanese-JISX0201,
772 and Japanese-JISX0208-Kana (half-width katakana, the right half of
773 JISX0201). It uses 8-bit bytes.
775 Note that Printing-ASCII and Japanese-JISX0201-Kana are 94-character
776 charsets, while Japanese-JISX0208 is a 94x94-character charset.
778 The encoding is as follows:
780 Character set Representation (PC=position-code)
781 ------------- --------------
783 Japanese-JISX0201-Kana 0x8E | PC1 + 0x80
784 Japanese-JISX0208 PC1 + 0x80 | PC2 + 0x80
785 Japanese-JISX0212 PC1 + 0x80 | PC2 + 0x80
788 File: internals.info, Node: JIS7, Prev: Japanese EUC (Extended Unix Code), Up: Encodings
793 This encompasses the character sets Printing-ASCII,
794 Japanese-JISX0201-Roman (the left half of JISX0201; this character set
795 is very similar to Printing-ASCII and is a 94-character charset),
796 Japanese-JISX0208, and Japanese-JISX0201-Kana. It uses 7-bit bytes.
798 Unlike Japanese EUC, this is a "modal" encoding, which means that
799 there are multiple states that the encoding can be in, which affect how
800 the bytes are to be interpreted. Special sequences of bytes (called
801 "escape sequences") are used to change states.
803 The encoding is as follows:
805 Character set Representation (PC=position-code)
806 ------------- --------------
808 Japanese-JISX0201-Roman PC1
809 Japanese-JISX0201-Kana PC1
810 Japanese-JISX0208 PC1 PC2
813 Escape sequence ASCII equivalent Meaning
814 --------------- ---------------- -------
815 0x1B 0x28 0x4A ESC ( J invoke Japanese-JISX0201-Roman
816 0x1B 0x28 0x49 ESC ( I invoke Japanese-JISX0201-Kana
817 0x1B 0x24 0x42 ESC $ B invoke Japanese-JISX0208
818 0x1B 0x28 0x42 ESC ( B invoke Printing-ASCII
820 Initially, Printing-ASCII is invoked.
823 File: internals.info, Node: Internal Mule Encodings, Next: CCL, Prev: Encodings, Up: MULE Character Sets and Encodings
825 17.3 Internal Mule Encodings
826 ============================
828 In XEmacs/Mule, each character set is assigned a unique number, called a
829 "leading byte". This is used in the encodings of a character. Leading
830 bytes are in the range 0x80 - 0xFF (except for ASCII, which has a
831 leading byte of 0), although some leading bytes are reserved.
833 Charsets whose leading byte is in the range 0x80 - 0x9F are called
834 "official" and are used for built-in charsets. Other charsets are
835 called "private" and have leading bytes in the range 0xA0 - 0xFF; these
836 are user-defined charsets.
840 Character set Leading byte
841 ------------- ------------
844 Dimension-1 Official 0x81 - 0x8D
847 Dimension-2 Official 0x90 - 0x99
848 (0x9A - 0x9D are free;
849 0x9E and 0x9F are reserved)
850 Dimension-1 Private 0xA0 - 0xEF
851 Dimension-2 Private 0xF0 - 0xFF
853 There are two internal encodings for characters in XEmacs/Mule. One
854 is called "string encoding" and is an 8-bit encoding that is used for
855 representing characters in a buffer or string. It uses 1 to 4 bytes per
856 character. The other is called "character encoding" and is a 19-bit
857 encoding that is used for representing characters individually in a
860 (In the following descriptions, we'll ignore composite characters for
861 the moment. We also give a general (structural) overview first,
862 followed later by the exact details.)
866 * Internal String Encoding::
867 * Internal Character Encoding::
870 File: internals.info, Node: Internal String Encoding, Next: Internal Character Encoding, Up: Internal Mule Encodings
872 17.3.1 Internal String Encoding
873 -------------------------------
875 ASCII characters are encoded using their position code directly. Other
876 characters are encoded using their leading byte followed by their
877 position code(s) with the high bit set. Characters in private character
878 sets have their leading byte prefixed with a "leading byte prefix",
879 which is either 0x9E or 0x9F. (No character sets are ever assigned these
880 leading bytes.) Specifically:
882 Character set Encoding (PC=position-code, LB=leading-byte)
883 ------------- --------
885 Control-1 LB | PC1 + 0xA0 |
886 Dimension-1 official LB | PC1 + 0x80 |
887 Dimension-1 private 0x9E | LB | PC1 + 0x80 |
888 Dimension-2 official LB | PC1 + 0x80 | PC2 + 0x80 |
889 Dimension-2 private 0x9F | LB | PC1 + 0x80 | PC2 + 0x80
891 The basic characteristic of this encoding is that the first byte of
892 all characters is in the range 0x00 - 0x9F, and the second and
893 following bytes of all characters is in the range 0xA0 - 0xFF. This
894 means that it is impossible to get out of sync, or more specifically:
896 1. Given any byte position, the beginning of the character it is
897 within can be determined in constant time.
899 2. Given any byte position at the beginning of a character, the
900 beginning of the next character can be determined in constant time.
902 3. Given any byte position at the beginning of a character, the
903 beginning of the previous character can be determined in constant
906 4. Textual searches can simply treat encoded strings as if they were
907 encoded in a one-byte-per-character fashion rather than the actual
910 None of the standard non-modal encodings meet all of these
911 conditions. For example, EUC satisfies only (2) and (3), while
912 Shift-JIS and Big5 (not yet described) satisfy only (2). (All non-modal
913 encodings must satisfy (2), in order to be unambiguous.)
916 File: internals.info, Node: Internal Character Encoding, Prev: Internal String Encoding, Up: Internal Mule Encodings
918 17.3.2 Internal Character Encoding
919 ----------------------------------
921 One 19-bit word represents a single character. The word is separated
924 Bit number: 18 17 16 15 14 13 12 11 10 09 08 07 06 05 04 03 02 01 00
925 <------------> <------------------> <------------------>
928 Note that fields 2 and 3 hold 7 bits each, while field 1 holds 5
931 Character set Field 1 Field 2 Field 3
932 ------------- ------- ------- -------
937 Dimension-1 official 0 LB - 0x80 PC1
938 range: (01 - 0D) (20 - 7F)
939 Dimension-1 private 0 LB - 0x80 PC1
940 range: (20 - 6F) (20 - 7F)
941 Dimension-2 official LB - 0x8F PC1 PC2
942 range: (01 - 0A) (20 - 7F) (20 - 7F)
943 Dimension-2 private LB - 0xE1 PC1 PC2
944 range: (0F - 1E) (20 - 7F) (20 - 7F)
947 Note that character codes 0 - 255 are the same as the "binary
948 encoding" described above.
951 File: internals.info, Node: CCL, Prev: Internal Mule Encodings, Up: MULE Character Sets and Encodings
957 CCL_PROGRAM := (CCL_MAIN_BLOCK
960 CCL_MAIN_BLOCK := CCL_BLOCK
961 CCL_EOF_BLOCK := CCL_BLOCK
963 CCL_BLOCK := STATEMENT | (STATEMENT [STATEMENT ...])
965 SET | IF | BRANCH | LOOP | REPEAT | BREAK
968 SET := (REG = EXPRESSION) | (REG SELF_OP EXPRESSION)
971 EXPRESSION := ARG | (EXPRESSION OP ARG)
973 IF := (if EXPRESSION CCL_BLOCK CCL_BLOCK)
974 BRANCH := (branch EXPRESSION CCL_BLOCK [CCL_BLOCK ...])
975 LOOP := (loop STATEMENT [STATEMENT ...])
978 | (write-repeat [REG | INT-OR-CHAR | string])
979 | (write-read-repeat REG [INT-OR-CHAR | string | ARRAY]?)
980 READ := (read REG) | (read REG REG)
981 | (read-if REG ARITH_OP ARG CCL_BLOCK CCL_BLOCK)
982 | (read-branch REG CCL_BLOCK [CCL_BLOCK ...])
983 WRITE := (write REG) | (write REG REG)
984 | (write INT-OR-CHAR) | (write STRING) | STRING
988 REG := r0 | r1 | r2 | r3 | r4 | r5 | r6 | r7
989 ARG := REG | INT-OR-CHAR
990 OP := + | - | * | / | % | & | '|' | ^ | << | >> | <8 | >8 | //
991 | < | > | == | <= | >= | !=
993 += | -= | *= | /= | %= | &= | '|=' | ^= | <<= | >>=
994 ARRAY := '[' INT-OR-CHAR ... ']'
995 INT-OR-CHAR := INT | CHAR
999 The machine code consists of a vector of 32-bit words.
1000 The first such word specifies the start of the EOF section of the code;
1001 this is the code executed to handle any stuff that needs to be done
1002 (e.g. designating back to ASCII and left-to-right mode) after all
1003 other encoded/decoded data has been written out. This is not used for
1004 charset CCL programs.
1006 REGISTER: 0..7 -- referred by RRR or rrr
1008 OPERATOR BIT FIELD (27-bit): XXXXXXXXXXXXXXX RRR TTTTT
1009 TTTTT (5-bit): operator type
1010 RRR (3-bit): register number
1011 XXXXXXXXXXXXXXXX (15-bit):
1012 CCCCCCCCCCCCCCC: constant or address
1013 000000000000rrr: register number
1040 OPERATORS: TTTTT RRR XX..
1042 SetCS: 00000 RRR C...C RRR = C...C
1043 SetCL: 00001 RRR ..... RRR = c...c
1045 SetR: 00010 RRR ..rrr RRR = rrr
1046 SetA: 00011 RRR ..rrr RRR = array[rrr]
1047 C.............C size of array = C...C
1048 c.............c contents = c...c
1050 Jump: 00100 000 c...c jump to c...c
1051 JumpCond: 00101 RRR c...c if (!RRR) jump to c...c
1052 WriteJump: 00110 RRR c...c Write1 RRR, jump to c...c
1053 WriteReadJump: 00111 RRR c...c Write1, Read1 RRR, jump to c...c
1054 WriteCJump: 01000 000 c...c Write1 C...C, jump to c...c
1056 WriteCReadJump: 01001 RRR c...c Write1 C...C, Read1 RRR,
1057 C.............C and jump to c...c
1058 WriteSJump: 01010 000 c...c WriteS, jump to c...c
1062 WriteSReadJump: 01011 RRR c...c WriteS, Read1 RRR, jump to c...c
1066 WriteAReadJump: 01100 RRR c...c WriteA, Read1 RRR, jump to c...c
1067 C.............C size of array = C...C
1068 c.............c contents = c...c
1070 Branch: 01101 RRR C...C if (RRR >= 0 && RRR < C..)
1071 c.............c branch to (RRR+1)th address
1072 Read1: 01110 RRR ... read 1-byte to RRR
1073 Read2: 01111 RRR ..rrr read 2-byte to RRR and rrr
1074 ReadBranch: 10000 RRR C...C Read1 and Branch
1077 Write1: 10001 RRR ..... write 1-byte RRR
1078 Write2: 10010 RRR ..rrr write 2-byte RRR and rrr
1079 WriteC: 10011 000 ..... write 1-char C...CC
1081 WriteS: 10100 000 ..... write C..-byte of string
1085 WriteA: 10101 RRR ..... write array[RRR]
1086 C.............C size of array = C...C
1087 c.............c contents = c...c
1089 End: 10110 000 ..... terminate the execution
1091 SetSelfCS: 10111 RRR C...C RRR AAAAA= C...C
1093 SetSelfCL: 11000 RRR ..... RRR AAAAA= c...c
1096 SetSelfR: 11001 RRR ..Rrr RRR AAAAA= rrr
1098 SetExprCL: 11010 RRR ..Rrr RRR = rrr AAAAA c...c
1101 SetExprR: 11011 RRR ..rrr RRR = rrr AAAAA Rrr
1104 JumpCondC: 11100 RRR c...c if !(RRR AAAAA C..) jump to c...c
1107 JumpCondR: 11101 RRR c...c if !(RRR AAAAA rrr) jump to c...c
1110 ReadJumpCondC: 11110 RRR c...c Read1 and JumpCondC
1113 ReadJumpCondR: 11111 RRR c...c Read1 and JumpCondR
1118 File: internals.info, Node: The Lisp Reader and Compiler, Next: Lstreams, Prev: MULE Character Sets and Encodings, Up: Top
1120 18 The Lisp Reader and Compiler
1121 *******************************
1126 File: internals.info, Node: Lstreams, Next: Consoles; Devices; Frames; Windows, Prev: The Lisp Reader and Compiler, Up: Top
1131 An "lstream" is an internal Lisp object that provides a generic
1132 buffering stream implementation. Conceptually, you send data to the
1133 stream or read data from the stream, not caring what's on the other end
1134 of the stream. The other end could be another stream, a file
1135 descriptor, a stdio stream, a fixed block of memory, a reallocating
1136 block of memory, etc. The main purpose of the stream is to provide a
1137 standard interface and to do buffering. Macros are defined to read or
1138 write characters, so the calling functions do not have to worry about
1139 blocking data together in order to achieve efficiency.
1143 * Creating an Lstream:: Creating an lstream object.
1144 * Lstream Types:: Different sorts of things that are streamed.
1145 * Lstream Functions:: Functions for working with lstreams.
1146 * Lstream Methods:: Creating new lstream types.
1149 File: internals.info, Node: Creating an Lstream, Next: Lstream Types, Up: Lstreams
1151 19.1 Creating an Lstream
1152 ========================
1154 Lstreams come in different types, depending on what is being interfaced
1155 to. Although the primitive for creating new lstreams is
1156 `Lstream_new()', generally you do not call this directly. Instead, you
1157 call some type-specific creation function, which creates the lstream
1158 and initializes it as appropriate for the particular type.
1160 All lstream creation functions take a MODE argument, specifying what
1161 mode the lstream should be opened as. This controls whether the
1162 lstream is for input and output, and optionally whether data should be
1163 blocked up in units of MULE characters. Note that some types of
1164 lstreams can only be opened for input; others only for output; and
1165 others can be opened either way. #### Richard Mlynarik thinks that
1166 there should be a strict separation between input and output streams,
1167 and he's probably right.
1169 MODE is a string, one of
1178 Open for reading, but "read" never returns partial MULE characters.
1181 Open for writing, but never writes partial MULE characters.
1184 File: internals.info, Node: Lstream Types, Next: Lstream Functions, Prev: Creating an Lstream, Up: Lstreams
1210 File: internals.info, Node: Lstream Functions, Next: Lstream Methods, Prev: Lstream Types, Up: Lstreams
1212 19.3 Lstream Functions
1213 ======================
1215 -- Function: Lstream * Lstream_new (Lstream_implementation *IMP, const
1217 Allocate and return a new Lstream. This function is not really
1218 meant to be called directly; rather, each stream type should
1219 provide its own stream creation function, which creates the stream
1220 and does any other necessary creation stuff (e.g. opening a file).
1222 -- Function: void Lstream_set_buffering (Lstream *LSTR,
1223 Lstream_buffering BUFFERING, int BUFFERING_SIZE)
1224 Change the buffering of a stream. See `lstream.h'. By default the
1225 buffering is `STREAM_BLOCK_BUFFERED'.
1227 -- Function: int Lstream_flush (Lstream *LSTR)
1228 Flush out any pending unwritten data in the stream. Clear any
1229 buffered input data. Returns 0 on success, -1 on error.
1231 -- Macro: int Lstream_putc (Lstream *STREAM, int C)
1232 Write out one byte to the stream. This is a macro and so it is
1233 very efficient. The C argument is only evaluated once but the
1234 STREAM argument is evaluated more than once. Returns 0 on
1235 success, -1 on error.
1237 -- Macro: int Lstream_getc (Lstream *STREAM)
1238 Read one byte from the stream. This is a macro and so it is very
1239 efficient. The STREAM argument is evaluated more than once.
1240 Return value is -1 for EOF or error.
1242 -- Macro: void Lstream_ungetc (Lstream *STREAM, int C)
1243 Push one byte back onto the input queue. This will be the next
1244 byte read from the stream. Any number of bytes can be pushed back
1245 and will be read in the reverse order they were pushed back--most
1246 recent first. (This is necessary for consistency--if there are a
1247 number of bytes that have been unread and I read and unread a
1248 byte, it needs to be the first to be read again.) This is a macro
1249 and so it is very efficient. The C argument is only evaluated
1250 once but the STREAM argument is evaluated more than once.
1252 -- Function: int Lstream_fputc (Lstream *STREAM, int C)
1253 -- Function: int Lstream_fgetc (Lstream *STREAM)
1254 -- Function: void Lstream_fungetc (Lstream *STREAM, int C)
1255 Function equivalents of the above macros.
1257 -- Function: ssize_t Lstream_read (Lstream *STREAM, void *DATA, size_t
1259 Read SIZE bytes of DATA from the stream. Return the number of
1260 bytes read. 0 means EOF. -1 means an error occurred and no bytes
1263 -- Function: ssize_t Lstream_write (Lstream *STREAM, void *DATA,
1265 Write SIZE bytes of DATA to the stream. Return the number of
1266 bytes written. -1 means an error occurred and no bytes were
1269 -- Function: void Lstream_unread (Lstream *STREAM, void *DATA, size_t
1271 Push back SIZE bytes of DATA onto the input queue. The next call
1272 to `Lstream_read()' with the same size will read the same bytes
1273 back. Note that this will be the case even if there is other
1274 pending unread data.
1276 -- Function: int Lstream_close (Lstream *STREAM)
1277 Close the stream. All data will be flushed out.
1279 -- Function: void Lstream_reopen (Lstream *STREAM)
1280 Reopen a closed stream. This enables I/O on it again. This is not
1281 meant to be called except from a wrapper routine that reinitializes
1282 variables and such--the close routine may well have freed some
1283 necessary storage structures, for example.
1285 -- Function: void Lstream_rewind (Lstream *STREAM)
1286 Rewind the stream to the beginning.
1289 File: internals.info, Node: Lstream Methods, Prev: Lstream Functions, Up: Lstreams
1291 19.4 Lstream Methods
1292 ====================
1294 -- Lstream Method: ssize_t reader (Lstream *STREAM, unsigned char
1296 Read some data from the stream's end and store it into DATA, which
1297 can hold SIZE bytes. Return the number of bytes read. A return
1298 value of 0 means no bytes can be read at this time. This may be
1299 because of an EOF, or because there is a granularity greater than
1300 one byte that the stream imposes on the returned data, and SIZE is
1301 less than this granularity. (This will happen frequently for
1302 streams that need to return whole characters, because
1303 `Lstream_read()' calls the reader function repeatedly until it has
1304 the number of bytes it wants or until 0 is returned.) The lstream
1305 functions do not treat a 0 return as EOF or do anything special;
1306 however, the calling function will interpret any 0 it gets back as
1307 EOF. This will normally not happen unless the caller calls
1308 `Lstream_read()' with a very small size.
1310 This function can be `NULL' if the stream is output-only.
1312 -- Lstream Method: ssize_t writer (Lstream *STREAM, const unsigned
1313 char *DATA, size_t SIZE)
1314 Send some data to the stream's end. Data to be sent is in DATA
1315 and is SIZE bytes. Return the number of bytes sent. This
1316 function can send and return fewer bytes than is passed in; in that
1317 case, the function will just be called again until there is no
1318 data left or 0 is returned. A return value of 0 means that no
1319 more data can be currently stored, but there is no error; the data
1320 will be squirreled away until the writer can accept data. (This is
1321 useful, e.g., if you're dealing with a non-blocking file
1322 descriptor and are getting `EWOULDBLOCK' errors.) This function
1323 can be `NULL' if the stream is input-only.
1325 -- Lstream Method: int rewinder (Lstream *STREAM)
1326 Rewind the stream. If this is `NULL', the stream is not seekable.
1328 -- Lstream Method: int seekable_p (Lstream *STREAM)
1329 Indicate whether this stream is seekable--i.e. it can be rewound.
1330 This method is ignored if the stream does not have a rewind
1331 method. If this method is not present, the result is determined
1332 by whether a rewind method is present.
1334 -- Lstream Method: int flusher (Lstream *STREAM)
1335 Perform any additional operations necessary to flush the data in
1338 -- Lstream Method: int pseudo_closer (Lstream *STREAM)
1340 -- Lstream Method: int closer (Lstream *STREAM)
1341 Perform any additional operations necessary to close this stream
1342 down. May be `NULL'. This function is called when
1343 `Lstream_close()' is called or when the stream is
1344 garbage-collected. When this function is called, all pending data
1345 in the stream will already have been written out.
1347 -- Lstream Method: Lisp_Object marker (Lisp_Object LSTREAM, void
1348 (*MARKFUN) (Lisp_Object))
1349 Mark this object for garbage collection. Same semantics as a
1350 standard `Lisp_Object' marker. This function can be `NULL'.
1353 File: internals.info, Node: Consoles; Devices; Frames; Windows, Next: The Redisplay Mechanism, Prev: Lstreams, Up: Top
1355 20 Consoles; Devices; Frames; Windows
1356 *************************************
1360 * Introduction to Consoles; Devices; Frames; Windows::
1362 * Window Hierarchy::
1363 * The Window Object::
1366 File: internals.info, Node: Introduction to Consoles; Devices; Frames; Windows, Next: Point, Up: Consoles; Devices; Frames; Windows
1368 20.1 Introduction to Consoles; Devices; Frames; Windows
1369 =======================================================
1371 A window-system window that you see on the screen is called a "frame"
1372 in Emacs terminology. Each frame is subdivided into one or more
1373 non-overlapping panes, called (confusingly) "windows". Each window
1374 displays the text of a buffer in it. (See above on Buffers.) Note that
1375 buffers and windows are independent entities: Two or more windows can
1376 be displaying the same buffer (potentially in different locations), and
1377 a buffer can be displayed in no windows.
1379 A single display screen that contains one or more frames is called a
1380 "display". Under most circumstances, there is only one display.
1381 However, more than one display can exist, for example if you have a
1382 "multi-headed" console, i.e. one with a single keyboard but multiple
1383 displays. (Typically in such a situation, the various displays act like
1384 one large display, in that the mouse is only in one of them at a time,
1385 and moving the mouse off of one moves it into another.) In some cases,
1386 the different displays will have different characteristics, e.g. one
1389 XEmacs can display frames on multiple displays. It can even deal
1390 simultaneously with frames on multiple keyboards (called "consoles" in
1391 XEmacs terminology). Here is one case where this might be useful: You
1392 are using XEmacs on your workstation at work, and leave it running.
1393 Then you go home and dial in on a TTY line, and you can use the
1394 already-running XEmacs process to display another frame on your local
1397 Thus, there is a hierarchy console -> display -> frame -> window.
1398 There is a separate Lisp object type for each of these four concepts.
1399 Furthermore, there is logically a "selected console", "selected
1400 display", "selected frame", and "selected window". Each of these
1401 objects is distinguished in various ways, such as being the default
1402 object for various functions that act on objects of that type. Note
1403 that every containing object remembers the "selected" object among the
1404 objects that it contains: e.g. not only is there a selected window, but
1405 every frame remembers the last window in it that was selected, and
1406 changing the selected frame causes the remembered window within it to
1407 become the selected window. Similar relationships apply for consoles
1408 to devices and devices to frames.
1411 File: internals.info, Node: Point, Next: Window Hierarchy, Prev: Introduction to Consoles; Devices; Frames; Windows, Up: Consoles; Devices; Frames; Windows
1416 Recall that every buffer has a current insertion position, called
1417 "point". Now, two or more windows may be displaying the same buffer,
1418 and the text cursor in the two windows (i.e. `point') can be in two
1419 different places. You may ask, how can that be, since each buffer has
1420 only one value of `point'? The answer is that each window also has a
1421 value of `point' that is squirreled away in it. There is only one
1422 selected window, and the value of "point" in that buffer corresponds to
1423 that window. When the selected window is changed from one window to
1424 another displaying the same buffer, the old value of `point' is stored
1425 into the old window's "point" and the value of `point' from the new
1426 window is retrieved and made the value of `point' in the buffer. This
1427 means that `window-point' for the selected window is potentially
1428 inaccurate, and if you want to retrieve the correct value of `point'
1429 for a window, you must special-case on the selected window and retrieve
1430 the buffer's point instead. This is related to why
1431 `save-window-excursion' does not save the selected window's value of
1435 File: internals.info, Node: Window Hierarchy, Next: The Window Object, Prev: Point, Up: Consoles; Devices; Frames; Windows
1437 20.3 Window Hierarchy
1438 =====================
1440 If a frame contains multiple windows (panes), they are always created
1441 by splitting an existing window along the horizontal or vertical axis.
1442 Terminology is a bit confusing here: to "split a window horizontally"
1443 means to create two side-by-side windows, i.e. to make a _vertical_ cut
1444 in a window. Likewise, to "split a window vertically" means to create
1445 two windows, one above the other, by making a _horizontal_ cut.
1447 If you split a window and then split again along the same axis, you
1448 will end up with a number of panes all arranged along the same axis.
1449 The precise way in which the splits were made should not be important,
1450 and this is reflected internally. Internally, all windows are arranged
1451 in a tree, consisting of two types of windows, "combination" windows
1452 (which have children, and are covered completely by those children) and
1453 "leaf" windows, which have no children and are visible. Every
1454 combination window has two or more children, all arranged along the same
1455 axis. There are (logically) two subtypes of windows, depending on
1456 whether their children are horizontally or vertically arrayed. There is
1457 always one root window, which is either a leaf window (if the frame
1458 contains only one window) or a combination window (if the frame contains
1459 more than one window). In the latter case, the root window will have
1460 two or more children, either horizontally or vertically arrayed, and
1461 each of those children will be either a leaf window or another
1464 Here are some rules:
1466 1. Horizontal combination windows can never have children that are
1467 horizontal combination windows; same for vertical.
1469 2. Only leaf windows can be split (obviously) and this splitting does
1470 one of two things: (a) turns the leaf window into a combination
1471 window and creates two new leaf children, or (b) turns the leaf
1472 window into one of the two new leaves and creates the other leaf.
1473 Rule (1) dictates which of these two outcomes happens.
1475 3. Every combination window must have at least two children.
1477 4. Leaf windows can never become combination windows. They can be
1478 deleted, however. If this results in a violation of (3), the
1479 parent combination window also gets deleted.
1481 5. All functions that accept windows must be prepared to accept
1482 combination windows, and do something sane (e.g. signal an error
1483 if so). Combination windows _do_ escape to the Lisp level.
1485 6. All windows have three fields governing their contents: these are
1486 "hchild" (a list of horizontally-arrayed children), "vchild" (a
1487 list of vertically-arrayed children), and "buffer" (the buffer
1488 contained in a leaf window). Exactly one of these will be
1489 non-`nil'. Remember that "horizontally-arrayed" means
1490 "side-by-side" and "vertically-arrayed" means "one above the
1493 7. Leaf windows also have markers in their `start' (the first buffer
1494 position displayed in the window) and `pointm' (the window's
1495 stashed value of `point'--see above) fields, while combination
1496 windows have `nil' in these fields.
1498 8. The list of children for a window is threaded through the `next'
1499 and `prev' fields of each child window.
1501 9. *Deleted windows can be undeleted*. This happens as a result of
1502 restoring a window configuration, and is unlike frames, displays,
1503 and consoles, which, once deleted, can never be restored.
1504 Deleting a window does nothing except set a special `dead' bit to
1505 1 and clear out the `next', `prev', `hchild', and `vchild' fields,
1508 10. Most frames actually have two top-level windows--one for the
1509 minibuffer and one (the "root") for everything else. The modeline
1510 (if present) separates these two. The `next' field of the root
1511 points to the minibuffer, and the `prev' field of the minibuffer
1512 points to the root. The other `next' and `prev' fields are `nil',
1513 and the frame points to both of these windows. Minibuffer-less
1514 frames have no minibuffer window, and the `next' and `prev' of the
1515 root window are `nil'. Minibuffer-only frames have no root
1516 window, and the `next' of the minibuffer window is `nil' but the
1517 `prev' points to itself. (#### This is an artifact that should be
1521 File: internals.info, Node: The Window Object, Prev: Window Hierarchy, Up: Consoles; Devices; Frames; Windows
1523 20.4 The Window Object
1524 ======================
1526 Windows have the following accessible fields:
1529 The frame that this window is on.
1532 Non-`nil' if this window is a minibuffer window.
1535 The buffer that the window is displaying. This may change often
1536 during the life of the window.
1539 Non-`nil' if this window is dedicated to its buffer.
1542 This is the value of point in the current buffer when this window
1543 is selected; when it is not selected, it retains its previous
1547 The position in the buffer that is the first character to be
1548 displayed in the window.
1551 If this flag is non-`nil', it says that the window has been
1552 scrolled explicitly by the Lisp program. This affects what the
1553 next redisplay does if point is off the screen: instead of
1554 scrolling the window to show the text around point, it moves point
1555 to a location that is on the screen.
1558 The `modified' field of the window's buffer, as of the last time a
1559 redisplay completed in this window.
1562 The buffer's value of point, as of the last time a redisplay
1563 completed in this window.
1566 This is the left-hand edge of the window, measured in columns.
1567 (The leftmost column on the screen is column 0.)
1570 This is the top edge of the window, measured in lines. (The top
1571 line on the screen is line 0.)
1574 The height of the window, measured in lines.
1577 The width of the window, measured in columns.
1580 This is the window that is the next in the chain of siblings. It
1581 is `nil' in a window that is the rightmost or bottommost of a
1585 This is the window that is the previous in the chain of siblings.
1586 It is `nil' in a window that is the leftmost or topmost of a group
1590 Internally, XEmacs arranges windows in a tree; each group of
1591 siblings has a parent window whose area includes all the siblings.
1592 This field points to a window's parent.
1594 Parent windows do not display buffers, and play little role in
1595 display except to shape their child windows. Emacs Lisp programs
1596 usually have no access to the parent windows; they operate on the
1597 windows at the leaves of the tree, which actually display buffers.
1600 This is the number of columns that the display in the window is
1601 scrolled horizontally to the left. Normally, this is 0.
1604 This is the last time that the window was selected. The function
1605 `get-lru-window' uses this field.
1608 The window's display table, or `nil' if none is specified for it.
1611 Non-`nil' means this window's mode line needs to be updated.
1614 The line number of a certain position in the buffer, or `nil'.
1615 This is used for displaying the line number of point in the mode
1619 The position in the buffer for which the line number is known, or
1620 `nil' meaning none is known.
1623 If the region (or part of it) is highlighted in this window, this
1624 field holds the mark position that made one end of that region.
1625 Otherwise, this field is `nil'.
1628 File: internals.info, Node: The Redisplay Mechanism, Next: Extents, Prev: Consoles; Devices; Frames; Windows, Up: Top
1630 21 The Redisplay Mechanism
1631 **************************
1633 The redisplay mechanism is one of the most complicated sections of
1634 XEmacs, especially from a conceptual standpoint. This is doubly so
1635 because, unlike for the basic aspects of the Lisp interpreter, the
1636 computer science theories of how to efficiently handle redisplay are not
1639 When working with the redisplay mechanism, remember the Golden Rules
1642 1. It Is Better To Be Correct Than Fast.
1644 2. Thou Shalt Not Run Elisp From Within Redisplay.
1646 3. It Is Better To Be Fast Than Not To Be.
1650 * Critical Redisplay Sections::
1651 * Line Start Cache::
1652 * Redisplay Piece by Piece::
1655 File: internals.info, Node: Critical Redisplay Sections, Next: Line Start Cache, Up: The Redisplay Mechanism
1657 21.1 Critical Redisplay Sections
1658 ================================
1660 Within this section, we are defenseless and assume that the following
1663 1. garbage collection
1665 2. Lisp code evaluation
1667 3. frame size changes
1669 We ensure (3) by calling `hold_frame_size_changes()', which will
1670 cause any pending frame size changes to get put on hold till after the
1671 end of the critical section. (1) follows automatically if (2) is met.
1672 #### Unfortunately, there are some places where Lisp code can be called
1673 within this section. We need to remove them.
1675 If `Fsignal()' is called during this critical section, we will
1678 If garbage collection is called during this critical section, we
1679 simply return. #### We should abort instead.
1681 #### If a frame-size change does occur we should probably actually
1682 be preempting redisplay.
1685 File: internals.info, Node: Line Start Cache, Next: Redisplay Piece by Piece, Prev: Critical Redisplay Sections, Up: The Redisplay Mechanism
1687 21.2 Line Start Cache
1688 =====================
1690 The traditional scrolling code in Emacs breaks in a variable height
1691 world. It depends on the key assumption that the number of lines that
1692 can be displayed at any given time is fixed. This led to a complete
1693 separation of the scrolling code from the redisplay code. In order to
1694 fully support variable height lines, the scrolling code must actually be
1695 tightly integrated with redisplay. Only redisplay can determine how
1696 many lines will be displayed on a screen for any given starting point.
1698 What is ideally wanted is a complete list of the starting buffer
1699 position for every possible display line of a buffer along with the
1700 height of that display line. Maintaining such a full list would be very
1701 expensive. We settle for having it include information for all areas
1702 which we happen to generate anyhow (i.e. the region currently being
1703 displayed) and for those areas we need to work with.
1705 In order to ensure that the cache accurately represents what
1706 redisplay would actually show, it is necessary to invalidate it in many
1707 situations. If the buffer changes, the starting positions may no longer
1708 be correct. If a face or an extent has changed then the line heights
1709 may have altered. These events happen frequently enough that the cache
1710 can end up being constantly disabled. With this potentially constant
1711 invalidation when is the cache ever useful?
1713 Even if the cache is invalidated before every single usage, it is
1714 necessary. Scrolling often requires knowledge about display lines which
1715 are actually above or below the visible region. The cache provides a
1716 convenient light-weight method of storing this information for multiple
1717 display regions. This knowledge is necessary for the scrolling code to
1718 always obey the First Golden Rule of Redisplay.
1720 If the cache already contains all of the information that the
1721 scrolling routines happen to need so that it doesn't have to go
1722 generate it, then we are able to obey the Third Golden Rule of
1723 Redisplay. The first thing we do to help out the cache is to always
1724 add the displayed region. This region had to be generated anyway, so
1725 the cache ends up getting the information basically for free. In those
1726 cases where a user is simply scrolling around viewing a buffer there is
1727 a high probability that this is sufficient to always provide the needed
1728 information. The second thing we can do is be smart about invalidating
1731 TODO--Be smart about invalidating the cache. Potential places:
1733 * Insertions at end-of-line which don't cause line-wraps do not
1734 alter the starting positions of any display lines. These types of
1735 buffer modifications should not invalidate the cache. This is
1736 actually a large optimization for redisplay speed as well.
1738 * Buffer modifications frequently only affect the display of lines
1739 at and below where they occur. In these situations we should only
1740 invalidate the part of the cache starting at where the
1741 modification occurs.
1743 In case you're wondering, the Second Golden Rule of Redisplay is not
1747 File: internals.info, Node: Redisplay Piece by Piece, Prev: Line Start Cache, Up: The Redisplay Mechanism
1749 21.3 Redisplay Piece by Piece
1750 =============================
1752 As you can begin to see redisplay is complex and also not well
1753 documented. Chuck no longer works on XEmacs so this section is my take
1754 on the workings of redisplay.
1756 Redisplay happens in three phases:
1758 1. Determine desired display in area that needs redisplay.
1759 Implemented by `redisplay.c'
1761 2. Compare desired display with current display Implemented by
1762 `redisplay-output.c'
1764 3. Output changes Implemented by `redisplay-output.c',
1765 `redisplay-x.c', `redisplay-msw.c' and `redisplay-tty.c'
1767 Steps 1 and 2 are device-independent and relatively complex. Step 3
1768 is mostly device-dependent.
1770 Determining the desired display
1772 Display attributes are stored in `display_line' structures. Each
1773 `display_line' consists of a set of `display_block''s and each
1774 `display_block' contains a number of `rune''s. Generally dynarr's of
1775 `display_line''s are held by each window representing the current
1776 display and the desired display.
1778 The `display_line' structures are tightly tied to buffers which
1779 presents a problem for redisplay as this connection is bogus for the
1780 modeline. Hence the `display_line' generation routines are duplicated
1781 for generating the modeline. This means that the modeline display code
1782 has many bugs that the standard redisplay code does not.
1784 The guts of `display_line' generation are in `create_text_block',
1785 which creates a single display line for the desired locale. This
1786 incrementally parses the characters on the current line and generates
1787 redisplay structures for each.
1789 Gutter redisplay is different. Because the data to display is stored
1790 in a string we cannot use `create_text_block'. Instead we use
1791 `create_text_string_block' which performs the same function as
1792 `create_text_block' but for strings. Many of the complexities of
1793 `create_text_block' to do with cursor handling and selective display
1797 File: internals.info, Node: Extents, Next: Faces, Prev: The Redisplay Mechanism, Up: Top
1804 * Introduction to Extents:: Extents are ranges over text, with properties.
1805 * Extent Ordering:: How extents are ordered internally.
1806 * Format of the Extent Info:: The extent information in a buffer or string.
1807 * Zero-Length Extents:: A weird special case.
1808 * Mathematics of Extent Ordering:: A rigorous foundation.
1809 * Extent Fragments:: Cached information useful for redisplay.
1812 File: internals.info, Node: Introduction to Extents, Next: Extent Ordering, Up: Extents
1814 22.1 Introduction to Extents
1815 ============================
1817 Extents are regions over a buffer, with a start and an end position
1818 denoting the region of the buffer included in the extent. In addition,
1819 either end can be closed or open, meaning that the endpoint is or is
1820 not logically included in the extent. Insertion of a character at a
1821 closed endpoint causes the character to go inside the extent; insertion
1822 at an open endpoint causes the character to go outside.
1824 Extent endpoints are stored using memory indices (see `insdel.c'),
1825 to minimize the amount of adjusting that needs to be done when
1826 characters are inserted or deleted.
1828 (Formerly, extent endpoints at the gap could be either before or
1829 after the gap, depending on the open/closedness of the endpoint. The
1830 intent of this was to make it so that insertions would automatically go
1831 inside or out of extents as necessary with no further work needing to
1832 be done. It didn't work out that way, however, and just ended up
1833 complexifying and buggifying all the rest of the code.)
1836 File: internals.info, Node: Extent Ordering, Next: Format of the Extent Info, Prev: Introduction to Extents, Up: Extents
1838 22.2 Extent Ordering
1839 ====================
1841 Extents are compared using memory indices. There are two orderings for
1842 extents and both orders are kept current at all times. The normal or
1843 "display" order is as follows:
1845 Extent A is ``less than'' extent B,
1846 that is, earlier in the display order,
1847 if: A-start < B-start,
1848 or if: A-start = B-start, and A-end > B-end
1850 So if two extents begin at the same position, the larger of them is
1851 the earlier one in the display order (`EXTENT_LESS' is true).
1853 For the e-order, the same thing holds:
1855 Extent A is ``less than'' extent B in e-order,
1856 that is, later in the buffer,
1858 or if: A-end = B-end, and A-start > B-start
1860 So if two extents end at the same position, the smaller of them is
1861 the earlier one in the e-order (`EXTENT_E_LESS' is true).
1863 The display order and the e-order are complementary orders: any
1864 theorem about the display order also applies to the e-order if you swap
1865 all occurrences of "display order" and "e-order", "less than" and
1866 "greater than", and "extent start" and "extent end".
1869 File: internals.info, Node: Format of the Extent Info, Next: Zero-Length Extents, Prev: Extent Ordering, Up: Extents
1871 22.3 Format of the Extent Info
1872 ==============================
1874 An extent-info structure consists of a list of the buffer or string's
1875 extents and a "stack of extents" that lists all of the extents over a
1876 particular position. The stack-of-extents info is used for
1877 optimization purposes--it basically caches some info that might be
1878 expensive to compute. Certain otherwise hard computations are easy
1879 given the stack of extents over a particular position, and if the stack
1880 of extents over a nearby position is known (because it was calculated
1881 at some prior point in time), it's easy to move the stack of extents to
1882 the proper position.
1884 Given that the stack of extents is an optimization, and given that
1885 it requires memory, a string's stack of extents is wiped out each time
1886 a garbage collection occurs. Therefore, any time you retrieve the
1887 stack of extents, it might not be there. If you need it to be there,
1888 use the `_force' version.
1890 Similarly, a string may or may not have an extent_info structure.
1891 (Generally it won't if there haven't been any extents added to the
1892 string.) So use the `_force' version if you need the extent_info
1893 structure to be there.
1895 A list of extents is maintained as a double gap array: one gap array
1896 is ordered by start index (the "display order") and the other is
1897 ordered by end index (the "e-order"). Note that positions in an extent
1898 list should logically be conceived of as referring _to_ a particular
1899 extent (as is the norm in programs) rather than sitting between two
1900 extents. Note also that callers of these functions should not be aware
1901 of the fact that the extent list is implemented as an array, except for
1902 the fact that positions are integers (this should be generalized to
1903 handle integers and linked list equally well).
1906 File: internals.info, Node: Zero-Length Extents, Next: Mathematics of Extent Ordering, Prev: Format of the Extent Info, Up: Extents
1908 22.4 Zero-Length Extents
1909 ========================
1911 Extents can be zero-length, and will end up that way if their endpoints
1912 are explicitly set that way or if their detachable property is `nil'
1913 and all the text in the extent is deleted. (The exception is open-open
1914 zero-length extents, which are barred from existing because there is no
1915 sensible way to define their properties. Deletion of the text in an
1916 open-open extent causes it to be converted into a closed-open extent.)
1917 Zero-length extents are primarily used to represent annotations, and
1920 1. Insertion at the position of a zero-length extent expands the
1921 extent if both endpoints are closed; goes after the extent if it
1922 is closed-open; and goes before the extent if it is open-closed.
1924 2. Deletion of a character on a side of a zero-length extent whose
1925 corresponding endpoint is closed causes the extent to be detached
1926 if it is detachable; if the extent is not detachable or the
1927 corresponding endpoint is open, the extent remains in the buffer,
1928 moving as necessary.
1930 Note that closed-open, non-detachable zero-length extents behave
1931 exactly like markers and that open-closed, non-detachable zero-length
1932 extents behave like the "point-type" marker in Mule.
1935 File: internals.info, Node: Mathematics of Extent Ordering, Next: Extent Fragments, Prev: Zero-Length Extents, Up: Extents
1937 22.5 Mathematics of Extent Ordering
1938 ===================================
1940 The extents in a buffer are ordered by "display order" because that is
1941 that order that the redisplay mechanism needs to process them in. The
1942 e-order is an auxiliary ordering used to facilitate operations over
1943 extents. The operations that can be performed on the ordered list of
1944 extents in a buffer are
1946 1. Locate where an extent would go if inserted into the list.
1948 2. Insert an extent into the list.
1950 3. Remove an extent from the list.
1952 4. Map over all the extents that overlap a range.
1954 (4) requires being able to determine the first and last extents that
1957 NOTE: "overlap" is used as follows:
1959 * two ranges overlap if they have at least one point in common.
1960 Whether the endpoints are open or closed makes a difference here.
1962 * a point overlaps a range if the point is contained within the
1963 range; this is equivalent to treating a point P as the range [P,
1966 * In the case of an _extent_ overlapping a point or range, the extent
1967 is normally treated as having closed endpoints. This applies
1968 consistently in the discussion of stacks of extents and such below.
1969 Note that this definition of overlap is not necessarily consistent
1970 with the extents that `map-extents' maps over, since `map-extents'
1971 sometimes pays attention to whether the endpoints of an extents
1972 are open or closed. But for our purposes, it greatly simplifies
1973 things to treat all extents as having closed endpoints.
1975 First, define >, <, <=, etc. as applied to extents to mean
1976 comparison according to the display order. Comparison between an
1977 extent E and an index I means comparison between E and the range [I, I].
1979 Also define e>, e<, e<=, etc. to mean comparison according to the
1982 For any range R, define R(0) to be the starting index of the range
1983 and R(1) to be the ending index of the range.
1985 For any extent E, define E(next) to be the extent directly following
1986 E, and E(prev) to be the extent directly preceding E. Assume E(next)
1987 and E(prev) can be determined from E in constant time. (This is
1988 because we store the extent list as a doubly linked list.)
1990 Similarly, define E(e-next) and E(e-prev) to be the extents directly
1991 following and preceding E in the e-order.
1995 Let R be a range. Let F be the first extent overlapping R. Let L
1996 be the last extent overlapping R.
1998 Theorem 1: R(1) lies between L and L(next), i.e. L <= R(1) < L(next).
2000 This follows easily from the definition of display order. The basic
2001 reason that this theorem applies is that the display order sorts by
2002 increasing starting index.
2004 Therefore, we can determine L just by looking at where we would
2005 insert R(1) into the list, and if we know F and are moving forward over
2006 extents, we can easily determine when we've hit L by comparing the
2007 extent we're at to R(1).
2009 Theorem 2: F(e-prev) e< [1, R(0)] e<= F.
2011 This is the analog of Theorem 1, and applies because the e-order
2012 sorts by increasing ending index.
2014 Therefore, F can be found in the same amount of time as operation
2015 (1), i.e. the time that it takes to locate where an extent would go if
2016 inserted into the e-order list.
2018 If the lists were stored as balanced binary trees, then operation (1)
2019 would take logarithmic time, which is usually quite fast. However,
2020 currently they're stored as simple doubly-linked lists, and instead we
2021 do some caching to try to speed things up.
2023 Define a "stack of extents" (or "SOE") as the set of extents
2024 (ordered in the display order) that overlap an index I, together with
2025 the SOE's "previous" extent, which is an extent that precedes I in the
2026 e-order. (Hopefully there will not be very many extents between I and
2027 the previous extent.)
2031 Let I be an index, let S be the stack of extents on I, let F be the
2032 first extent in S, and let P be S's previous extent.
2034 Theorem 3: The first extent in S is the first extent that overlaps
2037 Proof: Any extent that overlaps [I, J] but does not include I must
2038 have a start index > I, and thus be greater than any extent in S.
2040 Therefore, finding the first extent that overlaps a range R is the
2041 same as finding the first extent that overlaps R(0).
2043 Theorem 4: Let I2 be an index such that I2 > I, and let F2 be the
2044 first extent that overlaps I2. Then, either F2 is in S or F2 is
2045 greater than any extent in S.
2047 Proof: If F2 does not include I then its start index is greater than
2048 I and thus it is greater than any extent in S, including F. Otherwise,
2049 F2 includes I and thus is in S, and thus F2 >= F.
2052 File: internals.info, Node: Extent Fragments, Prev: Mathematics of Extent Ordering, Up: Extents
2054 22.6 Extent Fragments
2055 =====================
2057 Imagine that the buffer is divided up into contiguous, non-overlapping
2058 "runs" of text such that no extent starts or ends within a run (extents
2059 that abut the run don't count).
2061 An extent fragment is a structure that holds data about the run that
2062 contains a particular buffer position (if the buffer position is at the
2063 junction of two runs, the run after the position is used)--the
2064 beginning and end of the run, a list of all of the extents in that run,
2065 the "merged face" that results from merging all of the faces
2066 corresponding to those extents, the begin and end glyphs at the
2067 beginning of the run, etc. This is the information that redisplay needs
2068 in order to display this run.
2070 Extent fragments have to be very quick to update to a new buffer
2071 position when moving linearly through the buffer. They rely on the
2072 stack-of-extents code, which does the heavy-duty algorithmic work of
2073 determining which extents overly a particular position.
2076 File: internals.info, Node: Faces, Next: Glyphs, Prev: Extents, Up: Top
2084 File: internals.info, Node: Glyphs, Next: Specifiers, Prev: Faces, Up: Top
2089 Glyphs are graphical elements that can be displayed in XEmacs buffers or
2090 gutters. We use the term graphical element here in the broadest possible
2091 sense since glyphs can be as mundane as text or as arcane as a native
2094 In XEmacs, glyphs represent the uninstantiated state of graphical
2095 elements, i.e. they hold all the information necessary to produce an
2096 image on-screen but the image need not exist at this stage, and multiple
2097 screen images can be instantiated from a single glyph.
2099 Glyphs are lazily instantiated by calling one of the glyph
2100 functions. This usually occurs within redisplay when `Fglyph_height' is
2101 called. Instantiation causes an image-instance to be created and
2102 cached. This cache is on a per-device basis for all glyphs except
2103 widget-glyphs, and on a per-window basis for widgets-glyphs. The
2104 caching is done by `image_instantiate' and is necessary because it is
2105 generally possible to display an image-instance in multiple domains.
2106 For instance if we create a Pixmap, we can actually display this on
2107 multiple windows - even though we only need a single Pixmap instance to
2108 do this. If caching wasn't done then it would be necessary to create
2109 image-instances for every displayable occurrence of a glyph - and every
2110 usage - and this would be extremely memory and cpu intensive.
2112 Widget-glyphs (a.k.a native widgets) are not cached in this way.
2113 This is because widget-glyph image-instances on screen are toolkit
2114 windows, and thus cannot be reused in multiple XEmacs domains. Thus
2115 widget-glyphs are cached on an XEmacs window basis.
2117 Any action on a glyph first consults the cache before actually
2118 instantiating a widget.
2120 24.1 Glyph Instantiation
2121 ========================
2123 Glyph instantiation is a hairy topic and requires some explanation. The
2124 guts of glyph instantiation is contained within `image_instantiate'. A
2125 glyph contains an image which is a specifier. When a glyph function -
2126 for instance `Fglyph_height' - asks for a property of the glyph that
2127 can only be determined from its instantiated state, then the glyph
2128 image is instantiated and an image instance created. The instantiation
2129 process is governed by the specifier code and goes through a series of
2132 * Validation. Instantiation of image instances happens dynamically -
2133 often within the guts of redisplay. Thus it is often not feasible
2134 to catch instantiator errors at instantiation time. Instead the
2135 instantiator is validated at the time it is added to the image
2136 specifier. This function is defined by `image_validate' and at a
2137 simple level validates keyword value pairs.
2139 * Duplication. The specifier code by default takes a copy of the
2140 instantiator. This is reasonable for most specifiers but in the
2141 case of widget-glyphs can be problematic, since some of the
2142 properties in the instantiator - for instance callbacks - could
2143 cause infinite recursion in the copying process. Thus the image
2144 code defines a function - `image_copy_instantiator' - which will
2145 selectively copy values. This is controlled by the way that a
2146 keyword is defined either using `IIFORMAT_VALID_KEYWORD' or
2147 `IIFORMAT_VALID_NONCOPY_KEYWORD'. Note that the image caching and
2148 redisplay code relies on instantiator copying to ensure that
2149 current and new instantiators are actually different rather than
2150 referring to the same thing.
2152 * Normalization. Once the instantiator has been copied it must be
2153 converted into a form that is viable at instantiation time. This
2154 can involve no changes at all, but typically involves things like
2155 converting file names to the actual data. This function is defined
2156 by `image_going_to_add' and `normalize_image_instantiator'.
2158 * Instantiation. When an image instance is actually required for
2159 display it is instantiated using `image_instantiate'. This
2160 involves calling instantiate methods that are specific to the type
2161 of image being instantiated.
2163 The final instantiation phase also involves a number of steps. In
2164 order to understand these we need to describe a number of concepts.
2166 An image is instantiated in a "domain", where a domain can be any
2167 one of a device, frame, window or image-instance. The domain gives the
2168 image-instance context and identity and properties that affect the
2169 appearance of the image-instance may be different for the same glyph
2170 instantiated in different domains. An example is the face used to
2171 display the image-instance.
2173 Although an image is instantiated in a particular domain the
2174 instantiation domain is not necessarily the domain in which the
2175 image-instance is cached. For example a pixmap can be instantiated in a
2176 window be actually be cached on a per-device basis. The domain in which
2177 the image-instance is actually cached is called the "governing-domain".
2178 A governing-domain is currently either a device or a window.
2179 Widget-glyphs and text-glyphs have a window as a governing-domain, all
2180 other image-instances have a device as the governing-domain. The
2181 governing domain for an image-instance is determined using the
2182 governing_domain image-instance method.
2187 24.3 Widget-Glyphs in the MS-Windows Environment
2188 ================================================
2192 24.4 Widget-Glyphs in the X Environment
2193 =======================================
2195 Widget-glyphs under X make heavy use of lwlib (*note Lucid Widget
2196 Library::) for manipulating the native toolkit objects. This is
2197 primarily so that different toolkits can be supported for
2198 widget-glyphs, just as they are supported for features such as menubars
2201 Lwlib is extremely poorly documented and quite hairy so here is my
2202 understanding of what goes on.
2204 Lwlib maintains a set of widget_instances which mirror the
2205 hierarchical state of Xt widgets. I think this is so that widgets can
2206 be updated and manipulated generically by the lwlib library. For
2207 instance update_one_widget_instance can cope with multiple types of
2208 widget and multiple types of toolkit. Each element in the widget
2209 hierarchy is updated from its corresponding widget_instance by walking
2210 the widget_instance tree recursively.
2212 This has desirable properties such as lw_modify_all_widgets which is
2213 called from `glyphs-x.c' and updates all the properties of a widget
2214 without having to know what the widget is or what toolkit it is from.
2215 Unfortunately this also has hairy properties such as making the lwlib
2216 code quite complex. And of course lwlib has to know at some level what
2217 the widget is and how to set its properties.
2220 File: internals.info, Node: Specifiers, Next: Menus, Prev: Glyphs, Up: Top
2228 File: internals.info, Node: Menus, Next: Subprocesses, Prev: Specifiers, Up: Top
2233 A menu is set by setting the value of the variable `current-menubar'
2234 (which may be buffer-local) and then calling `set-menubar-dirty-flag'
2235 to signal a change. This will cause the menu to be redrawn at the next
2236 redisplay. The format of the data in `current-menubar' is described in
2239 Internally the data in current-menubar is parsed into a tree of
2240 `widget_value's' (defined in `lwlib.h'); this is accomplished by the
2241 recursive function `menu_item_descriptor_to_widget_value()', called by
2242 `compute_menubar_data()'. Such a tree is deallocated using
2243 `free_widget_value()'.
2245 `update_screen_menubars()' is one of the external entry points.
2246 This checks to see, for each screen, if that screen's menubar needs to
2247 be updated. This is the case if
2249 1. `set-menubar-dirty-flag' was called since the last redisplay.
2250 (This function sets the C variable menubar_has_changed.)
2252 2. The buffer displayed in the screen has changed.
2254 3. The screen has no menubar currently displayed.
2256 `set_screen_menubar()' is called for each such screen. This
2257 function calls `compute_menubar_data()' to create the tree of
2258 widget_value's, then calls `lw_create_widget()',
2259 `lw_modify_all_widgets()', and/or `lw_destroy_all_widgets()' to create
2260 the X-Toolkit widget associated with the menu.
2262 `update_psheets()', the other external entry point, actually changes
2263 the menus being displayed. It uses the widgets fixed by
2264 `update_screen_menubars()' and calls various X functions to ensure that
2265 the menus are displayed properly.
2267 The menubar widget is set up so that `pre_activate_callback()' is
2268 called when the menu is first selected (i.e. mouse button goes down),
2269 and `menubar_selection_callback()' is called when an item is selected.
2270 `pre_activate_callback()' calls the function in activate-menubar-hook,
2271 which can change the menubar (this is described in `menubar.c'). If
2272 the menubar is changed, `set_screen_menubars()' is called.
2273 `menubar_selection_callback()' enqueues a menu event, putting in it a
2274 function to call (either `eval' or `call-interactively') and its
2275 argument, which is the callback function or form given in the menu's
2279 File: internals.info, Node: Subprocesses, Next: Interface to the X Window System, Prev: Menus, Up: Top
2284 The fields of a process are:
2287 A string, the name of the process.
2290 A list containing the command arguments that were used to start
2294 A function used to accept output from the process instead of a
2298 A function called whenever the process receives a signal, or `nil'.
2301 The associated buffer of the process.
2304 An integer, the Unix process ID.
2307 A flag, non-`nil' if this is really a child process. It is `nil'
2308 for a network connection.
2311 A marker indicating the position of the end of the last output
2312 from this process inserted into the buffer. This is often but not
2313 always the end of the buffer.
2315 `kill_without_query'
2316 If this is non-`nil', killing XEmacs while this process is still
2317 running does not ask for confirmation about killing the process.
2321 These two fields record 16 bits each of the process status
2322 returned by the `wait' system call.
2325 The process status, as `process-status' should return it.
2329 If these two fields are not equal, a change in the status of the
2330 process needs to be reported, either by running the sentinel or by
2331 inserting a message in the process buffer.
2334 Non-`nil' if communication with the subprocess uses a PTY; `nil'
2338 The file descriptor for input from the process.
2341 The file descriptor for output to the process.
2344 The file descriptor for the terminal that the subprocess is using.
2345 (On some systems, there is no need to record this, so the value is
2349 The name of the terminal that the subprocess is using, or `nil' if
2353 File: internals.info, Node: Interface to the X Window System, Next: Index, Prev: Subprocesses, Up: Top
2355 28 Interface to the X Window System
2356 ***********************************
2358 Mostly undocumented.
2362 * Lucid Widget Library:: An interface to various widget sets.
2365 File: internals.info, Node: Lucid Widget Library, Up: Interface to the X Window System
2367 28.1 Lucid Widget Library
2368 =========================
2370 Lwlib is extremely poorly documented and quite hairy. The author(s)
2371 blame that on X, Xt, and Motif, with some justice, but also sufficient
2372 hypocrisy to avoid drawing the obvious conclusion about their own work.
2374 The Lucid Widget Library is composed of two more or less independent
2375 pieces. The first, as the name suggests, is a set of widgets. These
2376 widgets are intended to resemble and improve on widgets provided in the
2377 Motif toolkit but not in the Athena widgets, including menubars and
2378 scrollbars. Recent additions by Andy Piper integrate some "modern"
2379 widgets by Edward Falk, including checkboxes, radio buttons, progress
2380 gauges, and index tab controls (aka notebooks).
2382 The second piece of the Lucid widget library is a generic interface
2383 to several toolkits for X (including Xt, the Athena widget set, and
2384 Motif, as well as the Lucid widgets themselves) so that core XEmacs
2385 code need not know which widget set has been used to build the
2386 graphical user interface.
2390 * Generic Widget Interface:: The lwlib generic widget interface.
2393 * Checkboxes and Radio Buttons::
2398 File: internals.info, Node: Generic Widget Interface, Next: Scrollbars, Up: Lucid Widget Library
2400 28.1.1 Generic Widget Interface
2401 -------------------------------
2403 In general in any toolkit a widget may be a composite object. In Xt,
2404 all widgets have an X window that they manage, but typically a complex
2405 widget will have widget children, each of which manages a subwindow of
2406 the parent widget's X window. These children may themselves be
2407 composite widgets. Thus a widget is actually a tree or hierarchy of
2410 For each toolkit widget, lwlib maintains a tree of `widget_values'
2411 which mirror the hierarchical state of Xt widgets (including Motif,
2412 Athena, 3D Athena, and Falk's widget sets). Each `widget_value' has
2413 `contents' member, which points to the head of a linked list of its
2414 children. The linked list of siblings is chained through the `next'
2415 member of `widget_value'.
2423 +-------+ next +-------+ next +-------+
2424 | child |----->| child |----->| child |
2425 +-------+ +-------+ +-------+
2429 +-------------+ next +-------------+
2430 | grand child |----->| grand child |
2431 +-------------+ +-------------+
2433 The `widget_value' hierarchy of a composite widget with two simple
2434 children and one composite child.
2436 The `widget_instance' structure maintains the inverse view of the
2437 tree. As for the `widget_value', siblings are chained through the
2438 `next' member. However, rather than naming children, the
2439 `widget_instance' tree links to parents.
2447 +-------+ next +-------+ next +-------+
2448 | child |----->| child |----->| child |
2449 +-------+ +-------+ +-------+
2453 +-------------+ next +-------------+
2454 | grand child |----->| grand child |
2455 +-------------+ +-------------+
2457 The `widget_value' hierarchy of a composite widget with two simple
2458 children and one composite child.
2460 This permits widgets derived from different toolkits to be updated
2461 and manipulated generically by the lwlib library. For instance
2462 `update_one_widget_instance' can cope with multiple types of widget and
2463 multiple types of toolkit. Each element in the widget hierarchy is
2464 updated from its corresponding `widget_value' by walking the
2465 `widget_value' tree. This has desirable properties. For example,
2466 `lw_modify_all_widgets' is called from `glyphs-x.c' and updates all the
2467 properties of a widget without having to know what the widget is or
2468 what toolkit it is from. Unfortunately this also has its hairy
2469 properties; the lwlib code quite complex. And of course lwlib has to
2470 know at some level what the widget is and how to set its properties.
2472 The `widget_instance' structure also contains a pointer to the root
2473 of its tree. Widget instances are further confi
2476 File: internals.info, Node: Scrollbars, Next: Menubars, Prev: Generic Widget Interface, Up: Lucid Widget Library
2482 File: internals.info, Node: Menubars, Next: Checkboxes and Radio Buttons, Prev: Scrollbars, Up: Lucid Widget Library
2488 File: internals.info, Node: Checkboxes and Radio Buttons, Next: Progress Bars, Prev: Menubars, Up: Lucid Widget Library
2490 28.1.4 Checkboxes and Radio Buttons
2491 -----------------------------------
2494 File: internals.info, Node: Progress Bars, Next: Tab Controls, Prev: Checkboxes and Radio Buttons, Up: Lucid Widget Library
2496 28.1.5 Progress Bars
2497 --------------------
2500 File: internals.info, Node: Tab Controls, Prev: Progress Bars, Up: Lucid Widget Library
2506 File: internals.info, Node: Index, Prev: Interface to the X Window System, Up: Top
2514 * allocation from frob blocks: Allocation from Frob Blocks.
2516 * allocation of objects in XEmacs Lisp: Allocation of Objects in XEmacs Lisp.
2518 * allocation, introduction to: Introduction to Allocation.
2520 * allocation, low-level: Low-level allocation.
2522 * Amdahl Corporation: XEmacs. (line 6)
2523 * Andreessen, Marc: XEmacs. (line 6)
2524 * asynchronous subprocesses: Modules for Interfacing with the Operating System.
2526 * bars, progress: Progress Bars. (line 6)
2527 * Baur, Steve: XEmacs. (line 6)
2528 * Benson, Eric: Lucid Emacs. (line 28)
2529 * binding; the specbinding stack; unwind-protects, dynamic: Dynamic Binding; The specbinding Stack; Unwind-Protects.
2531 * bindings, evaluation; stack frames;: Evaluation; Stack Frames; Bindings.
2533 * bit vector: Bit Vector. (line 6)
2534 * bridge, playing: XEmacs From the Outside.
2536 * Buchholz, Martin: XEmacs. (line 6)
2537 * Bufbyte: Character-Related Data Types.
2539 * Bufbytes and Emchars: Bufbytes and Emchars.
2541 * buffer lists: Buffer Lists. (line 6)
2542 * buffer object, the: The Buffer Object. (line 6)
2543 * buffer, the text in a: The Text in a Buffer.
2545 * buffers and textual representation: Buffers and Textual Representation.
2547 * buffers, introduction to: Introduction to Buffers.
2549 * Bufpos: Character-Related Data Types.
2551 * building, XEmacs from the perspective of: XEmacs From the Perspective of Building.
2553 * buttons, checkboxes and radio: Checkboxes and Radio Buttons.
2555 * byte positions, working with character and: Working With Character and Byte Positions.
2557 * Bytecount: Character-Related Data Types.
2559 * bytecount_to_charcount: Working With Character and Byte Positions.
2561 * Bytind: Character-Related Data Types.
2563 * C code, rules when writing new: Rules When Writing New C Code.
2565 * C vs. Lisp: The Lisp Language. (line 6)
2566 * callback routines, the event stream: The Event Stream Callback Routines.
2568 * caller-protects (GCPRO rule): Writing Lisp Primitives.
2570 * case table: Modules for Other Aspects of the Lisp Interpreter and Object System.
2572 * catch and throw: Catch and Throw. (line 6)
2573 * CCL: CCL. (line 6)
2574 * character and byte positions, working with: Working With Character and Byte Positions.
2576 * character encoding, internal: Internal Character Encoding.
2578 * character sets: Character Sets. (line 6)
2579 * character sets and encodings, Mule: MULE Character Sets and Encodings.
2581 * character-related data types: Character-Related Data Types.
2583 * characters, integers and: Integers and Characters.
2585 * Charcount: Character-Related Data Types.
2587 * charcount_to_bytecount: Working With Character and Byte Positions.
2589 * charptr_emchar: Working With Character and Byte Positions.
2591 * charptr_n_addr: Working With Character and Byte Positions.
2593 * checkboxes and radio buttons: Checkboxes and Radio Buttons.
2595 * closer: Lstream Methods. (line 53)
2596 * closure: The XEmacs Object System (Abstractly Speaking).
2598 * code, an example of Mule-aware: An Example of Mule-Aware Code.
2600 * code, general guidelines for writing Mule-aware: General Guidelines for Writing Mule-Aware Code.
2602 * code, rules when writing new C: Rules When Writing New C Code.
2604 * coding conventions: A Reader's Guide to XEmacs Coding Conventions.
2606 * coding for Mule: Coding for Mule. (line 6)
2607 * coding rules, general: General Coding Rules.
2609 * coding rules, naming: A Reader's Guide to XEmacs Coding Conventions.
2611 * command builder, dispatching events; the: Dispatching Events; The Command Builder.
2613 * comments, writing good: Writing Good Comments.
2615 * Common Lisp: The Lisp Language. (line 6)
2616 * compact_string_chars: compact_string_chars.
2618 * compiled function: Compiled Function. (line 6)
2619 * compiler, the Lisp reader and: The Lisp Reader and Compiler.
2621 * cons: Cons. (line 6)
2622 * conservative garbage collection: GCPROing. (line 131)
2623 * consoles; devices; frames; windows: Consoles; Devices; Frames; Windows.
2625 * consoles; devices; frames; windows, introduction to: Introduction to Consoles; Devices; Frames; Windows.
2627 * control flow modules, editor-level: Editor-Level Control Flow Modules.
2629 * conversion to and from external data: Conversion to and from External Data.
2631 * converting events: Converting Events. (line 6)
2632 * copy-on-write: General Coding Rules.
2634 * creating Lisp object types: Techniques for XEmacs Developers.
2636 * critical redisplay sections: Critical Redisplay Sections.
2638 * data dumping: Data dumping. (line 6)
2639 * data types, character-related: Character-Related Data Types.
2641 * DEC_CHARPTR: Working With Character and Byte Positions.
2643 * developers, techniques for XEmacs: Techniques for XEmacs Developers.
2645 * devices; frames; windows, consoles;: Consoles; Devices; Frames; Windows.
2647 * devices; frames; windows, introduction to consoles;: Introduction to Consoles; Devices; Frames; Windows.
2649 * Devin, Matthieu: Lucid Emacs. (line 28)
2650 * dispatching events; the command builder: Dispatching Events; The Command Builder.
2652 * display order of extents: Mathematics of Extent Ordering.
2654 * display-related Lisp objects, modules for other: Modules for other Display-Related Lisp Objects.
2656 * displayable Lisp objects, modules for the basic: Modules for the Basic Displayable Lisp Objects.
2658 * dumping: Dumping. (line 6)
2659 * dumping address allocation: Address allocation. (line 6)
2660 * dumping and its justification, what is: Dumping. (line 9)
2661 * dumping data descriptions: Data descriptions. (line 6)
2662 * dumping object inventory: Object inventory. (line 6)
2663 * dumping overview: Overview. (line 6)
2664 * dumping phase: Dumping phase. (line 6)
2665 * dumping, data: Data dumping. (line 6)
2666 * dumping, file loading: Reloading phase. (line 9)
2667 * dumping, object relocation: Reloading phase. (line 32)
2668 * dumping, pointers: Pointers dumping. (line 6)
2669 * dumping, putting back the pdump_opaques: Reloading phase. (line 21)
2670 * dumping, putting back the pdump_root_objects and pdump_weak_object_chains: Reloading phase.
2672 * dumping, putting back the pdump_root_struct_ptrs: Reloading phase.
2674 * dumping, reloading phase: Reloading phase. (line 6)
2675 * dumping, remaining issues: Remaining issues. (line 6)
2676 * dumping, reorganize the hash tables: Reloading phase. (line 44)
2677 * dumping, the header: The header. (line 6)
2678 * dynamic array: Low-Level Modules. (line 146)
2679 * dynamic binding; the specbinding stack; unwind-protects: Dynamic Binding; The specbinding Stack; Unwind-Protects.
2681 * dynamic scoping: The Lisp Language. (line 6)
2682 * dynamic types: The Lisp Language. (line 6)
2683 * editing operations, modules for standard: Modules for Standard Editing Operations.
2685 * Emacs 19, GNU: GNU Emacs 19. (line 6)
2686 * Emacs 20, GNU: GNU Emacs 20. (line 6)
2687 * Emacs, a history of: A History of Emacs. (line 6)
2688 * Emchar: Character-Related Data Types.
2690 * Emchars, Bufbytes and: Bufbytes and Emchars.
2692 * encoding, internal character: Internal Character Encoding.
2694 * encoding, internal string: Internal String Encoding.
2696 * encodings, internal Mule: Internal Mule Encodings.
2698 * encodings, Mule: Encodings. (line 6)
2699 * encodings, Mule character sets and: MULE Character Sets and Encodings.
2701 * Energize: Lucid Emacs. (line 6)
2702 * Epoch <1>: XEmacs. (line 6)
2703 * Epoch: Lucid Emacs. (line 6)
2704 * error checking: Techniques for XEmacs Developers.
2706 * EUC (Extended Unix Code), Japanese: Japanese EUC (Extended Unix Code).
2708 * evaluation: Evaluation. (line 6)
2709 * evaluation; stack frames; bindings: Evaluation; Stack Frames; Bindings.
2711 * event gathering mechanism, specifics of the: Specifics of the Event Gathering Mechanism.
2713 * event loop functions, other: Other Event Loop Functions.
2715 * event loop, events and the: Events and the Event Loop.
2717 * event stream callback routines, the: The Event Stream Callback Routines.
2719 * event, specifics about the Lisp object: Specifics About the Emacs Event.
2721 * events and the event loop: Events and the Event Loop.
2723 * events, converting: Converting Events. (line 6)
2724 * events, introduction to: Introduction to Events.
2726 * events, main loop: Main Loop. (line 6)
2727 * events; the command builder, dispatching: Dispatching Events; The Command Builder.
2729 * Extbyte: Character-Related Data Types.
2731 * Extcount: Character-Related Data Types.
2733 * Extended Unix Code, Japanese EUC: Japanese EUC (Extended Unix Code).
2735 * extent fragments: Extent Fragments. (line 6)
2736 * extent info, format of the: Format of the Extent Info.
2738 * extent mathematics: Mathematics of Extent Ordering.
2740 * extent ordering <1>: Mathematics of Extent Ordering.
2742 * extent ordering: Extent Ordering. (line 6)
2743 * extents: Extents. (line 6)
2744 * extents, display order: Mathematics of Extent Ordering.
2746 * extents, introduction to: Introduction to Extents.
2748 * extents, markers and: Markers and Extents. (line 6)
2749 * extents, zero-length: Zero-Length Extents. (line 6)
2750 * external data, conversion to and from: Conversion to and from External Data.
2752 * external widget: Modules for Interfacing with X Windows.
2754 * faces: Faces. (line 6)
2755 * file system, modules for interfacing with the: Modules for Interfacing with the File System.
2757 * flusher: Lstream Methods. (line 47)
2758 * fragments, extent: Extent Fragments. (line 6)
2759 * frames; windows, consoles; devices;: Consoles; Devices; Frames; Windows.
2761 * frames; windows, introduction to consoles; devices;: Introduction to Consoles; Devices; Frames; Windows.
2763 * Free Software Foundation: A History of Emacs. (line 6)
2764 * frob blocks, allocation from: Allocation from Frob Blocks.
2766 * FSF: A History of Emacs. (line 6)
2767 * FSF Emacs <1>: GNU Emacs 20. (line 6)
2768 * FSF Emacs: GNU Emacs 19. (line 6)
2769 * function, compiled: Compiled Function. (line 6)
2770 * garbage collection: Garbage Collection. (line 6)
2771 * garbage collection - step by step: Garbage Collection - Step by Step.
2773 * garbage collection protection <1>: GCPROing. (line 6)
2774 * garbage collection protection: Writing Lisp Primitives.
2776 * garbage collection, conservative: GCPROing. (line 131)
2777 * garbage collection, invocation: Invocation. (line 6)
2778 * garbage_collect_1: garbage_collect_1. (line 6)
2779 * gc_sweep: gc_sweep. (line 6)
2780 * GCPROing: GCPROing. (line 6)
2781 * global Lisp variables, adding: Adding Global Lisp Variables.
2783 * glyph instantiation: Glyphs. (line 40)
2784 * glyphs: Glyphs. (line 6)
2785 * GNU Emacs 19: GNU Emacs 19. (line 6)
2786 * GNU Emacs 20: GNU Emacs 20. (line 6)
2787 * Gosling, James <1>: The Lisp Language. (line 6)
2788 * Gosling, James: Through Version 18. (line 6)
2789 * Great Usenet Renaming: Through Version 18. (line 6)
2790 * Hackers (Steven Levy): A History of Emacs. (line 6)
2791 * header files, inline functions: Techniques for XEmacs Developers.
2793 * hierarchy of windows: Window Hierarchy. (line 6)
2794 * history of Emacs, a: A History of Emacs. (line 6)
2795 * Illinois, University of: XEmacs. (line 6)
2796 * INC_CHARPTR: Working With Character and Byte Positions.
2798 * inline functions: Techniques for XEmacs Developers.
2800 * inline functions, headers: Techniques for XEmacs Developers.
2802 * inside, XEmacs from the: XEmacs From the Inside.
2804 * instantiation, glyph: Glyphs. (line 40)
2805 * integers and characters: Integers and Characters.
2807 * interactive: Modules for Standard Editing Operations.
2809 * interfacing with the file system, modules for: Modules for Interfacing with the File System.
2811 * interfacing with the operating system, modules for: Modules for Interfacing with the Operating System.
2813 * interfacing with X Windows, modules for: Modules for Interfacing with X Windows.
2815 * internal character encoding: Internal Character Encoding.
2817 * internal Mule encodings: Internal Mule Encodings.
2819 * internal string encoding: Internal String Encoding.
2821 * internationalization, modules for: Modules for Internationalization.
2823 * interning: The XEmacs Object System (Abstractly Speaking).
2825 * interpreter and object system, modules for other aspects of the Lisp: Modules for Other Aspects of the Lisp Interpreter and Object System.
2827 * ITS (Incompatible Timesharing System): A History of Emacs. (line 6)
2828 * Japanese EUC (Extended Unix Code): Japanese EUC (Extended Unix Code).
2830 * Java: The Lisp Language. (line 6)
2831 * Java vs. Lisp: The Lisp Language. (line 6)
2832 * JIS7: JIS7. (line 6)
2833 * Jones, Kyle: XEmacs. (line 45)
2834 * Kaplan, Simon: XEmacs. (line 6)
2835 * Levy, Steven: A History of Emacs. (line 6)
2836 * library, Lucid Widget: Lucid Widget Library.
2838 * line start cache: Line Start Cache. (line 6)
2839 * Lisp interpreter and object system, modules for other aspects of the: Modules for Other Aspects of the Lisp Interpreter and Object System.
2841 * Lisp language, the: The Lisp Language. (line 6)
2842 * Lisp modules, basic: Basic Lisp Modules. (line 6)
2843 * Lisp object types, creating: Techniques for XEmacs Developers.
2845 * Lisp objects are represented in C, how: How Lisp Objects Are Represented in C.
2847 * Lisp objects, allocation of in XEmacs: Allocation of Objects in XEmacs Lisp.
2849 * Lisp objects, modules for other display-related: Modules for other Display-Related Lisp Objects.
2851 * Lisp objects, modules for the basic displayable: Modules for the Basic Displayable Lisp Objects.
2853 * Lisp primitives, writing: Writing Lisp Primitives.
2855 * Lisp reader and compiler, the: The Lisp Reader and Compiler.
2857 * Lisp vs. C: The Lisp Language. (line 6)
2858 * Lisp vs. Java: The Lisp Language. (line 6)
2859 * low-level allocation: Low-level allocation.
2861 * low-level modules: Low-Level Modules. (line 6)
2862 * lrecords: lrecords. (line 6)
2863 * lstream: Modules for Interfacing with the File System.
2865 * lstream functions: Lstream Functions. (line 6)
2866 * lstream methods: Lstream Methods. (line 6)
2867 * lstream types: Lstream Types. (line 6)
2868 * lstream, creating an: Creating an Lstream. (line 6)
2869 * Lstream_close: Lstream Functions. (line 68)
2870 * Lstream_fgetc: Lstream Functions. (line 45)
2871 * Lstream_flush: Lstream Functions. (line 19)
2872 * Lstream_fputc: Lstream Functions. (line 44)
2873 * Lstream_fungetc: Lstream Functions. (line 46)
2874 * Lstream_getc: Lstream Functions. (line 29)
2875 * Lstream_new: Lstream Functions. (line 8)
2876 * Lstream_putc: Lstream Functions. (line 23)
2877 * Lstream_read: Lstream Functions. (line 50)
2878 * Lstream_reopen: Lstream Functions. (line 71)
2879 * Lstream_rewind: Lstream Functions. (line 77)
2880 * Lstream_set_buffering: Lstream Functions. (line 15)
2881 * Lstream_ungetc: Lstream Functions. (line 34)
2882 * Lstream_unread: Lstream Functions. (line 62)
2883 * Lstream_write: Lstream Functions. (line 56)
2884 * lstreams: Lstreams. (line 6)
2885 * Lucid Emacs: Lucid Emacs. (line 6)
2886 * Lucid Inc.: Lucid Emacs. (line 6)
2887 * Lucid Widget Library: Lucid Widget Library.
2889 * macro hygiene: Techniques for XEmacs Developers.
2891 * main loop: Main Loop. (line 6)
2892 * mark and sweep: Garbage Collection. (line 6)
2893 * mark method <1>: lrecords. (line 107)
2894 * mark method: Modules for Other Aspects of the Lisp Interpreter and Object System.
2896 * mark_object: mark_object. (line 6)
2897 * marker <1>: Lstream Methods. (line 61)
2898 * marker: Marker. (line 6)
2899 * markers and extents: Markers and Extents. (line 6)
2900 * mathematics of extent ordering: Mathematics of Extent Ordering.
2902 * MAX_EMCHAR_LEN: Working With Character and Byte Positions.
2904 * menubars: Menubars. (line 6)
2905 * menus: Menus. (line 6)
2906 * merging attempts: XEmacs. (line 52)
2907 * MIT: A History of Emacs. (line 6)
2908 * Mlynarik, Richard: GNU Emacs 19. (line 68)
2909 * modules for interfacing with the file system: Modules for Interfacing with the File System.
2911 * modules for interfacing with the operating system: Modules for Interfacing with the Operating System.
2913 * modules for interfacing with X Windows: Modules for Interfacing with X Windows.
2915 * modules for internationalization: Modules for Internationalization.
2917 * modules for other aspects of the Lisp interpreter and object system: Modules for Other Aspects of the Lisp Interpreter and Object System.
2919 * modules for other display-related Lisp objects: Modules for other Display-Related Lisp Objects.
2921 * modules for regression testing: Modules for Regression Testing.
2923 * modules for standard editing operations: Modules for Standard Editing Operations.
2925 * modules for the basic displayable Lisp objects: Modules for the Basic Displayable Lisp Objects.
2927 * modules for the redisplay mechanism: Modules for the Redisplay Mechanism.
2929 * modules, a summary of the various XEmacs: A Summary of the Various XEmacs Modules.
2931 * modules, basic Lisp: Basic Lisp Modules. (line 6)
2932 * modules, editor-level control flow: Editor-Level Control Flow Modules.
2934 * modules, low-level: Low-Level Modules. (line 6)
2935 * MS-Windows environment, widget-glyphs in the: Glyphs. (line 107)
2936 * Mule character sets and encodings: MULE Character Sets and Encodings.
2938 * Mule encodings: Encodings. (line 6)
2939 * Mule encodings, internal: Internal Mule Encodings.
2941 * MULE merged XEmacs appears: XEmacs. (line 35)
2942 * Mule, coding for: Coding for Mule. (line 6)
2943 * Mule-aware code, an example of: An Example of Mule-Aware Code.
2945 * Mule-aware code, general guidelines for writing: General Guidelines for Writing Mule-Aware Code.
2947 * NAS: Modules for Interfacing with the Operating System.
2949 * native sound: Modules for Interfacing with the Operating System.
2951 * network connections: Modules for Interfacing with the Operating System.
2953 * network sound: Modules for Interfacing with the Operating System.
2955 * Niksic, Hrvoje: XEmacs. (line 45)
2956 * obarrays: Obarrays. (line 6)
2957 * object system (abstractly speaking), the XEmacs: The XEmacs Object System (Abstractly Speaking).
2959 * object system, modules for other aspects of the Lisp interpreter and: Modules for Other Aspects of the Lisp Interpreter and Object System.
2961 * object types, creating Lisp: Techniques for XEmacs Developers.
2963 * object, the buffer: The Buffer Object. (line 6)
2964 * object, the window: The Window Object. (line 6)
2965 * objects are represented in C, how Lisp: How Lisp Objects Are Represented in C.
2967 * objects in XEmacs Lisp, allocation of: Allocation of Objects in XEmacs Lisp.
2969 * objects, modules for the basic displayable Lisp: Modules for the Basic Displayable Lisp Objects.
2971 * operating system, modules for interfacing with the: Modules for Interfacing with the Operating System.
2973 * outside, XEmacs from the: XEmacs From the Outside.
2975 * pane: Modules for the Basic Displayable Lisp Objects.
2977 * permanent objects: The XEmacs Object System (Abstractly Speaking).
2979 * pi, calculating: XEmacs From the Outside.
2981 * point: Point. (line 6)
2982 * pointers dumping: Pointers dumping. (line 6)
2983 * positions, working with character and byte: Working With Character and Byte Positions.
2985 * primitives, writing Lisp: Writing Lisp Primitives.
2987 * progress bars: Progress Bars. (line 6)
2988 * protection, garbage collection: GCPROing. (line 6)
2989 * pseudo_closer: Lstream Methods. (line 51)
2990 * Purify: Techniques for XEmacs Developers.
2992 * Quantify: Techniques for XEmacs Developers.
2994 * radio buttons, checkboxes and: Checkboxes and Radio Buttons.
2996 * read syntax: The XEmacs Object System (Abstractly Speaking).
2998 * read-eval-print: XEmacs From the Outside.
3000 * reader: Lstream Methods. (line 8)
3001 * reader and compiler, the Lisp: The Lisp Reader and Compiler.
3003 * reader's guide: A Reader's Guide to XEmacs Coding Conventions.
3005 * redisplay mechanism, modules for the: Modules for the Redisplay Mechanism.
3007 * redisplay mechanism, the: The Redisplay Mechanism.
3009 * redisplay piece by piece: Redisplay Piece by Piece.
3011 * redisplay sections, critical: Critical Redisplay Sections.
3013 * regression testing, modules for: Modules for Regression Testing.
3015 * reloading phase: Reloading phase. (line 6)
3016 * relocating allocator: Low-Level Modules. (line 105)
3017 * rename to XEmacs: XEmacs. (line 21)
3018 * represented in C, how Lisp objects are: How Lisp Objects Are Represented in C.
3020 * rewinder: Lstream Methods. (line 38)
3021 * RMS: A History of Emacs. (line 6)
3022 * scanner: Modules for Other Aspects of the Lisp Interpreter and Object System.
3024 * scoping, dynamic: The Lisp Language. (line 6)
3025 * scrollbars: Scrollbars. (line 6)
3026 * seekable_p: Lstream Methods. (line 41)
3027 * selections: Modules for Interfacing with X Windows.
3029 * set_charptr_emchar: Working With Character and Byte Positions.
3031 * Sexton, Harlan: Lucid Emacs. (line 28)
3032 * sound, native: Modules for Interfacing with the Operating System.
3034 * sound, network: Modules for Interfacing with the Operating System.
3036 * SPARCWorks: XEmacs. (line 6)
3037 * specbinding stack; unwind-protects, dynamic binding; the: Dynamic Binding; The specbinding Stack; Unwind-Protects.
3039 * special forms, simple: Simple Special Forms.
3041 * specifiers: Specifiers. (line 6)
3042 * stack frames; bindings, evaluation;: Evaluation; Stack Frames; Bindings.
3044 * Stallman, Richard: A History of Emacs. (line 6)
3045 * string: String. (line 6)
3046 * string encoding, internal: Internal String Encoding.
3048 * subprocesses: Subprocesses. (line 6)
3049 * subprocesses, asynchronous: Modules for Interfacing with the Operating System.
3051 * subprocesses, synchronous: Modules for Interfacing with the Operating System.
3053 * Sun Microsystems: XEmacs. (line 6)
3054 * sweep_bit_vectors_1: sweep_bit_vectors_1. (line 6)
3055 * sweep_lcrecords_1: sweep_lcrecords_1. (line 6)
3056 * sweep_strings: sweep_strings. (line 6)
3057 * symbol: Symbol. (line 6)
3058 * symbol values: Symbol Values. (line 6)
3059 * symbols and variables: Symbols and Variables.
3061 * symbols, introduction to: Introduction to Symbols.
3063 * synchronous subprocesses: Modules for Interfacing with the Operating System.
3065 * tab controls: Tab Controls. (line 6)
3066 * taxes, doing: XEmacs From the Outside.
3068 * techniques for XEmacs developers: Techniques for XEmacs Developers.
3070 * TECO: A History of Emacs. (line 6)
3071 * temporary objects: The XEmacs Object System (Abstractly Speaking).
3073 * testing, regression: Regression Testing XEmacs.
3075 * text in a buffer, the: The Text in a Buffer.
3077 * textual representation, buffers and: Buffers and Textual Representation.
3079 * Thompson, Chuck: XEmacs. (line 6)
3080 * throw, catch and: Catch and Throw. (line 6)
3081 * types, dynamic: The Lisp Language. (line 6)
3082 * types, lstream: Lstream Types. (line 6)
3083 * types, proper use of unsigned: Proper Use of Unsigned Types.
3085 * University of Illinois: XEmacs. (line 6)
3086 * unsigned types, proper use of: Proper Use of Unsigned Types.
3088 * unwind-protects, dynamic binding; the specbinding stack;: Dynamic Binding; The specbinding Stack; Unwind-Protects.
3090 * values, symbol: Symbol Values. (line 6)
3091 * variables, adding global Lisp: Adding Global Lisp Variables.
3093 * variables, symbols and: Symbols and Variables.
3095 * vector: Vector. (line 6)
3096 * vector, bit: Bit Vector. (line 6)
3097 * version 18, through: Through Version 18. (line 6)
3098 * version 19, GNU Emacs: GNU Emacs 19. (line 6)
3099 * version 20, GNU Emacs: GNU Emacs 20. (line 6)
3100 * widget interface, generic: Generic Widget Interface.
3102 * widget library, Lucid: Lucid Widget Library.
3104 * widget-glyphs: Glyphs. (line 104)
3105 * widget-glyphs in the MS-Windows environment: Glyphs. (line 107)
3106 * widget-glyphs in the X environment: Glyphs. (line 112)
3107 * Win-Emacs: XEmacs. (line 6)
3108 * window (in Emacs): Modules for the Basic Displayable Lisp Objects.
3110 * window hierarchy: Window Hierarchy. (line 6)
3111 * window object, the: The Window Object. (line 6)
3112 * window point internals: The Window Object. (line 22)
3113 * windows, consoles; devices; frames;: Consoles; Devices; Frames; Windows.
3115 * windows, introduction to consoles; devices; frames;: Introduction to Consoles; Devices; Frames; Windows.
3117 * Wing, Ben: XEmacs. (line 6)
3118 * writer: Lstream Methods. (line 26)
3119 * writing good comments: Writing Good Comments.
3121 * writing Lisp primitives: Writing Lisp Primitives.
3123 * writing Mule-aware code, general guidelines for: General Guidelines for Writing Mule-Aware Code.
3125 * writing new C code, rules when: Rules When Writing New C Code.
3127 * X environment, widget-glyphs in the: Glyphs. (line 112)
3128 * X Window System, interface to the: Interface to the X Window System.
3130 * X Windows, modules for interfacing with: Modules for Interfacing with X Windows.
3132 * XEmacs: XEmacs. (line 6)
3133 * XEmacs from the inside: XEmacs From the Inside.
3135 * XEmacs from the outside: XEmacs From the Outside.
3137 * XEmacs from the perspective of building: XEmacs From the Perspective of Building.
3139 * XEmacs goes it alone: XEmacs. (line 45)
3140 * XEmacs object system (abstractly speaking), the: The XEmacs Object System (Abstractly Speaking).
3142 * Zawinski, Jamie: Lucid Emacs. (line 28)
3143 * zero-length extents: Zero-Length Extents. (line 6)