1 This is ../info/internals.info, produced by makeinfo version 4.0 from
2 internals/internals.texi.
4 INFO-DIR-SECTION XEmacs Editor
6 * Internals: (internals). XEmacs Internals Manual.
9 Copyright (C) 1992 - 1996 Ben Wing. Copyright (C) 1996, 1997 Sun
10 Microsystems. Copyright (C) 1994 - 1998 Free Software Foundation.
11 Copyright (C) 1994, 1995 Board of Trustees, University of Illinois.
13 Permission is granted to make and distribute verbatim copies of this
14 manual provided the copyright notice and this permission notice are
15 preserved on all copies.
17 Permission is granted to copy and distribute modified versions of
18 this manual under the conditions for verbatim copying, provided that the
19 entire resulting derived work is distributed under the terms of a
20 permission notice identical to this one.
22 Permission is granted to copy and distribute translations of this
23 manual into another language, under the above conditions for modified
24 versions, except that this permission notice may be stated in a
25 translation approved by the Foundation.
27 Permission is granted to copy and distribute modified versions of
28 this manual under the conditions for verbatim copying, provided also
29 that the section entitled "GNU General Public License" is included
30 exactly as in the original, and provided that the entire resulting
31 derived work is distributed under the terms of a permission notice
32 identical to this one.
34 Permission is granted to copy and distribute translations of this
35 manual into another language, under the above conditions for modified
36 versions, except that the section entitled "GNU General Public License"
37 may be included in a translation approved by the Free Software
38 Foundation instead of in the original English.
41 File: internals.info, Node: Top, Next: A History of Emacs, Prev: (dir), Up: (dir)
43 This Info file contains v1.4 of the XEmacs Internals Manual, March
48 * A History of Emacs:: Times, dates, important events.
49 * XEmacs From the Outside:: A broad conceptual overview.
50 * The Lisp Language:: An overview.
51 * XEmacs From the Perspective of Building::
52 * XEmacs From the Inside::
53 * The XEmacs Object System (Abstractly Speaking)::
54 * How Lisp Objects Are Represented in C::
55 * Rules When Writing New C Code::
56 * A Summary of the Various XEmacs Modules::
57 * Allocation of Objects in XEmacs Lisp::
59 * Events and the Event Loop::
60 * Evaluation; Stack Frames; Bindings::
61 * Symbols and Variables::
62 * Buffers and Textual Representation::
63 * MULE Character Sets and Encodings::
64 * The Lisp Reader and Compiler::
66 * Consoles; Devices; Frames; Windows::
67 * The Redisplay Mechanism::
74 * Interface to the X Window System::
78 --- The Detailed Node Listing ---
82 * Through Version 18:: Unification prevails.
83 * Lucid Emacs:: One version 19 Emacs.
84 * GNU Emacs 19:: The other version 19 Emacs.
85 * GNU Emacs 20:: The other version 20 Emacs.
86 * XEmacs:: The continuation of Lucid Emacs.
88 Rules When Writing New C Code
90 * General Coding Rules::
91 * Writing Lisp Primitives::
92 * Adding Global Lisp Variables::
94 * Techniques for XEmacs Developers::
98 * Character-Related Data Types::
99 * Working With Character and Byte Positions::
100 * Conversion to and from External Data::
101 * General Guidelines for Writing Mule-Aware Code::
102 * An Example of Mule-Aware Code::
104 A Summary of the Various XEmacs Modules
106 * Low-Level Modules::
107 * Basic Lisp Modules::
108 * Modules for Standard Editing Operations::
109 * Editor-Level Control Flow Modules::
110 * Modules for the Basic Displayable Lisp Objects::
111 * Modules for other Display-Related Lisp Objects::
112 * Modules for the Redisplay Mechanism::
113 * Modules for Interfacing with the File System::
114 * Modules for Other Aspects of the Lisp Interpreter and Object System::
115 * Modules for Interfacing with the Operating System::
116 * Modules for Interfacing with X Windows::
117 * Modules for Internationalization::
119 Allocation of Objects in XEmacs Lisp
121 * Introduction to Allocation::
122 * Garbage Collection::
124 * Garbage Collection - Step by Step::
125 * Integers and Characters::
126 * Allocation from Frob Blocks::
128 * Low-level allocation::
135 * Compiled Function::
137 Garbage Collection - Step by Step
140 * garbage_collect_1::
143 * sweep_lcrecords_1::
144 * compact_string_chars::
146 * sweep_bit_vectors_1::
151 * Data descriptions::
158 * Address allocation::
163 Events and the Event Loop
165 * Introduction to Events::
167 * Specifics of the Event Gathering Mechanism::
168 * Specifics About the Emacs Event::
169 * The Event Stream Callback Routines::
170 * Other Event Loop Functions::
171 * Converting Events::
172 * Dispatching Events; The Command Builder::
174 Evaluation; Stack Frames; Bindings
177 * Dynamic Binding; The specbinding Stack; Unwind-Protects::
178 * Simple Special Forms::
181 Symbols and Variables
183 * Introduction to Symbols::
187 Buffers and Textual Representation
189 * Introduction to Buffers:: A buffer holds a block of text such as a file.
190 * The Text in a Buffer:: Representation of the text in a buffer.
191 * Buffer Lists:: Keeping track of all buffers.
192 * Markers and Extents:: Tagging locations within a buffer.
193 * Bufbytes and Emchars:: Representation of individual characters.
194 * The Buffer Object:: The Lisp object corresponding to a buffer.
196 MULE Character Sets and Encodings
200 * Internal Mule Encodings::
205 * Japanese EUC (Extended Unix Code)::
208 Internal Mule Encodings
210 * Internal String Encoding::
211 * Internal Character Encoding::
215 * Creating an Lstream:: Creating an lstream object.
216 * Lstream Types:: Different sorts of things that are streamed.
217 * Lstream Functions:: Functions for working with lstreams.
218 * Lstream Methods:: Creating new lstream types.
220 Consoles; Devices; Frames; Windows
222 * Introduction to Consoles; Devices; Frames; Windows::
225 * The Window Object::
227 The Redisplay Mechanism
229 * Critical Redisplay Sections::
231 * Redisplay Piece by Piece::
235 * Introduction to Extents:: Extents are ranges over text, with properties.
236 * Extent Ordering:: How extents are ordered internally.
237 * Format of the Extent Info:: The extent information in a buffer or string.
238 * Zero-Length Extents:: A weird special case.
239 * Mathematics of Extent Ordering:: A rigorous foundation.
240 * Extent Fragments:: Cached information useful for redisplay.
243 File: internals.info, Node: A History of Emacs, Next: XEmacs From the Outside, Prev: Top, Up: Top
248 XEmacs is a powerful, customizable text editor and development
249 environment. It began as Lucid Emacs, which was in turn derived from
250 GNU Emacs, a program written by Richard Stallman of the Free Software
251 Foundation. GNU Emacs dates back to the 1970's, and was modelled after
252 a package called "Emacs", written in 1976, that was a set of macros on
253 top of TECO, an old, old text editor written at MIT on the DEC PDP 10
254 under one of the earliest time-sharing operating systems, ITS
255 (Incompatible Timesharing System). (ITS dates back well before Unix.)
256 ITS, TECO, and Emacs were products of a group of people at MIT who
257 called themselves "hackers", who shared an idealistic belief system
258 about the free exchange of information and were fanatical in their
259 devotion to and time spent with computers. (The hacker subculture dates
260 back to the late 1950's at MIT and is described in detail in Steven
261 Levy's book `Hackers'. This book also includes a lot of information
262 about Stallman himself and the development of Lisp, a programming
263 language developed at MIT that underlies Emacs.)
267 * Through Version 18:: Unification prevails.
268 * Lucid Emacs:: One version 19 Emacs.
269 * GNU Emacs 19:: The other version 19 Emacs.
270 * GNU Emacs 20:: The other version 20 Emacs.
271 * XEmacs:: The continuation of Lucid Emacs.
274 File: internals.info, Node: Through Version 18, Next: Lucid Emacs, Up: A History of Emacs
279 Although the history of the early versions of GNU Emacs is unclear,
280 the history is well-known from the middle of 1985. A time line is:
282 * GNU Emacs version 15 (15.34) was released sometime in 1984 or 1985
283 and shared some code with a version of Emacs written by James
284 Gosling (the same James Gosling who later created the Java
287 * GNU Emacs version 16 (first released version was 16.56) was
288 released on July 15, 1985. All Gosling code was removed due to
289 potential copyright problems with the code.
291 * version 16.57: released on September 16, 1985.
293 * versions 16.58, 16.59: released on September 17, 1985.
295 * version 16.60: released on September 19, 1985. These later
296 version 16's incorporated patches from the net, esp. for getting
297 Emacs to work under System V.
299 * version 17.36 (first official v17 release) released on December 20,
300 1985. Included a TeX-able user manual. First official unpatched
301 version that worked on vanilla System V machines.
303 * version 17.43 (second official v17 release) released on January 25,
306 * version 17.45 released on January 30, 1986.
308 * version 17.46 released on February 4, 1986.
310 * version 17.48 released on February 10, 1986.
312 * version 17.49 released on February 12, 1986.
314 * version 17.55 released on March 18, 1986.
316 * version 17.57 released on March 27, 1986.
318 * version 17.58 released on April 4, 1986.
320 * version 17.61 released on April 12, 1986.
322 * version 17.63 released on May 7, 1986.
324 * version 17.64 released on May 12, 1986.
326 * version 18.24 (a beta version) released on October 2, 1986.
328 * version 18.30 (a beta version) released on November 15, 1986.
330 * version 18.31 (a beta version) released on November 23, 1986.
332 * version 18.32 (a beta version) released on December 7, 1986.
334 * version 18.33 (a beta version) released on December 12, 1986.
336 * version 18.35 (a beta version) released on January 5, 1987.
338 * version 18.36 (a beta version) released on January 21, 1987.
340 * January 27, 1987: The Great Usenet Renaming. net.emacs is now
343 * version 18.37 (a beta version) released on February 12, 1987.
345 * version 18.38 (a beta version) released on March 3, 1987.
347 * version 18.39 (a beta version) released on March 14, 1987.
349 * version 18.40 (a beta version) released on March 18, 1987.
351 * version 18.41 (the first "official" release) released on March 22,
354 * version 18.45 released on June 2, 1987.
356 * version 18.46 released on June 9, 1987.
358 * version 18.47 released on June 18, 1987.
360 * version 18.48 released on September 3, 1987.
362 * version 18.49 released on September 18, 1987.
364 * version 18.50 released on February 13, 1988.
366 * version 18.51 released on May 7, 1988.
368 * version 18.52 released on September 1, 1988.
370 * version 18.53 released on February 24, 1989.
372 * version 18.54 released on April 26, 1989.
374 * version 18.55 released on August 23, 1989. This is the earliest
375 version that is still available by FTP.
377 * version 18.56 released on January 17, 1991.
379 * version 18.57 released late January, 1991.
381 * version 18.58 released ?????.
383 * version 18.59 released October 31, 1992.
386 File: internals.info, Node: Lucid Emacs, Next: GNU Emacs 19, Prev: Through Version 18, Up: A History of Emacs
391 Lucid Emacs was developed by the (now-defunct) Lucid Inc., a maker of
392 C++ and Lisp development environments. It began when Lucid decided they
393 wanted to use Emacs as the editor and cornerstone of their C++
394 development environment (called "Energize"). They needed many features
395 that were not available in the existing version of GNU Emacs (version
396 18.5something), in particular good and integrated support for GUI
397 elements such as mouse support, multiple fonts, multiple window-system
398 windows, etc. A branch of GNU Emacs called Epoch, written at the
399 University of Illinois, existed that supplied many of these features;
400 however, Lucid needed more than what existed in Epoch. At the time, the
401 Free Software Foundation was working on version 19 of Emacs (this was
402 sometime around 1991), which was planned to have similar features, and
403 so Lucid decided to work with the Free Software Foundation. Their plan
404 was to add features that they needed, and coordinate with the FSF so
405 that the features would get included back into Emacs version 19.
407 Delays in the release of version 19 occurred, however (resulting in
408 it finally being released more than a year after what was initially
409 planned), and Lucid encountered unexpected technical resistance in
410 getting their changes merged back into version 19, so they decided to
411 release their own version of Emacs, which became Lucid Emacs 19.0.
413 The initial authors of Lucid Emacs were Matthieu Devin, Harlan
414 Sexton, and Eric Benson, and the work was later taken over by Jamie
415 Zawinski, who became "Mr. Lucid Emacs" for many releases.
417 A time line for Lucid Emacs is
419 * version 19.0 shipped with Energize 1.0, April 1992.
421 * version 19.1 released June 4, 1992.
423 * version 19.2 released June 19, 1992.
425 * version 19.3 released September 9, 1992.
427 * version 19.4 released January 21, 1993.
429 * version 19.5 was a repackaging of 19.4 with a few bug fixes and
430 shipped with Energize 2.0. Never released to the net.
432 * version 19.6 released April 9, 1993.
434 * version 19.7 was a repackaging of 19.6 with a few bug fixes and
435 shipped with Energize 2.1. Never released to the net.
437 * version 19.8 released September 6, 1993.
439 * version 19.9 released January 12, 1994.
441 * version 19.10 released May 27, 1994.
443 * version 19.11 (first XEmacs) released September 13, 1994.
445 * version 19.12 released June 23, 1995.
447 * version 19.13 released September 1, 1995.
449 * version 19.14 released June 23, 1996.
451 * version 20.0 released February 9, 1997.
453 * version 19.15 released March 28, 1997.
455 * version 20.1 (not released to the net) April 15, 1997.
457 * version 20.2 released May 16, 1997.
459 * version 19.16 released October 31, 1997.
461 * version 20.3 (the first stable version of XEmacs 20.x) released
464 * version 20.4 released February 28, 1998.
466 * version 21.1.2 released May 14, 1999. (The version naming scheme
467 was changed at this point: [a] the second version number is odd
468 for stable versions, even for beta versions; [b] a third version
469 number is added, replacing the "beta xxx" ending for beta versions
470 and allowing for periodic maintenance releases for stable
471 versions. Therefore, 21.0 was never "officially" released;
472 similarly for 21.2, etc.)
474 * version 21.1.3 released June 26, 1999.
476 * version 21.1.4 released July 8, 1999.
478 * version 21.1.6 released August 14, 1999. (There was no 21.1.5.)
480 * version 21.1.7 released September 26, 1999.
482 * version 21.1.8 released November 2, 1999.
484 * version 21.1.9 released February 13, 2000.
486 * version 21.1.10 released May 7, 2000.
488 * version 21.1.10a released June 24, 2000.
490 * version 21.1.11 released July 18, 2000.
492 * version 21.1.12 released August 5, 2000.
494 * version 21.1.13 released January 7, 2001.
496 * version 21.1.14 released January 27, 2001.
499 File: internals.info, Node: GNU Emacs 19, Next: GNU Emacs 20, Prev: Lucid Emacs, Up: A History of Emacs
504 About a year after the initial release of Lucid Emacs, the FSF
505 released a beta of their version of Emacs 19 (referred to here as "GNU
506 Emacs"). By this time, the current version of Lucid Emacs was 19.6.
507 (Strangely, the first released beta from the FSF was GNU Emacs 19.7.) A
508 time line for GNU Emacs version 19 is
510 * version 19.8 (beta) released May 27, 1993.
512 * version 19.9 (beta) released May 27, 1993.
514 * version 19.10 (beta) released May 30, 1993.
516 * version 19.11 (beta) released June 1, 1993.
518 * version 19.12 (beta) released June 2, 1993.
520 * version 19.13 (beta) released June 8, 1993.
522 * version 19.14 (beta) released June 17, 1993.
524 * version 19.15 (beta) released June 19, 1993.
526 * version 19.16 (beta) released July 6, 1993.
528 * version 19.17 (beta) released late July, 1993.
530 * version 19.18 (beta) released August 9, 1993.
532 * version 19.19 (beta) released August 15, 1993.
534 * version 19.20 (beta) released November 17, 1993.
536 * version 19.21 (beta) released November 17, 1993.
538 * version 19.22 (beta) released November 28, 1993.
540 * version 19.23 (beta) released May 17, 1994.
542 * version 19.24 (beta) released May 16, 1994.
544 * version 19.25 (beta) released June 3, 1994.
546 * version 19.26 (beta) released September 11, 1994.
548 * version 19.27 (beta) released September 14, 1994.
550 * version 19.28 (first "official" release) released November 1, 1994.
552 * version 19.29 released June 21, 1995.
554 * version 19.30 released November 24, 1995.
556 * version 19.31 released May 25, 1996.
558 * version 19.32 released July 31, 1996.
560 * version 19.33 released August 11, 1996.
562 * version 19.34 released August 21, 1996.
564 * version 19.34b released September 6, 1996.
566 In some ways, GNU Emacs 19 was better than Lucid Emacs; in some ways,
567 worse. Lucid soon began incorporating features from GNU Emacs 19 into
568 Lucid Emacs; the work was mostly done by Richard Mlynarik, who had been
569 working on and using GNU Emacs for a long time (back as far as version
573 File: internals.info, Node: GNU Emacs 20, Next: XEmacs, Prev: GNU Emacs 19, Up: A History of Emacs
578 On February 2, 1997 work began on GNU Emacs to integrate Mule. The
579 first release was made in September of that year.
581 A timeline for Emacs 20 is
583 * version 20.1 released September 17, 1997.
585 * version 20.2 released September 20, 1997.
587 * version 20.3 released August 19, 1998.
590 File: internals.info, Node: XEmacs, Prev: GNU Emacs 20, Up: A History of Emacs
595 Around the time that Lucid was developing Energize, Sun Microsystems
596 was developing their own development environment (called "SPARCWorks")
597 and also decided to use Emacs. They joined forces with the Epoch team
598 at the University of Illinois and later with Lucid. The maintainer of
599 the last-released version of Epoch was Marc Andreessen, but he dropped
600 out and the Epoch project, headed by Simon Kaplan, lured Chuck Thompson
601 away from a system administration job to become the primary Lucid Emacs
602 author for Epoch and Sun. Chuck's area of specialty became the
603 redisplay engine (he replaced the old Lucid Emacs redisplay engine with
604 a ported version from Epoch and then later rewrote it from scratch).
605 Sun also hired Ben Wing (the author of Win-Emacs, a port of Lucid Emacs
606 to Microsoft Windows 3.1) in 1993, for what was initially a one-month
607 contract to fix some event problems but later became a many-year
608 involvement, punctuated by a six-month contract with Amdahl Corporation.
610 In 1994, Sun and Lucid agreed to rename Lucid Emacs to XEmacs (a name
611 not favorable to either company); the first release called XEmacs was
612 version 19.11. In June 1994, Lucid folded and Jamie quit to work for
613 the newly formed Mosaic Communications Corp., later Netscape
614 Communications Corp. (co-founded by the same Marc Andreessen, who had
615 quit his Epoch job to work on a graphical browser for the World Wide
616 Web). Chuck then become the primary maintainer of XEmacs, and put out
617 versions 19.11 through 19.14 in conjunction with Ben. For 19.12 and
618 19.13, Chuck added the new redisplay and many other display improvements
619 and Ben added MULE support (support for Asian and other languages) and
620 redesigned most of the internal Lisp subsystems to better support the
621 MULE work and the various other features being added to XEmacs. After
622 19.14 Chuck retired as primary maintainer and Steve Baur stepped in.
624 Soon after 19.13 was released, work began in earnest on the MULE
625 internationalization code and the source tree was divided into two
626 development paths. The MULE version was initially called 19.20, but was
627 soon renamed to 20.0. In 1996 Martin Buchholz of Sun Microsystems took
628 over the care and feeding of it and worked on it in parallel with the
629 19.14 development that was occurring at the same time. After much work
630 by Martin, it was decided to release 20.0 ahead of 19.15 in February
631 1997. The source tree remained divided until 20.2 when the version 19
632 source was finally retired at version 19.16.
634 In 1997, Sun finally dropped all pretense of support for XEmacs and
635 Martin Buchholz left the company in November. Since then, and mostly
636 for the previous year, because Steve Baur was never paid to work on
637 XEmacs, XEmacs has existed solely on the contributions of volunteers
638 from the Free Software Community. Starting from 1997, Hrvoje Niksic and
639 Kyle Jones have figured prominently in XEmacs development.
641 Many attempts have been made to merge XEmacs and GNU Emacs, but they
642 have consistently failed.
644 A more detailed history is contained in the XEmacs About page.
646 A time line for XEmacs is
648 * version 19.11 (first XEmacs) released September 13, 1994.
650 * version 19.12 released June 23, 1995.
652 * version 19.13 released September 1, 1995.
654 * version 19.14 released June 23, 1996.
656 * version 20.0 released February 9, 1997.
658 * version 19.15 released March 28, 1997.
660 * version 20.1 (not released to the net) April 15, 1997.
662 * version 20.2 released May 16, 1997.
664 * version 19.16 released October 31, 1997.
666 * version 20.3 (the first stable version of XEmacs 20.x) released
669 * version 20.4 released February 28, 1998.
671 * version 21.0.60 released December 10, 1998. (The version naming
672 scheme was changed at this point: [a] the second version number is
673 odd for stable versions, even for beta versions; [b] a third
674 version number is added, replacing the "beta xxx" ending for beta
675 versions and allowing for periodic maintenance releases for stable
676 versions. Therefore, 21.0 was never "officially" released;
677 similarly for 21.2, etc.)
679 * version 21.0.61 released January 4, 1999.
681 * version 21.0.63 released February 3, 1999.
683 * version 21.0.64 released March 1, 1999.
685 * version 21.0.65 released March 5, 1999.
687 * version 21.0.66 released March 12, 1999.
689 * version 21.0.67 released March 25, 1999.
691 * version 21.1.2 released May 14, 1999. (This is the followup to
692 21.0.67. The second version number was bumped to indicate the
693 beginning of the "stable" series.)
695 * version 21.1.3 released June 26, 1999.
697 * version 21.1.4 released July 8, 1999.
699 * version 21.1.6 released August 14, 1999. (There was no 21.1.5.)
701 * version 21.1.7 released September 26, 1999.
703 * version 21.1.8 released November 2, 1999.
705 * version 21.1.9 released February 13, 2000.
707 * version 21.1.10 released May 7, 2000.
709 * version 21.1.10a released June 24, 2000.
711 * version 21.1.11 released July 18, 2000.
713 * version 21.1.12 released August 5, 2000.
715 * version 21.1.13 released January 7, 2001.
717 * version 21.1.14 released January 27, 2001.
719 * version 21.2.9 released February 3, 1999.
721 * version 21.2.10 released February 5, 1999.
723 * version 21.2.11 released March 1, 1999.
725 * version 21.2.12 released March 5, 1999.
727 * version 21.2.13 released March 12, 1999.
729 * version 21.2.14 released May 14, 1999.
731 * version 21.2.15 released June 4, 1999.
733 * version 21.2.16 released June 11, 1999.
735 * version 21.2.17 released June 22, 1999.
737 * version 21.2.18 released July 14, 1999.
739 * version 21.2.19 released July 30, 1999.
741 * version 21.2.20 released November 10, 1999.
743 * version 21.2.21 released November 28, 1999.
745 * version 21.2.22 released November 29, 1999.
747 * version 21.2.23 released December 7, 1999.
749 * version 21.2.24 released December 14, 1999.
751 * version 21.2.25 released December 24, 1999.
753 * version 21.2.26 released December 31, 1999.
755 * version 21.2.27 released January 18, 2000.
757 * version 21.2.28 released February 7, 2000.
759 * version 21.2.29 released February 16, 2000.
761 * version 21.2.30 released February 21, 2000.
763 * version 21.2.31 released February 23, 2000.
765 * version 21.2.32 released March 20, 2000.
767 * version 21.2.33 released May 1, 2000.
769 * version 21.2.34 released May 28, 2000.
771 * version 21.2.35 released July 19, 2000.
773 * version 21.2.36 released October 4, 2000.
775 * version 21.2.37 released November 14, 2000.
777 * version 21.2.38 released December 5, 2000.
779 * version 21.2.39 released December 31, 2000.
781 * version 21.2.40 released January 8, 2001.
783 * version 21.2.41 released January 17, 2001.
785 * version 21.2.42 released January 20, 2001.
787 * version 21.2.43 released January 26, 2001.
789 * version 21.2.44 released February 8, 2001.
791 * version 21.2.45 released February 23, 2001.
793 * version 21.2.46 released March 21, 2001.
796 File: internals.info, Node: XEmacs From the Outside, Next: The Lisp Language, Prev: A History of Emacs, Up: Top
798 XEmacs From the Outside
799 ***********************
801 XEmacs appears to the outside world as an editor, but it is really a
802 Lisp environment. At its heart is a Lisp interpreter; it also
803 "happens" to contain many specialized object types (e.g. buffers,
804 windows, frames, events) that are useful for implementing an editor.
805 Some of these objects (in particular windows and frames) have
806 displayable representations, and XEmacs provides a function
807 `redisplay()' that ensures that the display of all such objects matches
808 their internal state. Most of the time, a standard Lisp environment is
809 in a "read-eval-print" loop--i.e. "read some Lisp code, execute it, and
810 print the results". XEmacs has a similar loop:
814 * dispatch the event (i.e. "do it")
818 Reading an event is done using the Lisp function `next-event', which
819 waits for something to happen (typically, the user presses a key or
820 moves the mouse) and returns an event object describing this.
821 Dispatching an event is done using the Lisp function `dispatch-event',
822 which looks up the event in a keymap object (a particular kind of
823 object that associates an event with a Lisp function) and calls that
824 function. The function "does" what the user has requested by changing
825 the state of particular frame objects, buffer objects, etc. Finally,
826 `redisplay()' is called, which updates the display to reflect those
827 changes just made. Thus is an "editor" born.
829 Note that you do not have to use XEmacs as an editor; you could just
830 as well make it do your taxes, compute pi, play bridge, etc. You'd just
831 have to write functions to do those operations in Lisp.
834 File: internals.info, Node: The Lisp Language, Next: XEmacs From the Perspective of Building, Prev: XEmacs From the Outside, Up: Top
839 Lisp is a general-purpose language that is higher-level than C and in
840 many ways more powerful than C. Powerful dialects of Lisp such as
841 Common Lisp are probably much better languages for writing very large
842 applications than is C. (Unfortunately, for many non-technical reasons
843 C and its successor C++ have become the dominant languages for
844 application development. These languages are both inadequate for
845 extremely large applications, which is evidenced by the fact that newer,
846 larger programs are becoming ever harder to write and are requiring ever
847 more programmers despite great increases in C development environments;
848 and by the fact that, although hardware speeds and reliability have been
849 growing at an exponential rate, most software is still generally
850 considered to be slow and buggy.)
852 The new Java language holds promise as a better general-purpose
853 development language than C. Java has many features in common with
854 Lisp that are not shared by C (this is not a coincidence, since Java
855 was designed by James Gosling, a former Lisp hacker). This will be
856 discussed more later.
858 For those used to C, here is a summary of the basic differences
861 1. Lisp has an extremely regular syntax. Every function, expression,
862 and control statement is written in the form
866 This is as opposed to C, which writes functions as
868 func(ARG1, ARG2, ...)
870 but writes expressions involving operators as (e.g.)
874 and writes control statements as (e.g.)
876 while (EXPR) { STATEMENT1; STATEMENT2; ... }
878 Lisp equivalents of the latter two would be
884 (while EXPR STATEMENT1 STATEMENT2 ...)
886 2. Lisp is a safe language. Assuming there are no bugs in the Lisp
887 interpreter/compiler, it is impossible to write a program that
888 "core dumps" or otherwise causes the machine to execute an illegal
889 instruction. This is very different from C, where perhaps the most
890 common outcome of a bug is exactly such a crash. A corollary of
891 this is that the C operation of casting a pointer is impossible
892 (and unnecessary) in Lisp, and that it is impossible to access
893 memory outside the bounds of an array.
895 3. Programs and data are written in the same form. The
896 parenthesis-enclosing form described above for statements is the
897 same form used for the most common data type in Lisp, the list.
898 Thus, it is possible to represent any Lisp program using Lisp data
899 types, and for one program to construct Lisp statements and then
900 dynamically "evaluate" them, or cause them to execute.
902 4. All objects are "dynamically typed". This means that part of every
903 object is an indication of what type it is. A Lisp program can
904 manipulate an object without knowing what type it is, and can
905 query an object to determine its type. This means that,
906 correspondingly, variables and function parameters can hold
907 objects of any type and are not normally declared as being of any
908 particular type. This is opposed to the "static typing" of C,
909 where variables can hold exactly one type of object and must be
910 declared as such, and objects do not contain an indication of
911 their type because it's implicit in the variables they are stored
912 in. It is possible in C to have a variable hold different types
913 of objects (e.g. through the use of `void *' pointers or
914 variable-argument functions), but the type information must then be
915 passed explicitly in some other fashion, leading to additional
918 5. Allocated memory is automatically reclaimed when it is no longer
919 in use. This operation is called "garbage collection" and
920 involves looking through all variables to see what memory is being
921 pointed to, and reclaiming any memory that is not pointed to and
922 is thus "inaccessible" and out of use. This is as opposed to C,
923 in which allocated memory must be explicitly reclaimed using
924 `free()'. If you simply drop all pointers to memory without
925 freeing it, it becomes "leaked" memory that still takes up space.
926 Over a long period of time, this can cause your program to grow
927 and grow until it runs out of memory.
929 6. Lisp has built-in facilities for handling errors and exceptions.
930 In C, when an error occurs, usually either the program exits
931 entirely or the routine in which the error occurs returns a value
932 indicating this. If an error occurs in a deeply-nested routine,
933 then every routine currently called must unwind itself normally
934 and return an error value back up to the next routine. This means
935 that every routine must explicitly check for an error in all the
936 routines it calls; if it does not do so, unexpected and often
937 random behavior results. This is an extremely common source of
938 bugs in C programs. An alternative would be to do a non-local
939 exit using `longjmp()', but that is often very dangerous because
940 the routines that were exited past had no opportunity to clean up
941 after themselves and may leave things in an inconsistent state,
942 causing a crash shortly afterwards.
944 Lisp provides mechanisms to make such non-local exits safe. When
945 an error occurs, a routine simply signals that an error of a
946 particular class has occurred, and a non-local exit takes place.
947 Any routine can trap errors occurring in routines it calls by
948 registering an error handler for some or all classes of errors.
949 (If no handler is registered, a default handler, generally
950 installed by the top-level event loop, is executed; this prints
951 out the error and continues.) Routines can also specify cleanup
952 code (called an "unwind-protect") that will be called when control
953 exits from a block of code, no matter how that exit occurs--i.e.
954 even if a function deeply nested below it causes a non-local exit
955 back to the top level.
957 Note that this facility has appeared in some recent vintages of C,
958 in particular Visual C++ and other PC compilers written for the
961 7. In Emacs Lisp, local variables are "dynamically scoped". This
962 means that if you declare a local variable in a particular
963 function, and then call another function, that subfunction can
964 "see" the local variable you declared. This is actually
965 considered a bug in Emacs Lisp and in all other early dialects of
966 Lisp, and was corrected in Common Lisp. (In Common Lisp, you can
967 still declare dynamically scoped variables if you want to--they
968 are sometimes useful--but variables by default are "lexically
971 For those familiar with Lisp, Emacs Lisp is modelled after MacLisp,
972 an early dialect of Lisp developed at MIT (no relation to the Macintosh
973 computer). There is a Common Lisp compatibility package available for
974 Emacs that provides many of the features of Common Lisp.
976 The Java language is derived in many ways from C, and shares a
977 similar syntax, but has the following features in common with Lisp (and
980 1. Java is a safe language, like Lisp.
982 2. Java provides garbage collection, like Lisp.
984 3. Java has built-in facilities for handling errors and exceptions,
987 4. Java has a type system that combines the best advantages of both
988 static and dynamic typing. Objects (except very simple types) are
989 explicitly marked with their type, as in dynamic typing; but there
990 is a hierarchy of types and functions are declared to accept only
991 certain types, thus providing the increased compile-time
992 error-checking of static typing.
994 The Java language also has some negative attributes:
996 1. Java uses the edit/compile/run model of software development. This
997 makes it hard to use interactively. For example, to use Java like
998 `bc' it is necessary to write a special purpose, albeit tiny,
999 application. In Emacs Lisp, a calculator comes built-in without
1000 any effort - one can always just type an expression in the
1003 2. Java tries too hard to enforce, not merely enable, portability,
1004 making ordinary access to standard OS facilities painful. Java
1005 has an "agenda". I think this is why `chdir' is not part of
1006 standard Java, which is inexcusable.
1008 Unfortunately, there is no perfect language. Static typing allows a
1009 compiler to catch programmer errors and produce more efficient code, but
1010 makes programming more tedious and less fun. For the foreseeable
1011 future, an Ideal Editing and Programming Environment (and that is what
1012 XEmacs aspires to) will be programmable in multiple languages: high
1013 level ones like Lisp for user customization and prototyping, and lower
1014 level ones for infrastructure and industrial strength applications. If
1015 I had my way, XEmacs would be friendly towards the Python, Scheme, C++,
1016 ML, etc... communities. But there are serious technical difficulties to
1017 achieving that goal.
1019 The word "application" in the previous paragraph was used
1020 intentionally. XEmacs implements an API for programs written in Lisp
1021 that makes it a full-fledged application platform, very much like an OS
1025 File: internals.info, Node: XEmacs From the Perspective of Building, Next: XEmacs From the Inside, Prev: The Lisp Language, Up: Top
1027 XEmacs From the Perspective of Building
1028 ***************************************
1030 The heart of XEmacs is the Lisp environment, which is written in C.
1031 This is contained in the `src/' subdirectory. Underneath `src/' are
1032 two subdirectories of header files: `s/' (header files for particular
1033 operating systems) and `m/' (header files for particular machine
1034 types). In practice the distinction between the two types of header
1035 files is blurred. These header files define or undefine certain
1036 preprocessor constants and macros to indicate particular
1037 characteristics of the associated machine or operating system. As part
1038 of the configure process, one `s/' file and one `m/' file is identified
1039 for the particular environment in which XEmacs is being built.
1041 XEmacs also contains a great deal of Lisp code. This implements the
1042 operations that make XEmacs useful as an editor as well as just a Lisp
1043 environment, and also contains many add-on packages that allow XEmacs to
1044 browse directories, act as a mail and Usenet news reader, compile Lisp
1045 code, etc. There is actually more Lisp code than C code associated with
1046 XEmacs, but much of the Lisp code is peripheral to the actual operation
1047 of the editor. The Lisp code all lies in subdirectories underneath the
1050 The `lwlib/' directory contains C code that implements a generalized
1051 interface onto different X widget toolkits and also implements some
1052 widgets of its own that behave like Motif widgets but are faster, free,
1053 and in some cases more powerful. The code in this directory compiles
1054 into a library and is mostly independent from XEmacs.
1056 The `etc/' directory contains various data files associated with
1057 XEmacs. Some of them are actually read by XEmacs at startup; others
1058 merely contain useful information of various sorts.
1060 The `lib-src/' directory contains C code for various auxiliary
1061 programs that are used in connection with XEmacs. Some of them are used
1062 during the build process; others are used to perform certain functions
1063 that cannot conveniently be placed in the XEmacs executable (e.g. the
1064 `movemail' program for fetching mail out of `/var/spool/mail', which
1065 must be setgid to `mail' on many systems; and the `gnuclient' program,
1066 which allows an external script to communicate with a running XEmacs
1069 The `man/' directory contains the sources for the XEmacs
1070 documentation. It is mostly in a form called Texinfo, which can be
1071 converted into either a printed document (by passing it through TeX) or
1072 into on-line documentation called "info files".
1074 The `info/' directory contains the results of formatting the XEmacs
1075 documentation as "info files", for on-line use. These files are used
1076 when you enter the Info system using `C-h i' or through the Help menu.
1078 The `dynodump/' directory contains auxiliary code used to build
1079 XEmacs on Solaris platforms.
1081 The other directories contain various miscellaneous code and
1082 information that is not normally used or needed.
1084 The first step of building involves running the `configure' program
1085 and passing it various parameters to specify any optional features you
1086 want and compiler arguments and such, as described in the `INSTALL'
1087 file. This determines what the build environment is, chooses the
1088 appropriate `s/' and `m/' file, and runs a series of tests to determine
1089 many details about your environment, such as which library functions
1090 are available and exactly how they work. The reason for running these
1091 tests is that it allows XEmacs to be compiled on a much wider variety
1092 of platforms than those that the XEmacs developers happen to be
1093 familiar with, including various sorts of hybrid platforms. This is
1094 especially important now that many operating systems give you a great
1095 deal of control over exactly what features you want installed, and allow
1096 for easy upgrading of parts of a system without upgrading the rest. It
1097 would be impossible to pre-determine and pre-specify the information for
1098 all possible configurations.
1100 In fact, the `s/' and `m/' files are basically _evil_, since they
1101 contain unmaintainable platform-specific hard-coded information.
1102 XEmacs has been moving in the direction of having all system-specific
1103 information be determined dynamically by `configure'. Perhaps someday
1104 we can `rm -rf src/s src/m'.
1106 When configure is done running, it generates `Makefile's and
1107 `GNUmakefile's and the file `src/config.h' (which describes the
1108 features of your system) from template files. You then run `make',
1109 which compiles the auxiliary code and programs in `lib-src/' and
1110 `lwlib/' and the main XEmacs executable in `src/'. The result of
1111 compiling and linking is an executable called `temacs', which is _not_
1112 the final XEmacs executable. `temacs' by itself is not intended to
1113 function as an editor or even display any windows on the screen, and if
1114 you simply run it, it will exit immediately. The `Makefile' runs
1115 `temacs' with certain options that cause it to initialize itself, read
1116 in a number of basic Lisp files, and then dump itself out into a new
1117 executable called `xemacs'. This new executable has been
1118 pre-initialized and contains pre-digested Lisp code that is necessary
1119 for the editor to function (this includes most basic editing functions,
1120 e.g. `kill-line', that can be defined in terms of other Lisp
1121 primitives; some initialization code that is called when certain
1122 objects, such as frames, are created; and all of the standard
1123 keybindings and code for the actions they result in). This executable,
1124 `xemacs', is the executable that you run to use the XEmacs editor.
1126 Although `temacs' is not intended to be run as an editor, it can, by
1127 using the incantation `temacs -batch -l loadup.el run-temacs'. This is
1128 useful when the dumping procedure described above is broken, or when
1129 using certain program debugging tools such as Purify. These tools get
1130 mighty confused by the tricks played by the XEmacs build process, such
1131 as allocation memory in one process, and freeing it in the next.
1134 File: internals.info, Node: XEmacs From the Inside, Next: The XEmacs Object System (Abstractly Speaking), Prev: XEmacs From the Perspective of Building, Up: Top
1136 XEmacs From the Inside
1137 **********************
1139 Internally, XEmacs is quite complex, and can be very confusing. To
1140 simplify things, it can be useful to think of XEmacs as containing an
1141 event loop that "drives" everything, and a number of other subsystems,
1142 such as a Lisp engine and a redisplay mechanism. Each of these other
1143 subsystems exists simultaneously in XEmacs, and each has a certain
1144 state. The flow of control continually passes in and out of these
1145 different subsystems in the course of normal operation of the editor.
1147 It is important to keep in mind that, most of the time, the editor is
1148 "driven" by the event loop. Except during initialization and batch
1149 mode, all subsystems are entered directly or indirectly through the
1150 event loop, and ultimately, control exits out of all subsystems back up
1151 to the event loop. This cycle of entering a subsystem, exiting back out
1152 to the event loop, and starting another iteration of the event loop
1153 occurs once each keystroke, mouse motion, etc.
1155 If you're trying to understand a particular subsystem (other than the
1156 event loop), think of it as a "daemon" process or "servant" that is
1157 responsible for one particular aspect of a larger system, and
1158 periodically receives commands or environment changes that cause it to
1159 do something. Ultimately, these commands and environment changes are
1160 always triggered by the event loop. For example:
1162 * The window and frame mechanism is responsible for keeping track of
1163 what windows and frames exist, what buffers are in them, etc. It
1164 is periodically given commands (usually from the user) to make a
1165 change to the current window/frame state: i.e. create a new frame,
1166 delete a window, etc.
1168 * The buffer mechanism is responsible for keeping track of what
1169 buffers exist and what text is in them. It is periodically given
1170 commands (usually from the user) to insert or delete text, create
1171 a buffer, etc. When it receives a text-change command, it
1172 notifies the redisplay mechanism.
1174 * The redisplay mechanism is responsible for making sure that
1175 windows and frames are displayed correctly. It is periodically
1176 told (by the event loop) to actually "do its job", i.e. snoop
1177 around and see what the current state of the environment (mostly
1178 of the currently-existing windows, frames, and buffers) is, and
1179 make sure that that state matches what's actually displayed. It
1180 keeps lots and lots of information around (such as what is
1181 actually being displayed currently, and what the environment was
1182 last time it checked) so that it can minimize the work it has to
1183 do. It is also helped along in that whenever a relevant change to
1184 the environment occurs, the redisplay mechanism is told about
1185 this, so it has a pretty good idea of where it has to look to find
1186 possible changes and doesn't have to look everywhere.
1188 * The Lisp engine is responsible for executing the Lisp code in
1189 which most user commands are written. It is entered through a
1190 call to `eval' or `funcall', which occurs as a result of
1191 dispatching an event from the event loop. The functions it calls
1192 issue commands to the buffer mechanism, the window/frame
1195 * The Lisp allocation subsystem is responsible for keeping track of
1196 Lisp objects. It is given commands from the Lisp engine to
1197 allocate objects, garbage collect, etc.
1201 The important idea here is that there are a number of independent
1202 subsystems each with its own responsibility and persistent state, just
1203 like different employees in a company, and each subsystem is
1204 periodically given commands from other subsystems. Commands can flow
1205 from any one subsystem to any other, but there is usually some sort of
1206 hierarchy, with all commands originating from the event subsystem.
1208 XEmacs is entered in `main()', which is in `emacs.c'. When this is
1209 called the first time (in a properly-invoked `temacs'), it does the
1212 1. It does some very basic environment initializations, such as
1213 determining where it and its directories (e.g. `lisp/' and `etc/')
1214 reside and setting up signal handlers.
1216 2. It initializes the entire Lisp interpreter.
1218 3. It sets the initial values of many built-in variables (including
1219 many variables that are visible to Lisp programs), such as the
1220 global keymap object and the built-in faces (a face is an object
1221 that describes the display characteristics of text). This
1222 involves creating Lisp objects and thus is dependent on step (2).
1224 4. It performs various other initializations that are relevant to the
1225 particular environment it is running in, such as retrieving
1226 environment variables, determining the current date and the user
1227 who is running the program, examining its standard input, creating
1228 any necessary file descriptors, etc.
1230 5. At this point, the C initialization is complete. A Lisp program
1231 that was specified on the command line (usually `loadup.el') is
1232 called (temacs is normally invoked as `temacs -batch -l loadup.el
1233 dump'). `loadup.el' loads all of the other Lisp files that are
1234 needed for the operation of the editor, calls the `dump-emacs'
1235 function to write out `xemacs', and then kills the temacs process.
1237 When `xemacs' is then run, it only redoes steps (1) and (4) above;
1238 all variables already contain the values they were set to when the
1239 executable was dumped, and all memory that was allocated with
1240 `malloc()' is still around. (XEmacs knows whether it is being run as
1241 `xemacs' or `temacs' because it sets the global variable `initialized'
1242 to 1 after step (4) above.) At this point, `xemacs' calls a Lisp
1243 function to do any further initialization, which includes parsing the
1244 command-line (the C code can only do limited command-line parsing,
1245 which includes looking for the `-batch' and `-l' flags and a few other
1246 flags that it needs to know about before initialization is complete),
1247 creating the first frame (or "window" in standard window-system
1248 parlance), running the user's init file (usually the file `.emacs' in
1249 the user's home directory), etc. The function to do this is usually
1250 called `normal-top-level'; `loadup.el' tells the C code about this
1251 function by setting its name as the value of the Lisp variable
1254 When the Lisp initialization code is done, the C code enters the
1255 event loop, and stays there for the duration of the XEmacs process.
1256 The code for the event loop is contained in `cmdloop.c', and is called
1257 `Fcommand_loop_1()'. Note that this event loop could very well be
1258 written in Lisp, and in fact a Lisp version exists; but apparently,
1259 doing this makes XEmacs run noticeably slower.
1261 Notice how much of the initialization is done in Lisp, not in C. In
1262 general, XEmacs tries to move as much code as is possible into Lisp.
1263 Code that remains in C is code that implements the Lisp interpreter
1264 itself, or code that needs to be very fast, or code that needs to do
1265 system calls or other such stuff that needs to be done in C, or code
1266 that needs to have access to "forbidden" structures. (One conscious
1267 aspect of the design of Lisp under XEmacs is a clean separation between
1268 the external interface to a Lisp object's functionality and its internal
1269 implementation. Part of this design is that Lisp programs are
1270 forbidden from accessing the contents of the object other than through
1271 using a standard API. In this respect, XEmacs Lisp is similar to
1272 modern Lisp dialects but differs from GNU Emacs, which tends to expose
1273 the implementation and allow Lisp programs to look at it directly. The
1274 major advantage of hiding the implementation is that it allows the
1275 implementation to be redesigned without affecting any Lisp programs,
1276 including those that might want to be "clever" by looking directly at
1277 the object's contents and possibly manipulating them.)
1279 Moving code into Lisp makes the code easier to debug and maintain and
1280 makes it much easier for people who are not XEmacs developers to
1281 customize XEmacs, because they can make a change with much less chance
1282 of obscure and unwanted interactions occurring than if they were to
projects / chise / xemacs-chise.git.1 / blob