* The XEmacs Object System (Abstractly Speaking)::
* How Lisp Objects Are Represented in C::
* Rules When Writing New C Code::
+* Regression Testing XEmacs::
* A Summary of the Various XEmacs Modules::
* Allocation of Objects in XEmacs Lisp::
* Dumping::
* General Guidelines for Writing Mule-Aware Code::
* An Example of Mule-Aware Code::
+Regression Testing XEmacs
+
A Summary of the Various XEmacs Modules
* Low-Level Modules::
* Modules for Interfacing with the Operating System::
* Modules for Interfacing with X Windows::
* Modules for Internationalization::
+* Modules for Regression Testing::
Allocation of Objects in XEmacs Lisp
frames are displayed correctly. It is periodically told (by the event
loop) to actually ``do its job'', i.e. snoop around and see what the
current state of the environment (mostly of the currently-existing
-windows, frames, and buffers) is, and make sure that that state matches
+windows, frames, and buffers) is, and make sure that state matches
what's actually displayed. It keeps lots and lots of information around
(such as what is actually being displayed currently, and what the
environment was last time it checked) so that it can minimize the work
used for the Lisp object can vary. It can be either a simple type
(@code{long} on the DEC Alpha, @code{int} on other machines) or a
structure whose fields are bit fields that line up properly (actually, a
-union of structures is used). Generally the simple integral type is
-preferable because it ensures that the compiler will actually use a
-machine word to represent the object (some compilers will use more
-general and less efficient code for unions and structs even if they can
-fit in a machine word). The union type, however, has the advantage of
-stricter type checking. If you accidentally pass an integer where a Lisp
-object is desired, you get a compile error. The choice of which type
-to use is determined by the preprocessor constant @code{USE_UNION_TYPE}
-which is defined via the @code{--use-union-type} option to
-@code{configure}.
+union of structures is used). The choice of which type to use is
+determined by the preprocessor constant @code{USE_UNION_TYPE} which is
+defined via the @code{--use-union-type} option to @code{configure}.
+
+Generally the simple integral type is preferable because it ensures that
+the compiler will actually use a machine word to represent the object
+(some compilers will use more general and less efficient code for unions
+and structs even if they can fit in a machine word). The union type,
+however, has the advantage of stricter @emph{static} type checking.
+Places where a @code{Lisp_Object} is mistakenly passed to a routine
+expecting an @code{int} (or vice-versa), or a check is written @samp{if
+(foo)} (instead of @samp{if (!NILP (foo))}, will be flagged as errors.
+None of these lead to the expected results! @code{Qnil} is not
+represented as 0 (so @samp{if (foo)} will *ALWAYS* be true for a
+@code{Lisp_Object}), and the representation of an integer as a
+@code{Lisp_Object} is not just the integer's numeric value, but usually
+2x the integer +/- 1.)
+
+There used to be a claim that the union type simplified debugging.
+There may have been a grain of truth to this pre-19.8, when there was no
+@samp{lrecord} type and all objects had a separate type appearing in the
+tag. Nowadays, however, there is no debugging gain, and in fact
+frequent debugging *@emph{loss}*, since many debuggers don't handle
+unions very well, and usually there is no way to directly specify a
+union from a debugging prompt.
+
+Furthermore, release builds should *@emph{not}* be done with union type
+because (a) you may get less efficiency, with compilers that can't
+figure out how to optimize the union into a machine word; (b) even
+worse, the union type often triggers miscompilation, especially when
+combined with Mule and error-checking. This has been the case at
+various times when using GCC and MS VC, at least with @samp{--pdump}.
+Therefore, be warned!
+
+As of 2002 4Q, miscompilation is known to happen with current versions
+of @strong{Microsoft VC++} and @strong{GCC in combination with Mule,
+pdump, and KKCC} (no error checking).
Various macros are used to convert between Lisp_Objects and the
corresponding C type. Macros of the form @code{XINT()}, @code{XCHAR()},
nothing unless the corresponding configure error checking flag was
specified.
-@node Rules When Writing New C Code, A Summary of the Various XEmacs Modules, How Lisp Objects Are Represented in C, Top
+@node Rules When Writing New C Code, Regression Testing XEmacs, How Lisp Objects Are Represented in C, Top
@chapter Rules When Writing New C Code
@cindex writing new C code, rules when
@cindex C code, rules when writing new
If you want to make XEmacs faster, target your favorite slow benchmark,
run a profiler like Quantify, @code{gprof}, or @code{tcov}, and figure
-out where the cycles are going. Specific projects:
+out where the cycles are going. In many cases you can localize the
+problem (because a particular new feature or even a single patch
+elicited it). Don't hesitate to use brute force techniques like a
+global counter incremented at strategic places, especially in
+combination with other performance indications (@emph{e.g.}, degree of
+buffer fragmentation into extents).
+
+Specific projects:
@itemize @bullet
@item
@item
Speed up redisplay.
@item
-Speed up syntax highlighting. Maybe moving some of the syntax
-highlighting capabilities into C would make a difference.
+Speed up syntax highlighting. It was suggested that ``maybe moving some
+of the syntax highlighting capabilities into C would make a
+difference.'' Wrong idea, I think. When processing one large file a
+particular low-level routine was being called 40 @emph{million} times
+simply for @emph{one} call to @code{newline-and-indent}. Syntax
+highlighting needs to be rewritten to use a reliable, fast parser, then
+to trust the pre-parsed structure, and only do re-highlighting locally
+to a text change. Modern machines are fast enough to implement such
+parsers in Lisp; but no machine will ever be fast enough to deal with
+quadratic (or worse) algorithms!
@item
Implement tail recursion in Emacs Lisp (hard!).
@end itemize
Before submitting a patch, please try compiling at least once with
@example
-configure --with-mule --with-union-type --error-checking=all
+configure --with-mule --use-union-type --error-checking=all
@end example
Here are things to know when you create a new source file:
add an INIT_LRECORD_IMPLEMENTATION call to @code{syms_of_@var{foo}.c}
@end enumerate
-@node A Summary of the Various XEmacs Modules, Allocation of Objects in XEmacs Lisp, Rules When Writing New C Code, Top
+
+@node Regression Testing XEmacs, A Summary of the Various XEmacs Modules, Rules When Writing New C Code, Top
+@chapter Regression Testing XEmacs
+@cindex testing, regression
+
+The source directory @file{tests/automated} contains XEmacs' automated
+test suite. The usual way of running all the tests is running
+@code{make check} from the top-level source directory.
+
+The test suite is unfinished and it's still lacking some essential
+features. It is nevertheless recommended that you run the tests to
+confirm that XEmacs behaves correctly.
+
+If you want to run a specific test case, you can do it from the
+command-line like this:
+
+@example
+$ xemacs -batch -l test-harness.elc -f batch-test-emacs TEST-FILE
+@end example
+
+If something goes wrong, you can run the test suite interactively by
+loading @file{test-harness.el} into a running XEmacs and typing
+@kbd{M-x test-emacs-test-file RET <filename> RET}. You will see a log of
+passed and failed tests, which should allow you to investigate the
+source of the error and ultimately fix the bug.
+
+Adding a new test file is trivial: just create a new file here and it
+will be run. There is no need to byte-compile any of the files in
+this directory---the test-harness will take care of any necessary
+byte-compilation.
+
+Look at the existing test cases for the examples of coding test cases.
+It all boils down to your imagination and judicious use of the macros
+@code{Assert}, @code{Check-Error}, @code{Check-Error-Message}, and
+@code{Check-Message}.
+
+Here's a simple example checking case-sensitive and case-insensitive
+comparisons from @file{case-tests.el}.
+
+@example
+(with-temp-buffer
+ (insert "Test Buffer")
+ (let ((case-fold-search t))
+ (goto-char (point-min))
+ (Assert (eq (search-forward "test buffer" nil t) 12))
+ (goto-char (point-min))
+ (Assert (eq (search-forward "Test buffer" nil t) 12))
+ (goto-char (point-min))
+ (Assert (eq (search-forward "Test Buffer" nil t) 12))
+
+ (setq case-fold-search nil)
+ (goto-char (point-min))
+ (Assert (not (search-forward "test buffer" nil t)))
+ (goto-char (point-min))
+ (Assert (not (search-forward "Test buffer" nil t)))
+ (goto-char (point-min))
+ (Assert (eq (search-forward "Test Buffer" nil t) 12))))
+@end example
+
+This example could be inserted in a file in @file{tests/automated}, and
+it would be a complete test, automatically executed when you run
+@kbd{make check} after building XEmacs. More complex tests may require
+substantial temporary scaffolding to create the environment that elicits
+the bugs, but the top-level Makefile and @file{test-harness.el} handle
+the running and collection of results from the @code{Assert},
+@code{Check-Error}, @code{Check-Error-Message}, and @code{Check-Message}
+macros.
+
+In general, you should avoid using functionality from packages in your
+tests, because you can't be sure that everyone will have the required
+package. However, if you've got a test that works, by all means add it.
+Simply wrap the test in an appropriate test, add a notice that the test
+was skipped, and update the @code{skipped-test-reasons} hashtable.
+Here's an example from @file{syntax-tests.el}:
+
+@example
+;; Test forward-comment at buffer boundaries
+(with-temp-buffer
+
+ ;; try to use exactly what you need: featurep, boundp, fboundp
+ (if (not (fboundp 'c-mode))
+
+ ;; We should provide a standard function for this boilerplate,
+ ;; probably called `Skip-Test' -- check for that API with C-h f
+ (let* ((reason "c-mode unavailable")
+ (count (gethash reason skipped-test-reasons)))
+ (puthash reason (if (null count) 1 (1+ count))
+ skipped-test-reasons)
+ (Print-Skip "comment and parse-partial-sexp tests" reason))
+
+ ;; and here's the test code
+ (c-mode)
+ (insert "// comment\n")
+ (forward-comment -2)
+ (Assert (eq (point) (point-min)))
+ (let ((point (point)))
+ (insert "/* comment */")
+ (goto-char point)
+ (forward-comment 2)
+ (Assert (eq (point) (point-max)))
+ (parse-partial-sexp point (point-max)))))
+@end example
+
+@code{Skip-Test} is intended for use with features that are normally
+present in typical configurations. For truly optional features, or
+tests that apply to one of several alternative implementations (eg, to
+GTK widgets, but not Athena, Motif, MS Windows, or Carbon), simply
+silently omit the test.
+
+
+@node A Summary of the Various XEmacs Modules, Allocation of Objects in XEmacs Lisp, Regression Testing XEmacs, Top
@chapter A Summary of the Various XEmacs Modules
@cindex modules, a summary of the various XEmacs
* Modules for Interfacing with the Operating System::
* Modules for Interfacing with X Windows::
* Modules for Internationalization::
+* Modules for Regression Testing::
@end menu
@node Low-Level Modules
different colors, for easy reading. The C support is provided so that
this is fast.
+As of 21.4.10, bugs introduced at the very end of the 21.2 series in the
+``syntax properties'' code were fixed, and highlighting is acceptably
+quick again. However, presumably more improvements are possible, and
+the places to look are probably here, in the defun-traversing code, and
+in @file{syntax.c}, in the comment-traversing code.
@example
@code{forward-sexp}, and by @file{font-lock.c} to locate quoted strings,
comments, etc.
+@c #### Break this out into a separate node somewhere!
+Syntax codes are implemented as bitfields in an int. Bits 0-6 contain
+the syntax code itself, bit 7 is a special prefix flag used for Lisp,
+and bits 16-23 contain comment syntax flags. From the Lisp programmer's
+point of view, there are 11 flags: 2 styles X 2 characters X @{start,
+end@} flags for two-character comment delimiters, 2 style flags for
+one-character comment delimiters, and the prefix flag.
+
+Internally, however, the characters used in multi-character delimiters
+will have non-comment-character syntax classes (@emph{e.g.}, the
+@samp{/} in C's @samp{/*} comment-start delimiter has ``punctuation''
+(here meaning ``operator-like'') class in C modes). Thus in a mixed
+comment style, such as C++'s @samp{//} to end of line, is represented by
+giving @samp{/} the ``punctuation'' class and the ``style b first
+character of start sequence'' and ``style b second character of start
+sequence'' flags. The fact that class is @emph{not} punctuation allows
+the syntax scanner to recognize that this is a multi-character
+delimiter. The @samp{newline} character is given (single-character)
+``comment-end'' @emph{class} and the ``style b first character of end
+sequence'' @emph{flag}. The ``comment-end'' class allows the scanner to
+determine that no second character is needed to terminate the comment.
@example
+@node Modules for Regression Testing
+@section Modules for Regression Testing
+@cindex modules for regression testing
+@cindex regression testing, modules for
+
+@example
+test-harness.el
+base64-tests.el
+byte-compiler-tests.el
+case-tests.el
+ccl-tests.el
+c-tests.el
+database-tests.el
+extent-tests.el
+hash-table-tests.el
+lisp-tests.el
+md5-tests.el
+mule-tests.el
+regexp-tests.el
+symbol-tests.el
+syntax-tests.el
+@end example
+
+@file{test-harness.el} defines the macros @code{Assert},
+@code{Check-Error}, @code{Check-Error-Message}, and
+@code{Check-Message}. The other files are test files, testing various
+XEmacs modules.
+
+
+
@node Allocation of Objects in XEmacs Lisp, Dumping, A Summary of the Various XEmacs Modules, Top
@chapter Allocation of Objects in XEmacs Lisp
@cindex allocation of objects in XEmacs Lisp
different section of code.
@end enumerate
+If you don't understand whether to @code{GCPRO} in a particular
+instance, ask on the mailing lists. A general hint is that @code{prog1}
+is the canonical example
+
@cindex garbage collection, conservative
@cindex conservative garbage collection
Given the extremely error-prone nature of the @code{GCPRO} scheme, and
projects / chise / xemacs-chise.git / blobdiff