2 @c This is part of the XEmacs Lisp Reference Manual.
3 @c Copyright (C) 2001 Free Software Foundation, Inc.
4 @c See the file lispref.texi for copying conditions.
6 @setfilename ../../info/packaging.info
8 @c Macro to make formatting of the XEmacs pms name consistent.
9 @c Maybe @sc looks OK in HTML? If so, condition on Info.
12 XE@sc{macs} Packaging System
17 XEmacs Packaging System
21 @node Packaging, Lisp Data Types, Introduction, Top
26 The XEmacs distribution, starting with version 21, comes only with a
27 very basic set of built-in modes and libraries. Most of the libraries
28 that were part of the distribution of earlier versions of XEmacs are now
29 available separately. The user as well as the system administrator can
30 choose which packages to install; the actual installation process is
31 easy. This gives an installer the ability to tailor an XEmacs
32 installation for local needs with safe removal of unnecessary code.
34 This chapter describes how to package Lisp libraries for use with the
37 @emph{Please note carefully} that the term @dfn{package} as used in
38 XEmacs refers to an aggregation of Lisp code and/or data distributed as
39 a unit. It does not, as it does in many Lisps, refer to a way of
40 creating separate name spaces. XEmacs has no facility for providing
41 separate name spaces. (If we ever do get separate name spaces, we'll
42 probably regret overloading the nomenclature in this way, but it's
47 * Package Overview:: Lisp Libraries and Packages.
49 Packaging Lisp Libraries:
50 * Package Terminology:: Basic stuff.
51 * Building Packages:: Turn packaged source into a tarball.
52 * Makefile Targets:: Package @file{Makefile} targets
53 * Local.rules File:: Tell the @xpms{} about your host.
54 * Creating Packages:: Tell the @xpms{} about your package.
55 * Documenting Packages:: Explain your package to users and hackers.
56 @c * History:: History of the @xpms{}
57 @c * Installation:: Installing the @xpms{} with your (X)Emacs.
58 @c * Configuration:: Configuring the @xpms{} for use.
59 @c * Usage:: An overview of the operation of the @xpms{}.
60 @c * Bug Reports:: Reporting Bugs and Problems
61 @c * Frequently Asked Questions:: Questions and answers from the mailing list.
63 Internals and Package Release Engineering:
67 @node Package Overview, Package Terminology, , Packaging
68 @chapter An overview of the @xpms{}
70 The @xpms{} is a system for administering the installation, upgrade, and
71 removal of Lisp libraries. For the end user, it provides facilities for
72 determining availability of packages and which versions at remote
73 sites. It will download and automatically install a package, ensuring
74 that any old files from previous versions of the package are removed
75 first. By providing a standard set of hierarchies for installation, it
76 makes configuration of XEmacs simpler. Furthermore, packages normally
77 provide ancillary auto-autoloads and custom-loads libraries, which are
78 automatically detected and loaded by XEmacs upon startup. This means
79 that once installed, all facilities of package, including autoloading
80 the library upon invocation of a command provided by the library and
81 convenient configuration and customization, are automatically available
82 to the user. There is no need to add autoloads or keybindings to in the
83 init file, and structured configuration of the package is available
84 through the Customize system even before the libraries are loaded.
86 All of this convenience comes at a cost. The cost of administration at
87 the package level is negligible compared to the benefits, of course.
88 However, the requirement that XEmacs find and load auto-autoloads and
89 custom-loads libraries comes at a fairly large cost in startup time. In
90 order to reduce this cost, XEmacs imposes fairly strict conditions on
91 the structure of an installed package.
93 Meeting these requirements, as well as simply providing the
94 auto-autoloads and the information about availability and so on does
95 impose some costs on the library maintainer. The @xpms{} also provides
96 structure and utilities to the library maintainer to make these tasks
97 easier. This manual documents the requirements and the tools that the
98 @xpms{} provides to ensure that a package satisfies them.
102 * The Library Maintainer View::
103 * The Package Release Engineer View::
107 @node The User View, The Library Maintainer View, , Package Overview
108 @section The User View
110 @strong{N.B.} Much of the discussion in this section undoubtedly
111 belongs elsewhere, @ref{Packages,,,xemacs}.
113 From the user's point of view, an XEmacs binary package is simply a
114 standard tarball (usually gzipped) containing Lisp sources, compiled
115 Lisp, documentation, and possibly data files or supporting executables.
116 The tarball is unpacked using standard tools such as GNU tar and gzip.
117 The package system does impose certain requirements for automatic
118 configuration to work.
120 Here the main consideration is that the tarball ``expects'' to be
121 unpacked from the top of a package hierarchy. A @dfn{package hierarchy}
122 is basically an image of a classic Emacs ``run-in-place'' tree, with
123 @file{lisp}, @file{etc}, @file{info}, @file{man}, @file{lib-src}, and
124 @file{pkginfo} subdirectories of the top. The @file{pkginfo}
125 subdirectory is for use by the @xpms{} administration tools, and
126 currently contains a @file{MANIFEST.@var{package-name}} file for each
127 package to ensure that no cruft remains when a package is removed or
128 updated. The @file{lisp}, @file{etc}, and @file{lib-src} subdirectories
129 are further subdivided, with a subdirectory for each package. The
130 @file{info} directory obeys the usual conventions.
131 @emph{I.e.}, the @file{info} directory is flat
132 with a(n) (optional) @file{dir} file and one (set of) info file(s) per
133 package. The @file{man} subdirectory typically contains documentation
134 sources, separated by package. (It does not contain @file{man(1)}
135 pages, as Emacs provides very few of them.)
137 There are several standard package hierarchies, and administrators can
138 configure others at build time, while users can configure others at run
139 time. The standard system hierarchies are all subdirectories of an
140 @c #### This is possibly incorrect usage of "installation root."
141 XEmacs installation root, typically @file{/usr/local/lib/xemacs/}.
142 These are the @file{xemacs-packages}, @file{mule-packages},
143 @file{infodock-packages}, and @file{site-packages} hierarchies. Each
144 has the structure described above, but the purposes differ. The
145 @file{xemacs-packages} is the normal place for installing ``official''
146 packages and many third-party libraries. Unfortunately, it is not yet
147 quite possible to read libraries containing international characters
148 with a non-Mule XEmacs, so such libraries are sequestered in the
149 @file{mule-packages} hierarchy. Some packages are compatible only with
150 the Infodock development environment, and they will be installed in the
151 @file{infodock-packages} hierarchy. The @file{site-packages} hierarchy
152 is for packages not distributed by XEmacs.org, typically locally
155 Packages are in principle supposed to be XEmacs version-independent, but
156 if such dependencies are unavoidable, additional standard package
157 hierarchies may be installed under version directories, @emph{e.g.}
158 @file{/usr/local/lib/xemacs-21.4.6/}.
160 Users who do not have sufficient privilege to install packages in the
161 system hierarchies may install package hierarchies under @file{~/.xemacs}.
162 At present only the @file{xemacs-packages}, @file{mule-packages}, and
163 @file{site-packages} hierarchies are supported, but it might make sense to
164 extend this to support @file{infodock-packages} hierarchies in the future.
166 The package hierarchies are not searched directly for libraries to be
167 loaded; this would be very costly. Instead, the hierarchies are ordered
168 according to certain rules, and searched for package lisp directories at
169 invocation. These directories are added to the general
170 @code{load-path}. As usual, it is @code{load-path} that is searched at
171 run-time. This approach is somewhat costly at initialization, but
172 results in a very ``clean'' @code{load-path}.
174 The order of search can be changed at build time by specifying the
175 @samp{--package-path} option to @file{configure}, or at run-time by
176 specifying the @code{EMACSPACKAGEPATH} environment variable.
177 @xref{Packages,,,xemacs}.
179 @c #### The following description is quite possibly inaccurate.
180 @c Please, Michael, write some specs up!
181 The default order of search is hierarchically determined. First, the
182 roots are ordered. The @dfn{early} roots are the user-specific roots,
183 typically @file{~/.xemacs}. The @dfn{late} roots are the system roots,
184 typically @file{/usr/local/lib/xemacs-21.4.6} and
185 @file{/usr/local/lib/xemacs}, in that order. All hierarchies for a
186 given root are searched for package Lisp directories, which are appended
187 to @code{load-path} in the order found. Then the search proceeds to the
188 next root, whose results will be appended to the @code{load-path}
189 generated by previous roots.
191 Second, the hierarchies below each root are searched in the order
192 @file{site-packages}, @file{infodock-packages}, @file{mule-packages},
193 then @file{xemacs-packages}.
195 In each hierarchy there should be a @file{lisp} subdirectory, containing
196 directories named for the packages. Each package's Lisp libraries thus
197 are contained in a directory of the form
198 @var{root}/@var{hierarchy}/lisp/@var{package}/.
200 With such a complex search algorithm, the possibility of libraries being
201 shadowed by another library with the same name is quite real. There are
202 two considerations here. First, every XEmacs package contains certain
203 libraries with constant names. These are
207 Lisp code to inform the package administration system about the package
209 @item auto-autoloads.el
210 Lisp code to set up autoloaded functions and variables that may be
214 definitions of configuration variables for use with the Customize
218 They are special-cased, because the way they are used prevents shadowing
221 Second, it is possible that multiple copies of some library, or
222 different libraries with the same name, are installed in various places
223 in the hierarchies. To detect such shadows, use
224 @code{list-load-path-shadows}.
226 Finally, note that most basic Emacs functionality, including most of the
227 Lisp API, is implemented in Lisp libraries. Because they use internal
228 reserved APIs that are subject to change according the needs of the
229 developers, these libraries are distributed with the XEmacs binary, and
230 are called @dfn{core Lisp libraries}. Most core Lisp libraries are
231 ``preloaded'' into the Emacs binary and in normal usage are never
232 explicitly loaded. However, they can be explicitly loaded, and if so
233 they are searched on @code{load-path}.
234 @c #### Is this correct? It is not for C-h f, for example.
235 Furthermore, functions such as @code{locate-library} will also search on
236 the @code{load-path}. The searching takes place under somewhat
237 different rules from those used for packaged Lisp. It is probably
238 easiest to think of the package hierarchy searching algorithm as
239 receiving a @code{load-path} initialized to the core Lisp directories.
242 @node The Library Maintainer View, The Package Release Engineer View, The User View, Package Overview
243 @section The Library Maintainer View
245 From the library maintainer's viewpoint, the advantages to the @xpms{}
246 stem from the convenience to the user of installation and upgrade.
247 Since an installed package automatically registers its entry points via
248 autoload and its configuration variables with the Customize system,
249 configuration FAQs are reduced. When it's easy to upgrade, users learn
250 to try @samp{Tools | Packages | Update Installed Packages} before
251 posting a FAQ whose answer is ``long since fixed, please upgrade.''
253 This comes at some cost, as the library maintainer needs to arrange that
254 the package be installed in a directory structure that satisfies the
255 requirements of the @xpms{}. Autoload cookies and defcustoms must also
256 be added to existing libraries. The @xpms{} provides infrastructure to
257 assure that all of these annoyances need only be dealt with once. The
258 autoload cookies and defcustoms are beyond the scope of this chapter, but
259 most maintainers of modern packages are already familiar with these
262 The @xpms{} may be divided into the @dfn{infrastructure} common to all
263 packages, and the package-specific @dfn{control files}. The
264 infrastructure supports global builds, installation, and generation of
265 the ``sumo'' bundles of packages, as well as generation of individual
266 packages. The package control files describe the structure of the
267 package's source tree and provide administrative information.
270 * Infrastructure:: Global Makefiles and common rules.
271 * Control Files:: Package-specific Makefiles and administrative files.
272 * Obtaining:: Obtaining the @xpms{} and required utilities.
275 @node Infrastructure, Control Files, , The Library Maintainer View
276 @subsection Infrastructure
278 In order to get the greatest benefit from the @xpms{}, a library
279 maintainer should place the package sources in an appropriate place in
280 the XEmacs source package hierarchy, and arrange to have the source
281 package imported into the XEmacs CVS repository.
282 @c #### the parenthetical remark should go to "issues."
284 latter requirement can be quite burdensome. We are working on ways to
285 remove this requirement, but for the present it remains necessary.) The
286 library maintainer must also keep sources for any packages his/her
287 package requires. This requirement is somewhat burdensome, but unlikely
288 to be relaxed because of the implementation of compilation of macros in
289 Emacs Lisp. Macros cannot be called by compiled Lisp (the macro
290 expansion, which is always known at compile time, is inlined), so the
291 source of the macro must be loaded before compiling the called function.
293 The source package hierarchy may be rooted anywhere. The CVS module is
294 called ``packages,'' so we will refer to the top directory of the source
295 package hierarchy as ``the @file{packages} directory.'' The
296 @file{packages} directory contains two source subdirectories,
297 @file{xemacs-packages} and @file{mule-packages} (for convenience in
298 segregating the packages which depend on Mule, as they will cause
299 load-time errors in a non-Mule XEmacs). Each subdirectory contains many
300 package source directories, whose internal structure is not specified.
301 That structure is left up to the convenience of the library maintainers.
302 The requirements on the top directory of an individual package source
303 tree are given below, @ref{Control Files}.
305 The @file{packages} directory contains some auxiliary Lisp libraries
306 used in the compilation and packaging process. The content of these
307 libraries is of interest primarily to the packaging engineers, @ref{The
308 Package Release Engineer View}.
310 Finally, the @file{packages}, @file{packages/xemacs-packages}, and
311 @file{packages/mule-packages} directories contain @file{Makefile}s and
312 include files to control the package creation process. The
313 @file{Makefile}s in @file{packages/xemacs-packages} and
314 @file{packages/mule-packages} simply define the default sets of known
315 packages and include @file{../iterate.rules}, which implements recursive
316 building of all target packages.
318 The @samp{make} infrastructure in @file{packages} includes
322 controls building of individual packages, local installation, and
323 bundling of ``sumo'' tarballs
326 controls recursive builds of multiple packages
328 @item meta-iterate.rules
329 This is used by higher-level subdirectories that do not directly
330 contain packages. Subdirectories directly containing packages should
331 use iterate.rules instead.
334 provides the rules for building and packaging. Included by all package
338 provides local configuration, such as installation targets and staging
339 directories, as well as a number of kludges (many now obsolete) required
340 for building packages on the Windows platform.
342 @item Local.rules.template
343 a template for Local.rules, liberally commented
346 consistency checking for @file{Local.rules}, included by both the
347 top-level @file{Makefile} and by @file{XEmacs.rules}.
349 @item Local.rules.inc
350 a file to @code{include} in package @file{Makefile}s to be able to get
351 at variables in @file{Local.rules} @emph{before} including
354 @c #### Add to "issues"
355 @item package-compile.el
356 compile environment (@emph{e.g.}, load-path) setup.
359 Of these, only @file{Local.rules} and @file{package-compile.el} need to
360 be modified by the library maintainer. The changes to Local.rules
361 affect only your environment. This should need to be done only once
362 when first preparing the source environment. The necessary
363 modifications to @file{package-compile.el} need to be done for each
364 package and are discussed in the next section, @ref{Control Files}.
367 @node Control Files, Obtaining, Infrastructure, The Library Maintainer View
368 @subsection Control Files
370 Each package source must contain a number of control files in the
371 top-level directory. These files in general can be created and then
372 ignored, except for a few variables that need to be updated when new
373 versions are released. In most cases even adding, renaming, and
374 removing library source files can be handled by generic rules.
376 The package control files include
380 Must set a few @file{make} variables used by the administrative
381 utilities, and defines a couple of package-building targets to depend on
382 appropriate targets defined generically in @file{XEmacs.rules}. It may
383 also provide various variables and rules to transform the source tree
384 structure into that expected by the run-time system.
386 @item package-info.in
387 Provides a template for package information to be provided to the
388 administrative utilities. Static variables that are rarely changed
389 (such as the package's name) are entered as literals. Some variables
390 are generated by the build process (build dates and MD5 checksums) and
391 are automatically filled in. Finally, some variables that change
392 irregularly (dependences and even version numbers) are set as
393 @file{make} variables in the @file{Makefile}.
396 Not strictly required, but normally a ChangeLog will be added by the
397 XEmacs package maintainer if different from the upstream maintainer.
400 Generated. Simply does a @code{package-provide} for the package.
402 @item auto-autoloads.el
403 Generated. Read when XEmacs is initialized, and provides autoloads for
404 defuns and other forms in the sources that are marked with an
405 @dfn{autoload cookie} (@samp{;;;###autoload}.
407 @item custom-loads.el
408 Generated. Read when XEmacs is initialized, and informs the Customize
409 subsystem how to find the defcustom forms needed to create Customization
410 forms for the usre configuration variables of the package.
414 @node Obtaining, , Control Files, The Library Maintainer View
415 @subsection Obtaining the @xpms{} and Required Utilities
417 Currently both the infrastructure for creating XEmacs packages and the
418 package sources themselves are available only by CVS. See
419 @uref{http://www.xemacs.org/Develop/cvsaccess.html} for more
422 The @xpms{} currently requires GNU @file{make}, and XEmacs, to build
426 @node The Package Release Engineer View, , The Library Maintainer View, Package Overview
427 @subsection The Package Release Engineer View
429 The XEmacs Package Release Engineer is responsible for keeping the
430 system coherent. The changes to @file{packages/package-compile.el} and
431 @file{packages/xemacs-packages/Makefile} required to make the package
432 available to others, and for building SUMO tarballs, @emph{etc}, are
433 done by the Package Release Engineer, not individual library
436 The Package Release Engineer also maintains assorted infrastructure for
437 actually making releases. These are generally available for inspection
438 in the @code{xemacs-builds} module in the CVS repository.
440 @c #### To be completed.
443 @node Package Terminology, Building Packages, Package Overview, Packaging
444 @comment node-name, next, previous, up
445 @heading Package Terminology:
447 @subsection Libraries and Packages
451 A Lisp @dfn{library} is a single loadable file containing Lisp code. It
452 may be in source or byte-compiled form. A Lisp @dfn{package} is a set
453 of one or more libraries, usually related to each other in some way,
454 bundled with administrative information for convenient distribution.
456 @subsection Package Flavors
458 There are two main flavors of packages.
461 @item Regular Packages
462 @cindex regular package
463 A regular package is a set of Lisp libraries design to cooperate with
464 one another. A very complex example is Gnus. One may not in general
465 safely remove any of the component libraries.
467 @item Single-File Packages
468 @cindex single-file package
469 A single-file package is a collection of thematically related but
470 otherwise independent Lisp libraries. These libraries are bundled
471 together for convenience of the maintainers. Usually individual
472 libraries may be deleted at will without any loss of functionality of
473 other libraries in the package. However, we would recommend that you
474 follow this rule of thumb: "When in doubt, don't delete". If it's
475 really that big a deal, request that the maintainers split the package
476 into smaller aggregations.
479 @subsection Package Distributions
480 @cindex package distributions
481 @cindex binary packages
482 @cindex source packages
483 XEmacs Lisp packages are distributed in two ways. @dfn{Binary packages}
484 are used by system administrators and end users. They are packaged in a
485 form convenient for direct installation into an XEmacs package
486 hierarchy. @dfn{Source packages} are for developers and include all
487 files necessary for rebuilding byte-compiled lisp and creating tarballs
488 for distribution or installation. This is all of the package author's
489 source code plus all of the files necessary to build distribution
490 tarballs (Unix Tar format files, gzipped for space savings).
491 (Occasionally sources that are not relevant to XEmacs are usually
492 renamed to @file{file.upstream}.)
494 Currently, source packages are only available via CVS. See
495 @url{http://www.xemacs.org/Develop/cvsaccess.html} for details.
497 The package distributions are also split according to major features
498 required in XEmacs to support them. At present there are @dfn{generic}
499 packages, which can be loaded by @emph{any} XEmacs, and @dfn{Mule}
500 packages, which @emph{require} Mule support or they will cause errors
501 when loaded. Note that there is no guarantee that a generic package
502 will have any useful functionality in a minimally configured XEmacs. As
503 long as any XEmacs can successfully load the package's libraries
504 (perhaps given other required Lisp libraries), it will be classified as
505 generic. At the present time only Mule packages need be treated
506 specially, and even those only if they contain multibyte characters.
509 @node Building Packages, Makefile Targets, Package Terminology, Packaging
510 @comment node-name, next, previous, up
511 @cindex building packages
512 @cindex package building
513 @heading Building Packages:
514 Currently, source packages are only available via anonymous CVS. See
515 @url{http://www.xemacs.org/Develop/cvsaccess.html} for details of
516 checking out the @file{packages} module.
518 @subsection Prerequisites for Building Source Packages
523 (or a BSD compatible install program).
525 (3.79 or later preferred).
527 (4.2 from texinfo-4.2)
532 @item A properly configured @file{Local.rules} file.
533 @ref{Local.rules File}.
536 And of course, XEmacs, 21.0 or higher.
538 @section What You Can Do With Source Packages
540 The packages CVS sources are most useful for creating XEmacs package
541 tarballs for installation into your own XEmacs installations or for
542 distributing to others.
544 It should be noted that most of the package @file{Makefile}s do
545 @emph{not} need to contain @emph{any} target rules. Everything is
546 handled from the @file{XEmacs.rules} file, located in the toplevel
547 directory of the packages source tree.
550 @node Makefile Targets, Local.rules File, Building Packages, Packaging
551 @cindex package makefile targets
552 @chapter @file{Makefile} targets
553 The following targets can be used when running @code{make} to build the
558 Removes any documentation files that have been processed by @TeX{}.
561 Does a @code{mostlyclean}, plus removes generated postscript and dvi
562 files. Also removes any generated .elc files, along with the normal
563 .elc files in the package and HTML and .info files.
566 Use this when preparing a distribution. It kills anything that can be
570 Does a @code{distclean} and also removes any backup files (@file{*~})
571 and @file{core} files.
574 Creates the @file{package-info} file from the @file{package-info.in} and
575 writes an entry in the @file{package-index} file.
578 Builds the package, including any Texinfo documentation (info format),
579 writes an entry into the @file{package-index} file and builds a tarball
580 of the package. Also writes an entry into @file{setup-packages.ini}
581 which is later used in the creation of netinstaller's @file{setup.ini}.
584 Builds and installs a package
587 Doesn't build anything, just installs it.
590 Generate the package's @file{auto-autoloads.el} file.
593 Creates the directories needed for installation and copies the files
594 there. Basically this is an alias for @code{install-only}.
597 Builds the HTML versions of the documentation.
600 Does most of the work. Builds the elcs, infos at a minimum.
603 @subsection The targets that most people would be interested in would be:
610 @item @code{install-only}
612 @item @code{distclean}
616 @node Local.rules File, Creating Packages, Makefile Targets, Packaging
617 @comment node-name, next, previous, up
619 @heading The Local.rules File:
620 This file in @file{packages} provides the @xpms{} with information about
621 the local configuration and environment. To create @file{Local.rules},
622 simply copy @file{Local.rules.template} from that directory to
623 @file{Local.rules} and edit it to suit your needs.
625 These are the variables in @file{Local.rules} that you may need to
630 The name (and path if needed) of the XEmacs binary to use for building
631 the packages. The default is @code{xemacs}.
634 This will enable some, as yet, unimplemented features in XEmacs 21.5 and
635 above. For now leave this blank (the default) regardless of the XEmacs
636 version you are using.
638 @item BUILD_WITHOUT_MULE
639 Set this to @samp{t} if you are using a non-Mule XEmacs. The default is
640 that this variable is not set (blank) which means to build @emph{with}
643 @item XEMACS_NATIVE_NT
644 Set this to @samp{t} if you are using a native Microsoft Windows build
645 of XEmacs (not a Cygwin build) to build the packages.
646 @strong{N.B.} To Windows users, you still need the Cygwin environment to
647 actually build the packages.
649 @item XEMACS_INSTALLED_PACKAGES_ROOT
650 Set this to the root of where you want the packages to be installed.
651 Under this directory will hang @file{xemacs-packages} and
652 @file{mule-packages}. See @var{NONMULE_INSTALLED_PACKAGES_ROOT} and
653 @var{MULE_INSTALLED_PACKAGES_ROOT}. The default for this is
654 @file{/usr/local/lib/xemacs}. Which may not be what you want if you are
655 developing XEmacs. To quote the comments in
656 @file{Local.rules.template}:
659 If you are developing XEmacs, you probably don't want to install the
660 packages under /usr/local, which is where the stable, released version
661 of XEmacs goes. Instead, we suggest a layout as described in the base
662 README file of recent versions of XEmacs. In a nutshell, we suggest
663 you put your source under /src/xemacs, and under this put the package
664 sources in package-src/, and the installed packages in xemacs-packages/
665 and mule-packages/. If you do everything this way, you might want to
666 set things as follows:
668 XEMACS_INSTALLED_PACKAGES_ROOT = $@{XEMACS_PACKAGES_BASE@}/..
670 which puts the xemacs-packages/ and mule-packages/ directories as sisters
671 of the package-src/ directory, and you have to tell configure the location
672 of the installed packages using `--package-path', something like
674 configure --package-path=/src/xemacs/xemacs-packages;/src/xemacs/mule-packages
678 The default is unset (blank). If you set this to @samp{t} then
679 @code{make install} will create a @dfn{symlink farm} of the installed
680 packages under @var{XEMACS_INSTALLED_PACKAGES_ROOT}. Obviously, for
681 this to work, your system has to support symbolic links. This is as
682 close as you can get to @dfn{running in place} for the packages.
684 @item NONMULE_INSTALLED_PACKAGES_ROOT
685 This is where the non-Mule packages get installed to. The default is
686 @file{$@{XEMACS_INSTALLED_PACKAGES_ROOT@}/xemacs-packages}.
688 @item MULE_INSTALLED_PACKAGES_ROOT
689 This is where the Mule packages get installed to. The default is
690 @file{$@{XEMACS_INSTALLED_PACKAGES_ROOT@}/mule-packages}.
692 @item NONMULE_PACKAGES
693 A whitespace separated list of non-Mule packages to build/install.
696 NONMULE_PACKAGES = bbdb gnus xemacs-base prog-modes
699 The value for this variable can also be the symbol
700 @samp{xemacs-packages}, which means to build/install @emph{all} of the
701 non-Mule packages. The default is @samp{xemacs-packages}.
704 A whitespace separated list of Mule packages to build/install.
707 MULE_PACKAGES = mule-base leim locale
710 The value for this variable can also be the symbol
711 @samp{mule-packages}, which means to build/install @emph{all} of the
712 Mule packages. The default is @samp{mule-packages}.
715 The name of the package-index file. The default is @file{package-index}
716 and you probably don't need to worry about changing it.
719 The path to a BSD compatible install program. The default is
723 The path to GNU/tar. The default is @code{tar}.
726 The path to the bzip2 compression program. The default is unset
727 (blank). If this is set @file{.tar.bz2} archives will be built
728 @emph{in addition to} the @file{.tar.gz} archives.
731 For things that you @emph{don't} want to go into the package tarballs.
732 It takes the same format as GNU/tar's @code{--exclude} option. The
747 Set to the XEmacs command line option that forces running in
748 @dfn{vanilla} mode. The default is @samp{-vanilla}. You wouldn't ever
752 How to put XEmacs into @dfn{batch} mode. It also sets a couple of other
753 things and in the normal course of events you wouldn't need to alter
754 this from the default which is:
757 BATCH = $(VANILLA) -batch -eval \
758 '(setq stack-trace-on-error t \
759 load-always-display-messages t \
760 load-ignore-out-of-date-elc-files t \
761 load-show-full-path-in-messages t)'
765 The path to @code{makeinfo}. The default is @samp{makeinfo}
768 Set this to @samp{t} if you want to install HTML versions of the Texinfo
769 documentation. The default is unset (blank).
772 The path to the program that can convert Texinfo source to HTML. The
773 default is @code{texi2html}.
776 The path to the program that can convert Texinfo source to DVI. The
777 default is @code{texi2dvi}
780 The path to the program that can convert DVI to Postscript. The default
784 The path to the program that can convert Texinfo source to PDF format.
785 The default is @code{texi2pdf}.
788 The path to @TeX{}. The default is @code{tex}
791 The path to msgfmt. The default is @code{msgfmt}
794 The path to your copy command (GNU cp). The default is dependent on
795 whether or not @var{symlink} is set (@samp{t}).
797 If @var{symlink} is unset (blank), @var{RCOPY}'s default is
798 @code{cp -af}. If @var{symlink} is set (@samp{t}), @var{RCOPY}'s
799 default is @code{cp --force --recursive --symbolic-link}.
802 It should be noted that in most cases the defaults should be fine. Most
803 people will probably only need to alter:
806 @item @var{XEMACS_INSTALLED_PACKAGES_ROOT}
807 @item @var{NONMULE_INSTALLED_PACKAGES_ROOT}
808 @item @var{MULE_INSTALLED_PACKAGES_ROOT}
809 @item @var{NONMULE_PACKAGES}
810 @item @var{MULE_PACKAGES}
813 @node Creating Packages, Documenting Packages, Local.rules File, Packaging
814 @comment node-name, next, previous, up
815 @cindex creating packages
816 @chapter Creating Packages:
817 Creating a package from an existing Lisp library is not very difficult.
819 In addition to the Lisp libraries themselves, you need a
820 @ref{package-info.in} file and a simple @ref{Makefile}. The rest is
821 done by @file{XEmacs.rules}, part of the packaging system
825 * package-info.in:: package-info.in
826 * Makefile:: @file{Makefile}
829 @node package-info.in, Makefile,,Creating Packages
830 @chapter package-info.in
831 @cindex package-info.in
833 @file{package-info.in} contains information that gets injected into the
834 @file{package-index} file when @code{make bindist} is run. Here is a
835 real world example from the xemacs-base package (a description of each
836 field follows the example):
840 (standards-version 1.1
842 author-version AUTHOR_VERSION
844 build-date BUILD_DATE
845 maintainer MAINTAINER
850 description "Fundamental XEmacs support, you almost certainly need this."
854 provides (add-log advice-preload advice annotations assoc case-table chistory comint-xemacs comint compile debug ebuff-menu echistory edmacro ehelp electric enriched env facemenu ffap helper imenu iso-syntax macros novice outline passwd pp regexp-opt regi ring shell skeleton sort thing time-stamp timezone tq xbm-button xpm-button)
860 @subheading Description of the Fields in @file{package-info.in}:
863 The name of the package. In the case of the example it is
866 @item standards-version
867 Part of the internal package infrastructure, its value should always be
868 @samp{1.1}. Do not change this.
871 This is the XEmacs package version number of the package. It is set
872 from the @file{Makefile} variable @var{VERSION}. This is something that
873 the XEmacs Package Release Engineer deals with so there is no need for a
874 package maintainer to touch it. In @file{package-info.in} just put the
875 place-marker, @samp{VERSION} here.
878 This is the package's internal, or @samp{upstream} version number if it
879 has one. It is set from the @file{Makefile} variable
880 @var{AUTHOR_VERSION}.
883 This is the date of the last change made to the package. It is
884 auto-generated at build time, taken from the package's toplevel
888 The date the package was built. It is auto-generated.
891 This is the name and email address of the package's maintainer. It is
892 taken from the @file{Makefile} variable @var{MAINTAINER}.
895 An unused field, leave as @samp{xemacs}
898 An unused field, can be any of @samp{high}, @samp{medium}, or
902 The @samp{category} of the package. It is taken from the
903 @file{Makefile} variable @var{CATEGORY} and can be either
904 @samp{standard} for non-Mule packages, or @samp{mule} for Mule
905 packages. The is also provision for @samp{unsupported} in this field
906 which would be for packages that XEmacs.org do not distribute.
908 @strong{N.B.} As yet, the @xpms{} does @emph{not} support this type of
909 package. It will in the future.
912 Unused. Always @samp{nil}
915 A free form short description of the package.
918 The file name of the package's binary tarball. It is generated at build
919 time by @code{make bindist}.
922 The MD5 message digest of the package's binary tarball. Generated at
923 build time by @code{make bindist}.
926 The size in bytes of the package's binary tarball. Generated at build
930 A whitespace separated list of @emph{all} the features the package
931 provides. Surround the list with parens.
934 Taken from the @file{Makefile} variable @var{REQUIRES}. It is a list of
935 all the package's dependencies, including any macros and defstructs that
938 @samp{REQUIRES} cannot be correctly computed from the calls to
939 @code{require} in the package's library sources. @samp{REQUIRES} is
940 used to ensure that all macro and defstruct definitions used by the
941 package are available at build time. This is not merely a matter of
942 efficiency, to get the expansions inlined. In fact, it is
943 @emph{impossible} to call a macro by name in byte-compiled Emacs Lisp
944 code. Thus, if the macro expansion is not inlined, the call will result
945 in an error at run-time! Thus, packages providing libraries that would
946 be loaded because of autoload definitions must also be included.
949 Can either be @samp{regular} for a regular package, or
950 @samp{single-file} for a single file package.
952 @strong{N.B.} This doesn't refer to the number of lisp files in a
953 package. A single-file package can have multiple lisp files in it.
954 @xref{Package Terminology}.
957 The fields in @file{package-info.in} that need to be changed directly
967 Everything else is either set from the appropriate @file{Makefile}
968 variable, is auto-generated at build time, or is static.
970 @node Makefile,,package-info.in,Creating Packages
971 @chapter @file{Makefile}
972 @cindex Makefile, package
973 @cindex package Makefile
974 The @file{Makefile} is quite stylized. The idea is similar to an
975 @file{Imakefile} or an @code{automake} file: the complexity is hidden in
976 generic rules files, in this case the @file{XEmacs.rules} include file
977 in the top directory of the packages hierarchy.
979 It is important to note that the XEmacs used to compile packages is
980 the bare minimum: it is called with the @samp{-no-autoloads}. This
981 means that anything not dumped into XEmacs by default needs to be
982 specified in the @samp{REQUIRES} variable (for packaged Lisp) or in
983 some cases the @samp{PRELOADS} (autoloads used in libraries mentioned
986 There isn't much to an @xpms{} @file{Makefile}, basically it just
987 contains a few @file{Makefile} variables and that's it. See the
990 Here is a real world example, from the @samp{build} package:
993 # Makefile for build lisp code
995 # This file is part of XEmacs.
997 # XEmacs is free software; you can redistribute it and/or modify it
998 # under the terms of the GNU General Public License as published by the
999 # Free Software Foundation; either version 2, or (at your option) any
1002 # XEmacs is distributed in the hope that it will be useful, but WITHOUT
1003 # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
1004 # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
1007 # You should have received a copy of the GNU General Public License
1008 # along with XEmacs; see the file COPYING. If not, write to
1009 # the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
1010 # Boston, MA 02111-1307, USA.
1012 # For the time being, remove MULE_ELCS from the all dependencies if
1013 # building without Mule.
1016 AUTHOR_VERSION = 2.02
1017 MAINTAINER = Adrian Aichner <adrian@@xemacs.org>
1020 REQUIRES = xemacs-base pcl-cvs dired w3 prog-modes
1023 ELCS = build.elc build-report.elc
1027 include ../../XEmacs.rules
1030 Most packages don't need any more than what you see above. It is
1031 usually @emph{not} necessary to specify any special @file{Makefile}
1032 rules. Everything is handled from the @file{*.rules} files in the
1033 toplevel of the package source hierarchy.
1035 Of course, with that said, there are always exceptions to the rule. If
1036 you think that your package will need some special @file{Makefile}
1037 hackery contact the @email{xemacs-beta@@xemacs.org, XEmacs developers}.
1038 We distribute over 100 packages so the chances are good that you won't
1039 be the first to need such hackery and it is probably already catered
1042 @subheading @file{Makefile} Variables Explained:
1043 A number of @file{make} variables are defined by the @xpms{}. Some are
1044 required, others are optional. Of course your @file{Makefile} may
1045 define other variables for private use, but you should be careful not to
1046 choose names that conflict with variables defined and used by the
1049 The required variables are described in the table below.
1050 The corresponding field names for @file{package-info.in}, where
1051 relevant, are given in parentheses.
1053 @c This is the canonical place for this information. If there is
1054 @c unnecessary duplication with package-info.in documentation, shorten
1055 @c that and leave this full-length.
1059 The version of the XEmacs package, a numeric literal (a decimal
1060 fixed-point number with two-places of precision). The only person who
1061 ever needs to touch this is the XEmacs Packages Release Engineer.
1063 @item AUTHOR_VERSION
1065 The upstream author's version, an uninterpreted literal.
1069 A literal containing the XEmacs package's maintainer and his/her email
1073 The name of the package, a literal
1076 The type of package, a literal containing either @samp{regular} for
1077 regular packages, or @samp{single-file} for single-file packages. This
1078 should feed the @samp{type} field in @file{package-info.in}, but
1079 currently it doesn't.
1081 @strong{N.B.} @samp{single-file} here does @emph{not} refer to the
1082 number of lisp files in a package. @xref{Package Terminology}.
1086 A literal, either @samp{standard} or @samp{mule}. The non-Mule packages
1087 are @samp{standard} and the Mule packages are, you guessed it,
1088 @samp{mule}. This field is used at package installation time as part of
1089 the process of determining where a package should be installed to.
1093 A list of packages required to correctly build this package.
1095 Note that the usual form in @file{package-info.in} already has the
1096 parentheses, so the @file{make} variable should be set to a
1097 space-separated list of package names @emph{not} enclosed in
1100 The list is of @emph{packages}, not @emph{libraries}, as would
1101 ordinarily be provided to the Lisp @code{require} function.
1103 @samp{REQUIRES} cannot be correctly computed from the calls to
1104 @code{require} in the package's library sources. @samp{REQUIRES} is
1105 used to ensure that all macro and defstruct definitions used by the
1106 package are available at build time. This is not merely a matter of
1107 efficiency, to get the expansions inlined. In fact, it is
1108 @emph{impossible} to call a macro by name in byte-compiled Emacs Lisp
1109 code. Thus, if the macro expansion is not inlined, the call will result
1110 in an error at run-time! Thus, packages providing libraries that would
1111 be loaded because of autoload definitions must also be included.
1114 The list of the byte-compiled Lisp files used by the package. These
1115 files and their @file{.el} versions will be included in the binary
1116 package. This variable determines which libraries will be
1117 byte-compiled. These libraries are also deleted by @samp{make clean}.
1119 Note there is no sanity-checking done on this variable. If you put
1120 @samp{.el} files in here, they will not be compiled and they @emph{will}
1121 be deleted by @samp{make clean}. You would surely be very distressed if
1122 that happened, so be very careful. If this variable is left empty, none
1123 of your Lisp code will be compiled or packaged. This would be a less
1124 than amusing surprise, too.
1126 We don't consider this a feature, of course. Please do submit code to
1127 do sanity checking to @email{xemacs-patches@@xemacs.org}.
1130 Optional, but commonly used variables are explained below.
1134 A list of extra byte-compiled Lisp files used by the package to be
1135 installed in a subdirectory of the package's lisp directory. The same
1136 care should be taken with this as with @var{ELCS} in regard to
1140 The name of the subdirectory for the @var{ELCS_1} files to be installed
1141 to. Be sure to include @samp{$(PACKAGE)/} as part of the name.
1144 ELCS_1_DEST = $(PACKAGE)/extra
1147 Would put the @var{ELCS_1} files for the package, @samp{foo} into
1148 @file{xemacs-packages/lisp/foo/extra/}.
1150 @item EARLY_GENERATED_LISP
1151 For additional @file{.el} files that will be generated before any
1152 byte-compiling happens. Use this for @samp{autoload-type} files. You
1153 must write @file{Makefile} rules to build these files.
1155 @item GENERATED_LISP
1156 For additional @file{.el} files that will be generated at
1157 byte-compilation time. You must write @file{Makefile} rules to build
1161 This is used if you need to pass extra command line arguments to
1162 XEmacs to build the package. For instance, a specification for
1163 loading libraries containing macros before compiling the Lisp in the
1164 package. This is spliced directly into the invocation of XEmacs for
1165 byte-compilation, so it must contain the @samp{-l} flag for XEmacs:
1168 PRELOADS=-l ./apackage-macros.el -l ../bpackage/lisp/bpackage-macros.el
1171 Preloads are loaded before @file{package-compile.el}, so the
1172 @var{load-path} is minimal. Therefore @samp{PRELOADS} must specify a
1173 full path to packaged Lisp. The base @var{load-path} does include the
1174 core Lisp directory, so core libraries are found.
1177 The subdirectory in the package's source tree where the @file{.el} files
1178 reside. This is where the @file{auto-autoloads.el} file will be placed.
1180 @strong{N.B.} There is no need to use this variable if the @file{.el}
1181 files are in the package's toplevel directory. @var{AUTOLOAD_PATH}
1182 defaults to @samp{.}.
1184 @item PACKAGE_SUPPRESS
1185 Place calls to @code{package-suppress} here to indicate Lisp libraries
1186 that should only be available to particular versions of XEmacs. For
1190 PACKAGE_SUPPRESS = \
1191 (package-suppress 'xemacs-base \"regexp-opt\" '(emacs-version>= 21 5 11)) \
1192 (package-suppress 'xemacs-base \"easy-mmode\" '(emacs-version>= 21 5 11))
1195 @c Change this when Ben has committed the WS that implements
1196 @c `package-suppress' --SY.
1197 @strong{N.B.} This feature has not yet been implemented in XEmacs yet.
1198 It will appear in an upcoming version of XEmacs 21.5.
1201 Set this to @samp{t} if your package's Texinfo source file is located in
1202 the package's toplevel directory @emph{and} is named
1203 @file{$(PACKAGE).texi}.
1206 Use this to explicitly list Texinfo sources that @emph{aren't} in the
1207 package's toplevel directory. For example:
1210 EXPLICIT_DOCS = texi/$(PACKAGE).texi
1213 See @var{DOCS_TXI_EXTENSION} and @var{DOCS_TEXINFO_EXTENSION} if you
1214 don't use the @file{.texi} file extension on your Texinfo sources.
1216 @item EXTRA_TEXI_FILES
1217 List here extra Texinfo source files needed to build your
1218 documentation. Whatever is listed here is passed on to @code{makeinfo}
1221 @item EXTRA_HTML_FILES
1222 Use this to specify extra @file{.html} files to output.
1224 @item DOCS_TEXINFO_EXTENSION
1225 Set this to @samp{t} if your Texinfo source files have a @samp{.texinfo}
1228 @item DOCS_TXI_EXTENSION
1229 Set this to @samp{t} if your Texinfo source files have a @samp{.txi}
1232 @item EXTRA_DOC_FILES
1233 Files listed here will be installed to @file{.../man/$(PACKAGE)/}. For
1234 example, you might want to list @TeX{} files or @file{.eps} files here.
1237 Other files (such as extra Lisp sources or an upstream @file{Makefile})
1238 that are normally placed in the installed Lisp directory, but not
1239 byte-compiled. These files are @emph{preserved} by the @samp{clean}
1243 For files that need to be installed to @file{lib-src/$(PACKAGE)/}. If
1244 the files listed here need to be built you will have to write
1245 @file{Makefile} rules to do so.
1248 Any data files, such as pixmaps, READMEs, and ChangeLogs. These must be
1249 paths relative to the root of the package's source tree. These files
1250 will be copied to @samp{$(DATA_DEST)} for installation. Any directory
1251 component of the path for a file will be stripped, so that the
1252 file ends up in @samp{$(DATA_DEST)}, not in a subdiredtory.
1255 The directory where the files in @var{DATA_FILES} are installed to. It
1256 is a subdirectory of the installed @file{etc/} directory. Be sure to
1257 prefix this value with @samp{$(PACKAGE)}, for example:
1260 DATA_DEST = $(PACKAGE)/foo
1263 Would put files into @file{.../etc/$(PACKAGE)/foo/}.
1265 @item DATA_1_FILES ... DATA_35_FILES
1266 For data files that need to go into a different directory from
1269 @item DATA_1_DEST ... DATA_35_DEST
1270 The name of the subdirectory for files specified in @var{DATA_n_FILES}.
1271 And like @var{DATA_DEST}, be sure to prefix @samp{$(PACKAGE)} to the
1272 value of these variables.
1274 @item EXTRA_DEPENDENCIES
1275 For additional files to build that aren't appropriate to place in any
1276 other @file{Makefile} variable. You will need to write @file{Makefile}
1277 rules to build these files.
1280 @section @file{package-compile.el}
1281 @cindex package-compile.el
1282 @cindex compiling packages
1283 The @xpms{} does not automatically become aware of your package simply
1284 because there is a new subtree. If any package, including your own,
1285 requires any of your files, it must be explicitly added to the compile
1286 environment or loads/requires that search load-path will fail. The
1287 changes that need to be made are
1290 @item an entry in @code{package-directory-map}
1291 This tells the @xpms{} which distribution (currently
1292 @samp{xemacs-packages} or @samp{mule-packages}) your package is found
1293 in. It then looks in the distribution subdirectory whose name is the
1294 same as the package's.
1296 @item an entry in the @code{cond} in @code{package-name-to-directory}
1297 This is optional; it is necessary only if you keep your Lisp code
1298 somewhere other than the top-level directory of the package's source
1299 tree, eg, in @file{packages/xemacs-packages/@var{PACKAGE}/lisp}.
1302 This only needs to be done once, when the package is first added to the
1303 @xpms{}. (Well, when you randomly change the subdirectory layout, too.)
1304 Your changes to @file{package-compile.el} must be cleared and checked in
1305 by the XEmacs Package Release Engineer before your package will build
1306 correctly from a fresh checkout.
1308 This is unfortunate; it works pretty well once set up, but can cause
1309 confusion when first building a package in the @xpms{} context. In
1310 particular, if the @code{package-directory-map} entry for a required
1311 package, including the package itself, is not found, the necessary
1312 requires will not be executed by @file{package-compile.el}. If
1313 required functions are executed (under @code{eval-when-compile}),
1314 they won't be found and the compile will fail. If required function
1315 is actually a macro, the byte compiler will not recognize that,
1316 compile a function call to the macro. This will cause a run-time
1317 error because the byte-code interpreter does not know how to execute
1318 macros. (Macros can always be expanded at compile-time, and this is
1321 If your package keeps some or all Lisp code somewhere other than the top
1322 directory, then an entry in @code{package-name-to-directory} is also
1323 necessary, or requires will fail, leading to the problems just described.
1325 @node Documenting Packages, Issues, Creating Packages, Packaging
1326 @comment node-name, next, previous, up
1327 @cindex documenting packages
1328 @heading Documenting Packages:
1330 @c #### Add a documentation section to Internals, and xref here.
1331 Some random notes on documenting your package.
1333 Do write a Texinfo file. It's not that hard to do basically, and even
1334 using the more advanced features of Texinfo soon become natural. For a
1335 start, just grab the template @file{Samples/package.texi} from the
1336 @xpms{} source tree, and drop your current README into the Top node. At
1337 least this way your documentation will be accessible from the standard
1338 Info readers. Next, try to add lots of cross-referencing and logical
1339 markup, and then node structure.
1341 Address both end users and developer issues. You may not be the
1344 If you are maintaining a package that is part of the GNU Emacs
1345 distribution, you'll likely find that you occasionally synchronize your
1346 package with the GNU Emacs sources. When you synch a file,
1347 conventionally you should place a comment just above the standard
1348 @code{;;; Code} comment that looks like this:
1352 ;; GNU Emacs 21.1, 2002-02-08, Stephen Turnbull <stephen@@xemacs.org>
1355 This comment is a status flag; the ChangeLog doesn't really give the
1358 Do maintain a detailed ChangeLog.
1360 @node Issues, , Documenting Packages, Packaging