From: shuhei Date: Thu, 31 May 2001 02:38:29 +0000 (+0000) Subject: Merge flim-1_14-rfc2231. X-Git-Tag: flim-1_14-rfc2231-merged X-Git-Url: http://git.chise.org/gitweb/?p=elisp%2Fflim.git;a=commitdiff_plain;h=19065081c8222a78c5352b5cf3f913ad08a29e72 Merge flim-1_14-rfc2231. --- diff --git a/ChangeLog b/ChangeLog index cba1a3c..0488c4c 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,60 @@ +2001-05-31 Shuhei KOBAYASHI + + Merged MIME Parameter Value decoder. + + * mime-parse.el (mime-decode-parameters): Renamed from + `mime-decode-parameter-plist'. + (mime-decode-parameter-alist): Removed. + (mime-decode-alist-to-plist): Ditto. + + * FLIM-API.en (mime-entity-encoding): Abolish optional argument + `default-encoding'. + (mime-parse-Content-Tranfer-Encoding): Ditto. + +2001-05-02 Shuhei KOBAYASHI + + * eword-decode.el (eword-decode-encoded-word): Don't use `let'. + + * mime-parse.el (mime-decode-parameter-plist): Modified + description of return value. + (mime-parse-Content-Type): Ditto. + (mime-read-Content-Type): Ditto. + (mime-parse-Content-Disposition): Ditto. + (mime-read-Content-Disposition): Ditto. + (mime-parse-Content-Transfer-Encoding): Ditto. + (mime-read-Content-Transfer-Encoding): Ditto. + +2001-05-01 Shuhei KOBAYASHI + + * mime-parse.el (mime-parse-parameters): Don't use `equal' for + strings. + (mime-parse-Content-Type): Ditto. + +2001-04-30 Shuhei KOBAYASHI + + * mime-parse.el (mime-decode-parameter-value): Use one temporary + buffer. + (mime-decode-parameter-plist): Changed internal data structure. + +2001-04-28 Shuhei KOBAYASHI + + * eword-decode.el (eword-encoded-word-regexp): Match for language. + (eword-decode-region): Refer the 7th parens, not 6th. + (eword-decode-encoded-word): Extract language information. + (eword-decode-encoded-text): New optional argument `language'. + + * mime-def.el (mime-charset-regexp): Updated for RFC2231. + + * mime-parse.el (mime-decode-parameter-plist): Fix regexp. + Use symbol for language information. + + * tests/test-rfc2231.el: Renamed all testcases. + +2001-04-27 Shuhei KOBAYASHI + + * mime-parse.el (mime-decode-parameter-value): Don't use + `int-char'. + 2001-04-27 Shuhei KOBAYASHI * mime.el (mime-entity-read-field): Would capitalize twice. @@ -8,11 +65,88 @@ * mmgeneric.el (mime-entity-fetch-field): Ditto. +2001-04-26 Shuhei KOBAYASHI + + * mime-parse.el (mime-decode-parameter-plist): Modified + description of return value. + (mime-parse-Content-Type): Ditto. + (mime-read-Content-Type): Ditto. + (mime-parse-Content-Disposition): Ditto. + (mime-read-Content-Disposition): Ditto. + (mime-read-Content-Transfer-Encoding): Ditto. + +2001-04-25 Shuhei KOBAYASHI + + * mime-parse.el (mime-lexical-analyze): Removed comments. + +2001-04-22 Shuhei KOBAYASHI + + * mime-parse.el (mime-decode-parameter-value): New + implementation; use temporary buffer for conversion. + (mime-decode-parameter-encode-segment): Ditto. + (mime-decode-parameter-plist): Would put empty language info. + + * test/test-rfc2231.el (test-rfc2231-10, test-rfc2231-11, + test-rfc2231-12): New testcases for language info. + +2001-04-22 Shuhei KOBAYASHI + + * mime-parse.el: Fixed comments. + + * test/test-rfc2231.el (test-rfc2231-9): New testcase. + +2001-04-22 Shuhei KOBAYASHI + + * test/test-rfc2231.el (test-rfc2231-7, test-rfc2231-8): + New testcases. + +2001-04-22 Shuhei KOBAYASHI + + * mime-def.el (mime-content-type-parameter): Expand + `mime-content-type-parameters'. + + * mime-parse.el (mime-parse-Content-Disposition): Add + description of return value to the docstring. + (mime-parse-Content-Transfer-Encoding): Ditto. + + * test/test-rfc2231.el: Made independent of internal + representation of Content-Type. + +2001-04-22 Shuhei KOBAYASHI + + * FLIM-MK (check-flim): Limit filename of test files. + + * test/test-rfc2231.el: New file. + +2001-04-19 Shuhei KOBAYASHI + + * mime-parse.el (mime-decode-parameter-plist): Shortcut for + parameters without extensions. + +2001-04-19 Shuhei KOBAYASHI + + * mime-def.el (make-mime-content-type): Don't reverse parameters. + (make-mime-content-disposition): New function. + + * mime-parse.el (mime-decode-parameter-value): Removed comments. + (mime-decode-parameter-encode-segment): New function. + (mime-decode-parameter-plist): New implementation. + Switched from decode-then-concat to concat-then-decode model. + (mime-parse-parameters): Strip quoted-pair in quoted-string. + (mime-parse-Content-Type): Use `make-mime-content-type'. + (mime-parse-Content-Disposition): Use + `make-mime-content-disposition'. + 2001-04-15 Shuhei KOBAYASHI * eword-decode.el (eword-lexical-analyze-internal): Fix typo. [cf. ] +2001-04-11 Shuhei KOBAYASHI + + * mime-parse.el (mime-decode-parameter-value): Allow lowercase. + (mime-decode-parameter-plist): Ditto. + 2001-04-10 Akihiro Arisawa * std11.el (std11-lexical-analyze): Fix typo. @@ -91,6 +225,68 @@ md5-dl.el, md5.el, sha1-dl.el, sha1-el.el, sha1.el: Update Copyright header. +2001-02-28 Shuhei KOBAYASHI + + * mime-parse.el (mime-decode-parameter-plist, + mime-decode-parameter-alist, mime-decode-parameters): + Add doc string. + +2001-02-28 Shuhei KOBAYASHI + + * mime-parse.el (mime-decode-parameter-value): Decode MIME charset + in multibyte buffer. [cf. ] + (mime-decode-parameter-plist): Downcase attributes. + [cf. ] + (mime-decode-parameters): Alias for `mime-decode-parameter-plist' + instead of `mime-decode-parameter-alist'. + Add autoload cookie. + (mime-parse-parameters-from-list): Make obsolete. + (mime-parse-parameters): Return results as a plist. + (mime-parse-Content-Type, mime-read-Content-Type): Moved type check + to the caller side. + (mime-parse-Content-Disposition, mime-read-Content-Disposition): + Ditto. + (mime-parse-Content-Transfer-Encoding, + mime-read-Content-Transfer-Encoding): Ditto. + +2001-02-27 Shuhei KOBAYASHI + + MIME Parameter Value decoder support. + + * mime-def.el (std11-quoted-pair-regexp, std11-non-qtext-char-list, + std11-qtext-regexp, std11-quoted-string-regexp): Removed. + (mime-media-type/subtype-regexp): Ditto. + (mime-attribute-char-regexp, mime-language-regexp, + mime-encoding-regexp): New constants. + + * mime-parse.el (mime-lexical-analyze): New function. + (mime/content-parameter-value-regexp, mime::parameter-regexp): + Removed. + (mime-parse-parameter): Ditto. + (mime-decode-parameter-value, mime-decode-parameter-plist, + mime-parse-alist-to-plist, mime-decode-parameter-alist): + New functions. + (mime-parse-parameters-from-list): New alias for + `mime-decode-parameter-plist'. + (mime-decode-parameters): New alias for + `mime-decode-parameter-alist'. + (mime-parse-parameters): New function. + (mime-disposition-type-regexp): Removed. + (mime-parse-Content-Type, mime-parse-Content-Disposition, + mime-parse-Content-Transfer-Encoding): New implementation. + (mime-read-Content-Type, mime-read-Content-Disposition, + mime-read-Content-Transfer-Encoding): Ditto. + +2001-02-26 Shuhei KOBAYASHI + + * hmac-md5.el, hmac-sha1.el: Modify comments. + +2001-02-25 Shuhei KOBAYASHI + + * hex-util.el, hmac-def.el, hmac-md5.el, hmac-sha1.el, + md5-dl.el, md5.el, sha1-dl.el, sha1-el.el, sha1.el: + Update Copyright header. + 2000-12-27 MORIOKA Tomohiko * mime-conf.el (mime-mailcap-file): Turn to non user option. diff --git a/FLIM-API.en b/FLIM-API.en index 539a559..533ce5d 100644 --- a/FLIM-API.en +++ b/FLIM-API.en @@ -200,12 +200,11 @@ ULEVEL specifies implementation level: (Usage: SEMI 1.14 MIME-View) -[Function] mime-entity-encoding (entity &optional default-encoding) +[Function] mime-entity-encoding (entity) Return content-transfer-encoding of ENTITY. - If the ENTITY does not have Content-Transfer-Encoding field, this - function returns DEFAULT-ENCODING. If it is nil, "7bit" is used as - default value. + If the ENTITY does not have valid Content-Transfer-Encoding field, + return nil. [Suggest] (Usage: SEMI 1.14 MIME-View) @@ -905,16 +904,16 @@ and return parsed it. [Function] mime-parse-Content-Transfer-Encoding (string) Parse STRING as field-body of Content-Transfer-Encoding field. + If STRING is not a valid Content-Transfer-Encoding field, + return nil. [Suggest] -[Function] mime-read-Content-Transfer-Encoding (&optional default-encoding) +[Function] mime-read-Content-Transfer-Encoding () Read field-body of Content-Transfer-Encoding field from current-buffer, and return it. - If is is not found, return DEFAULT-ENCODING. - [Suggest] diff --git a/FLIM-MK b/FLIM-MK index 701ff61..68c42d4 100644 --- a/FLIM-MK +++ b/FLIM-MK @@ -40,7 +40,7 @@ LISPDIR=%s\n" PREFIX LISPDIR)))) (defun check-flim () (config-flim) (require 'lunit) - (let ((files (directory-files "tests" t)) + (let ((files (directory-files "tests" t "^test-.*\\.el$")) (suite (lunit-make-test-suite))) (while files (if (file-regular-p (car files)) diff --git a/eword-decode.el b/eword-decode.el index e4143e6..328cc28 100644 --- a/eword-decode.el +++ b/eword-decode.el @@ -55,13 +55,19 @@ (eval-when-compile (concat (regexp-quote "=?") "\\(" - mime-charset-regexp + mime-charset-regexp ; 1 "\\)" + "\\(" + (regexp-quote "*") + mime-language-regexp ; 2 + "\\)?" (regexp-quote "?") - "\\([BbQq]\\)" + "\\(" + mime-encoding-regexp ; 3 + "\\)" (regexp-quote "?") "\\(" - eword-encoded-text-regexp + eword-encoded-text-regexp ; 4 "\\)" (regexp-quote "?=")))) ) @@ -224,7 +230,7 @@ such as a version of Net$cape)." "\\(\n?[ \t]\\)+" "\\(" eword-encoded-word-regexp "\\)") nil t) - (replace-match "\\1\\6") + (replace-match "\\1\\7") (goto-char (point-min)) ) (while (re-search-forward eword-encoded-word-regexp nil t) @@ -522,64 +528,66 @@ If SEPARATOR is not nil, it is used as header separator." word)) (defun eword-decode-encoded-word (word &optional must-unfold) - "Decode WORD if it is an encoded-word. - -If your emacs implementation can not decode the charset of WORD, it -returns WORD. Similarly the encoded-word is broken, it returns WORD. - -If MUST-UNFOLD is non-nil, it unfolds and eliminates line-breaks even -if there are in decoded encoded-word (generated by bad manner MUA such -as a version of Net$cape)." - (or (if (string-match eword-encoded-word-regexp word) - (let ((charset - (substring word (match-beginning 1) (match-end 1)) - ) - (encoding - (upcase - (substring word (match-beginning 2) (match-end 2)) - )) - (text - (substring word (match-beginning 3) (match-end 3)) - )) - (condition-case err - (eword-decode-encoded-text charset encoding text must-unfold) - (error - (funcall eword-decode-encoded-word-error-handler word err) - )) - )) + "Decode WORD as an encoded-word. + +If charset is unknown or unsupported, return WORD. +If encoding is unknown, or some error occurs while decoding, +`eword-decode-encoded-word-error-handler' is called with WORD and an +error condition. + +If MUST-UNFOLD is non-nil, unfold decoded WORD." + (or (and (string-match eword-encoded-word-regexp word) + (condition-case err + (eword-decode-encoded-text + ;; charset + (substring word (match-beginning 1)(match-end 1)) + ;; language + (when (match-beginning 2) + (intern + (downcase + (substring word (1+ (match-beginning 2))(match-end 2))))) + ;; encoding + (upcase + (substring word (match-beginning 3)(match-end 3))) + ;; encoded-text + (substring word (match-beginning 4)(match-end 4)) + must-unfold) + (error + (funcall eword-decode-encoded-word-error-handler word err)))) word)) ;;; @ encoded-text decoder ;;; -(defun eword-decode-encoded-text (charset encoding string +(defun eword-decode-encoded-text (charset language encoding string &optional must-unfold) "Decode STRING as an encoded-text. If your emacs implementation can not decode CHARSET, it returns nil. +If LANGUAGE is non-nil, it is put to `mime-language' text-property. If ENCODING is not \"B\" or \"Q\", it occurs error. So you should write error-handling code if you don't want break by errors. If MUST-UNFOLD is non-nil, it unfolds and eliminates line-breaks even if there are in decoded encoded-text (generated by bad manner MUA such as a version of Net$cape)." - (let ((cs (mime-charset-to-coding-system charset))) - (if cs - (let ((dest (encoded-text-decode-string string encoding))) - (when dest - (setq dest (decode-mime-charset-string dest charset)) - (if must-unfold - (mapconcat (function - (lambda (chr) - (cond ((eq chr ?\n) "") - ((eq chr ?\t) " ") - (t (char-to-string chr))) - )) - (std11-unfold-string dest) - "") - dest)))))) + (when (mime-charset-to-coding-system charset) + (let ((dest (encoded-text-decode-string string encoding))) + (when dest + (setq dest (decode-mime-charset-string dest charset)) + (when must-unfold + (mapconcat + (function + (lambda (chr) + (cond ((eq chr ?\n) "") + ((eq chr ?\t) " ") + (t (char-to-string chr))))) + (std11-unfold-string dest) "")) + (when language + (put-text-property 0 (length dest) 'mime-language language dest)) + dest)))) ;;; @ lexical analyze diff --git a/mime-def.el b/mime-def.el index 5ff449e..682d445 100644 --- a/mime-def.el +++ b/mime-def.el @@ -1,8 +1,9 @@ ;;; mime-def.el --- definition module about MIME -*- coding: iso-8859-4; -*- -;; Copyright (C) 1995,96,97,98,99,2000 Free Software Foundation, Inc. +;; Copyright (C) 1995,96,97,98,99,2000,2001 Free Software Foundation, Inc. ;; Author: MORIOKA Tomohiko +;; Shuhei KOBAYASHI ;; Keywords: definition, MIME, multimedia, mail, news ;; This file is part of FLIM (Faithful Library about Internet Message). @@ -28,10 +29,7 @@ (require 'mcharset) (require 'alist) -(eval-when-compile - (require 'cl) ; list* - (require 'luna) ; luna-arglist-to-arguments - ) +(eval-when-compile (require 'luna)) ; luna-arglist-to-arguments (eval-and-compile (defconst mime-library-product ["FLIM" (1 14 2) "Yagi-Nishiguchi"] @@ -129,36 +127,28 @@ If method is nil, this field will not be encoded." (defalias 'char-int 'identity)) -;;; @ about STD 11 +;;; @ MIME constants ;;; -(eval-and-compile - (defconst std11-quoted-pair-regexp "\\\\.") - (defconst std11-non-qtext-char-list '(?\" ?\\ ?\r ?\n)) - (defconst std11-qtext-regexp - (eval-when-compile - (concat "[^" std11-non-qtext-char-list "]")))) -(defconst std11-quoted-string-regexp - (eval-when-compile - (concat "\"" - (regexp-* - (regexp-or std11-qtext-regexp std11-quoted-pair-regexp)) - "\""))) +(defconst mime-tspecial-char-list + '(?\] ?\[ ?\( ?\) ?< ?> ?@ ?, ?\; ?: ?\\ ?\" ?/ ?? ?=)) +(defconst mime-token-regexp + (concat "[^" mime-tspecial-char-list "\000-\040]+")) +(defconst mime-attribute-char-regexp + (concat "[^" mime-tspecial-char-list "\000-\040" + "*'%" ; introduced in RFC 2231. + "]")) +(defconst mime-charset-regexp + (concat "[^" mime-tspecial-char-list "\000-\040" + "*'%" ; should not include "%"? + "]+")) -;;; @ about MIME -;;; +;; More precisely, length of "[A-Za-z]+" is limited to at most 8. +;; (defconst mime-language-regexp "[A-Za-z]+\\(-[A-Za-z]+\\)*") +(defconst mime-language-regexp "[-A-Za-z]+") -(eval-and-compile - (defconst mime-tspecial-char-list - '(?\] ?\[ ?\( ?\) ?< ?> ?@ ?, ?\; ?: ?\\ ?\" ?/ ?? ?=))) -(defconst mime-token-regexp - (eval-when-compile - (concat "[^" mime-tspecial-char-list "\000-\040]+"))) -(defconst mime-charset-regexp mime-token-regexp) - -(defconst mime-media-type/subtype-regexp - (concat mime-token-regexp "/" mime-token-regexp)) +(defconst mime-encoding-regexp mime-token-regexp) ;;; @@ base64 / B @@ -204,10 +194,9 @@ If method is nil, this field will not be encoded." ;;; (defsubst make-mime-content-type (type subtype &optional parameters) - (list* (cons 'type type) - (cons 'subtype subtype) - (nreverse parameters)) - ) + (cons (cons 'type type) + (cons (cons 'subtype subtype) + parameters))) (defsubst mime-content-type-primary-type (content-type) "Return primary-type of CONTENT-TYPE." @@ -215,15 +204,15 @@ If method is nil, this field will not be encoded." (defsubst mime-content-type-subtype (content-type) "Return subtype of CONTENT-TYPE." - (cdr (cadr content-type))) + (cdr (car (cdr content-type)))) (defsubst mime-content-type-parameters (content-type) "Return parameters of CONTENT-TYPE." - (cddr content-type)) + (cdr (cdr content-type))) (defsubst mime-content-type-parameter (content-type parameter) "Return PARAMETER value of CONTENT-TYPE." - (cdr (assoc parameter (mime-content-type-parameters content-type)))) + (cdr (assoc parameter (cdr (cdr content-type))))) (defsubst mime-type/subtype-string (type &optional subtype) @@ -237,6 +226,10 @@ If method is nil, this field will not be encoded." ;;; @ Content-Disposition ;;; +(defsubst make-mime-content-disposition (type &optional parameters) + (cons (cons 'type type) + parameters)) + (defsubst mime-content-disposition-type (content-disposition) "Return disposition-type of CONTENT-DISPOSITION." (cdr (car content-disposition))) diff --git a/mime-parse.el b/mime-parse.el index 2323fba..fef2ac3 100644 --- a/mime-parse.el +++ b/mime-parse.el @@ -1,11 +1,12 @@ ;;; mime-parse.el --- MIME message parser -;; Copyright (C) 1994,1995,1996,1997,1998,1999 Free Software Foundation, Inc. +;; Copyright (C) 1994,95,96,97,98,99,2001 Free Software Foundation, Inc. ;; Author: MORIOKA Tomohiko +;; Shuhei KOBAYASHI ;; Keywords: parse, MIME, multimedia, mail, news -;; This file is part of SEMI (Spadework for Emacs MIME Interfaces). +;; This file is part of FLIM (Faithful Library about Internet Message). ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU General Public License as @@ -57,161 +58,312 @@ be the result." (defun mime-analyze-tspecial (string start) (if (and (> (length string) start) (memq (aref string start) mime-tspecial-char-list)) - (cons (cons 'tpecials (substring string start (1+ start))) - (1+ start)) - )) + (cons (cons 'tspecials (substring string start (1+ start))) + (1+ start)))) (defun mime-analyze-token (string start) (if (and (string-match mime-token-regexp string start) (= (match-beginning 0) start)) (let ((end (match-end 0))) (cons (cons 'mime-token (substring string start end)) - ;;(substring string end) - end) - ))) + end)))) + +(defun mime-lexical-analyze (string) + "Analyze STRING as lexical tokens of MIME." + (let ((ret (std11-lexical-analyze string mime-lexical-analyzer)) + prev tail) + ;; skip leading linear-white-space. + (while (memq (car (car ret)) '(spaces comment)) + (setq ret (cdr ret))) + (setq prev ret + tail (cdr ret)) + ;; remove linear-white-space. + (while tail + (if (memq (car (car tail)) '(spaces comment)) + (progn + (setcdr prev (cdr tail)) + (setq tail (cdr tail))) + (setq prev (cdr prev) + tail (cdr tail)))) + ret)) ;;; @ field parser ;;; -(defconst mime/content-parameter-value-regexp - (concat "\\(" - std11-quoted-string-regexp - "\\|[^; \t\n]*\\)")) - -(defconst mime::parameter-regexp - (concat "^[ \t]*\;[ \t]*\\(" mime-token-regexp "\\)" - "[ \t]*=[ \t]*\\(" mime/content-parameter-value-regexp "\\)")) - -(defun mime-parse-parameter (str) - (if (string-match mime::parameter-regexp str) - (let ((e (match-end 2))) - (cons - (cons (downcase (substring str (match-beginning 1) (match-end 1))) - (std11-strip-quoted-string - (substring str (match-beginning 2) e)) - ) - (substring str e) - )))) - - -;;; @ Content-Type +(defun mime-decode-parameter-value (text charset language) + (with-temp-buffer + (set-buffer-multibyte nil) + (insert text) + (goto-char (point-min)) + (while (re-search-forward "%[0-9A-Fa-f][0-9A-Fa-f]" nil t) + (insert (prog1 (string-to-int + (buffer-substring (point)(- (point) 2)) + 16) + (delete-region (point)(- (point) 3))))) + (setq text (buffer-string)) + (when charset + ;; I believe that `decode-mime-charset-string' of mcs-e20.el should + ;; be independent of the value of `enable-multibyte-characters'. + (erase-buffer) + (set-buffer-multibyte t) + (setq text (decode-mime-charset-string text charset))) + (when language + (put-text-property 0 (length text) 'mime-language language text)) + text)) + +(defun mime-decode-parameter-encode-segment (segment) + (with-temp-buffer + (set-buffer-multibyte nil) + (insert segment) + (goto-char (point-min)) + (while (progn + (when (looking-at (eval-when-compile + (concat mime-attribute-char-regexp "+"))) + (goto-char (match-end 0))) + (not (eobp))) + (insert (prog1 (format "%%%02X" (char-int (char-after))) + (delete-region (point)(1+ (point)))))) + (buffer-string))) + +(defun mime-decode-parameters (params) + "Decode PARAMS as a property list of MIME parameter values. +Return value is an association list of MIME parameter values. +If parameter continuation is used, segments of values are concatenated. +If parameters contain charset information, values are decoded. +If parameters contain language information, it is set to `mime-language' +property of the decoded-value." + ;; (unless (zerop (% (length params) 2)) ...) + (let ((len (/ (length params) 2)) + dest eparams) + (while params + (if (and (string-match (eval-when-compile + (concat "^\\(" mime-attribute-char-regexp "+\\)" + "\\(\\*[0-9]+\\)?" ; continuation + "\\(\\*\\)?$")) ; charset/language + (car params)) + (> (match-end 0) (match-end 1))) + ;; parameter value extensions are used. + (let* ((attribute (downcase + (substring (car params) 0 (match-end 1)))) + (section (if (match-beginning 2) + (string-to-int + (substring (car params) + (1+ (match-beginning 2)) + (match-end 2))) + 0)) + ;; EPARAM := (ATTRIBUTE VALUES CHARSET LANGUAGE) + ;; VALUES := [1*VALUE] ; vector of LEN elements. + (eparam (assoc attribute eparams)) + (value (progn + (setq params (cdr params)) + (car params)))) + (if eparam + (setq eparam (cdr eparam)) + (setq eparam (list (make-vector len nil) nil nil) + eparams (cons (cons attribute eparam) eparams))) + ;; if parameter-name ends with "*", it is an extended-parameter. + (if (match-beginning 3) + (if (zerop section) + ;; extended-initial-parameter. + (if (string-match (eval-when-compile + (concat + "^\\(" mime-charset-regexp "\\)?" + "'\\(" mime-language-regexp "\\)?" + "'\\(\\(" mime-attribute-char-regexp + "\\|%[0-9A-Fa-f][0-9A-Fa-f]\\)+\\)$")) + value) + (progn + ;; text + (aset (car eparam) 0 + (substring value (match-beginning 3))) + (setq eparam (cdr eparam)) + ;; charset + (when (match-beginning 1) + (setcar eparam + (downcase + (substring value 0 (match-end 1))))) + (setq eparam (cdr eparam)) + ;; language + (when (match-beginning 2) + (setcar eparam + (intern + (downcase + (substring value + (match-beginning 2) + (match-end 2))))))) + ;; invalid parameter-value. + (aset (car eparam) 0 + (mime-decode-parameter-encode-segment value))) + ;; extended-other-parameter. + (if (string-match (eval-when-compile + (concat + "^\\(\\(" mime-attribute-char-regexp + "\\|%[0-9A-Fa-f][0-9A-Fa-f]\\)+\\)$")) + value) + (aset (car eparam) section value) + ;; invalid parameter-value. + (aset (car eparam) section + (mime-decode-parameter-encode-segment value)))) + ;; regular-parameter. parameter continuation only. + (aset (car eparam) section + (mime-decode-parameter-encode-segment value)))) + ;; parameter value extensions are not used, + ;; or invalid attribute-name (in RFC2231, although valid in RFC2045). + (setq dest (cons (cons (downcase (car params)) +;;; ;; decode (invalid!) encoded-words. +;;; (eword-decode-string +;;; (decode-mime-charset-string +;;; (car (cdr params)) +;;; default-mime-charset) +;;; 'must-unfold) + (car (cdr params))) + dest) + params (cdr params))) + (setq params (cdr params))) + ;; concat and decode parameters. + (while eparams + (setq dest (cons (cons (car (car eparams)) ; attribute + (mime-decode-parameter-value + (mapconcat (function identity) + (nth 1 (car eparams)) ; values + "") + (nth 2 (car eparams)) ; charset + (nth 3 (car eparams)) ; language + )) + dest) + eparams (cdr eparams))) + dest)) + +;;; for compatibility with flim-1_13-rfc2231 API. +(defalias 'mime-parse-parameters-from-list 'mime-decode-parameters) +(make-obsolete 'mime-parse-parameters-from-list 'mime-decode-parameters) + +(defun mime-parse-parameters (tokens) + "Parse TOKENS as MIME parameter values. +Return a property list, which is a list of the form +\(PARAMETER-NAME1 VALUE1 PARAMETER-NAME2 VALUE2...)." + (let (params attribute) + (while (and tokens + (eq (car (car tokens)) 'tspecials) + (string= (cdr (car tokens)) ";") + (setq tokens (cdr tokens)) + (eq (car (car tokens)) 'mime-token) + (progn + (setq attribute (cdr (car tokens))) + (setq tokens (cdr tokens))) + (eq (car (car tokens)) 'tspecials) + (string= (cdr (car tokens)) "=") + (setq tokens (cdr tokens)) + (memq (car (car tokens)) '(mime-token quoted-string))) + (setq params (cons (if (eq (car (car tokens)) 'quoted-string) + (std11-strip-quoted-pair (cdr (car tokens))) + (cdr (car tokens))) + (cons attribute params)) + tokens (cdr tokens))) + (nreverse params))) + + +;;; @@ Content-Type ;;; ;;;###autoload -(defun mime-parse-Content-Type (string) - "Parse STRING as field-body of Content-Type field. -Return value is - (PRIMARY-TYPE SUBTYPE (NAME1 . VALUE1)(NAME2 . VALUE2) ...) -or nil. PRIMARY-TYPE and SUBTYPE are symbol and NAME_n and VALUE_n -are string." - (setq string (std11-unfold-string string)) - (if (string-match `,(concat "^\\(" mime-token-regexp - "\\)/\\(" mime-token-regexp "\\)") string) - (let* ((type (downcase - (substring string (match-beginning 1) (match-end 1)))) - (subtype (downcase - (substring string (match-beginning 2) (match-end 2)))) - ret dest) - (setq string (substring string (match-end 0))) - (while (setq ret (mime-parse-parameter string)) - (setq dest (cons (car ret) dest) - string (cdr ret)) - ) - (make-mime-content-type (intern type)(intern subtype) - (nreverse dest)) - ))) +(defun mime-parse-Content-Type (field-body) + "Parse FIELD-BODY as a Content-Type field. +FIELD-BODY is a string. +Return value is a mime-content-type object. +If FIELD-BODY is not a valid Content-Type field, return nil." + (let ((tokens (mime-lexical-analyze field-body))) + (when (eq (car (car tokens)) 'mime-token) + (let ((primary-type (cdr (car tokens)))) + (setq tokens (cdr tokens)) + (when (and (eq (car (car tokens)) 'tspecials) + (string= (cdr (car tokens)) "/") + (setq tokens (cdr tokens)) + (eq (car (car tokens)) 'mime-token)) + (make-mime-content-type + (intern (downcase primary-type)) + (intern (downcase (cdr (car tokens)))) + (mime-decode-parameters + (mime-parse-parameters (cdr tokens))))))))) ;;;###autoload (defun mime-read-Content-Type () - "Read field-body of Content-Type field from current-buffer, -and return parsed it. Format of return value is as same as -`mime-parse-Content-Type'." - (let ((str (std11-field-body "Content-Type"))) - (if str - (mime-parse-Content-Type str) + "Parse field-body of Content-Type field of current-buffer. +Return value is a mime-content-type object. +If Content-Type field is not found, return nil." + (let ((field-body (std11-field-body "Content-Type"))) + (if field-body + (mime-parse-Content-Type field-body) ))) -;;; @ Content-Disposition +;;; @@ Content-Disposition ;;; -(eval-and-compile - (defconst mime-disposition-type-regexp mime-token-regexp) - ) - ;;;###autoload -(defun mime-parse-Content-Disposition (string) - "Parse STRING as field-body of Content-Disposition field." - (setq string (std11-unfold-string string)) - (if (string-match (eval-when-compile - (concat "^" mime-disposition-type-regexp)) string) - (let* ((e (match-end 0)) - (type (downcase (substring string 0 e))) - ret dest) - (setq string (substring string e)) - (while (setq ret (mime-parse-parameter string)) - (setq dest (cons (car ret) dest) - string (cdr ret)) - ) - (cons (cons 'type (intern type)) - (nreverse dest)) - ))) +(defun mime-parse-Content-Disposition (field-body) + "Parse FIELD-BODY as a Content-Disposition field. +FIELD-BODY is a string. +Return value is a mime-content-disposition object. +If FIELD-BODY is not a valid Content-Disposition field, return nil." + (let ((tokens (mime-lexical-analyze field-body))) + (when (eq (car (car tokens)) 'mime-token) + (make-mime-content-disposition + (intern (downcase (cdr (car tokens)))) + (mime-decode-parameters + (mime-parse-parameters (cdr tokens))))))) ;;;###autoload (defun mime-read-Content-Disposition () - "Read field-body of Content-Disposition field from current-buffer, -and return parsed it." - (let ((str (std11-field-body "Content-Disposition"))) - (if str - (mime-parse-Content-Disposition str) + "Parse field-body of Content-Disposition field of current-buffer. +Return value is a mime-content-disposition object. +If Content-Disposition field is not found, return nil." + (let ((field-body (std11-field-body "Content-Disposition"))) + (if field-body + (mime-parse-Content-Disposition field-body) ))) -;;; @ Content-Transfer-Encoding +;;; @@ Content-Transfer-Encoding ;;; ;;;###autoload -(defun mime-parse-Content-Transfer-Encoding (string) - "Parse STRING as field-body of Content-Transfer-Encoding field." - (let ((tokens (std11-lexical-analyze string mime-lexical-analyzer)) - token) - (while (and tokens - (setq token (car tokens)) - (std11-ignored-token-p token)) - (setq tokens (cdr tokens))) - (if token - (if (eq (car token) 'mime-token) - (downcase (cdr token)) - )))) +(defun mime-parse-Content-Transfer-Encoding (field-body) + "Parse FIELD-BODY as a Content-Transfer-Encoding field. +FIELD-BODY is a string. +Return value is a string. +If FIELD-BODY is not a valid Content-Transfer-Encoding field, return nil." + (let ((tokens (mime-lexical-analyze field-body))) + (when (eq (car (car tokens)) 'mime-token) + (downcase (cdr (car tokens)))))) ;;;###autoload -(defun mime-read-Content-Transfer-Encoding (&optional default-encoding) - "Read field-body of Content-Transfer-Encoding field from -current-buffer, and return it. -If is is not found, return DEFAULT-ENCODING." - (let ((str (std11-field-body "Content-Transfer-Encoding"))) - (if str - (mime-parse-Content-Transfer-Encoding str) - default-encoding))) +(defun mime-read-Content-Transfer-Encoding () + "Parse field-body of Content-Transfer-Encoding field of current-buffer. +Return value is a string. +If Content-Transfer-Encoding field is not found, return nil." + (let ((field-body (std11-field-body "Content-Transfer-Encoding"))) + (if field-body + (mime-parse-Content-Transfer-Encoding field-body) + ))) -;;; @ Content-Id / Message-Id +;;; @@ Content-ID / Message-ID ;;; ;;;###autoload (defun mime-parse-msg-id (tokens) - "Parse TOKENS as msg-id of Content-Id or Message-Id field." + "Parse TOKENS as msg-id of Content-ID or Message-ID field." (car (std11-parse-msg-id tokens))) ;;;###autoload (defun mime-uri-parse-cid (string) "Parse STRING as cid URI." - (inline - (mime-parse-msg-id (cons '(specials . "<") - (nconc - (cdr (cdr (std11-lexical-analyze string))) - '((specials . ">"))))))) + (mime-parse-msg-id (cons '(specials . "<") + (nconc + (cdr (cdr (std11-lexical-analyze string))) + '((specials . ">")))))) ;;; @ message parser diff --git a/tests/test-rfc2231.el b/tests/test-rfc2231.el new file mode 100644 index 0000000..62a3b4b --- /dev/null +++ b/tests/test-rfc2231.el @@ -0,0 +1,218 @@ +(require 'lunit) +(require 'mime) + +(luna-define-class test-rfc2231 (lunit-test-case)) + +;;; +;;; Parameter Value Continuations +;;; + +;; The content-type field +;; +;; Content-Type: message/external-body; access-type=URL; +;; URL*0="ftp://"; +;; URL*1="cs.utk.edu/pub/moore/bulk-mailer/bulk-mailer.tar" +;; +;; is semantically identical to +;; +;; Content-Type: message/external-body; access-type=URL; +;; URL="ftp://cs.utk.edu/pub/moore/bulk-mailer/bulk-mailer.tar" +;; +(luna-define-method test-rfc2231-continuation-1 ((case test-rfc2231)) + (lunit-assert + (eq + (mime-content-type-primary-type + (mime-parse-Content-Type "message/external-body; access-type=URL; + URL*0=\"ftp://\"; + URL*1=\"cs.utk.edu/pub/moore/bulk-mailer/bulk-mailer.tar\"")) + (mime-content-type-primary-type + (mime-parse-Content-Type "message/external-body; access-type=URL; + URL=\"ftp://cs.utk.edu/pub/moore/bulk-mailer/bulk-mailer.tar\""))))) + +(luna-define-method test-rfc2231-continuation-2 ((case test-rfc2231)) + (lunit-assert + (eq + (mime-content-type-subtype + (mime-parse-Content-Type "message/external-body; access-type=URL; + URL*0=\"ftp://\"; + URL*1=\"cs.utk.edu/pub/moore/bulk-mailer/bulk-mailer.tar\"")) + (mime-content-type-subtype + (mime-parse-Content-Type "message/external-body; access-type=URL; + URL=\"ftp://cs.utk.edu/pub/moore/bulk-mailer/bulk-mailer.tar\""))))) + +(luna-define-method test-rfc2231-continuation-3 ((case test-rfc2231)) + (lunit-assert + (string= + (mime-content-type-parameter + (mime-parse-Content-Type "message/external-body; access-type=URL; + URL*0=\"ftp://\"; + URL*1=\"cs.utk.edu/pub/moore/bulk-mailer/bulk-mailer.tar\"") + "access-type") + (mime-content-type-parameter + (mime-parse-Content-Type "message/external-body; access-type=URL; + URL=\"ftp://cs.utk.edu/pub/moore/bulk-mailer/bulk-mailer.tar\"") + "access-type")))) + +(luna-define-method test-rfc2231-continuation-4 ((case test-rfc2231)) + (lunit-assert + (string= + (mime-content-type-parameter + (mime-parse-Content-Type "message/external-body; access-type=URL; + URL*0=\"ftp://\"; + URL*1=\"cs.utk.edu/pub/moore/bulk-mailer/bulk-mailer.tar\"") + "url") + (mime-content-type-parameter + (mime-parse-Content-Type "message/external-body; access-type=URL; + URL=\"ftp://cs.utk.edu/pub/moore/bulk-mailer/bulk-mailer.tar\"") + "url")))) + +;;; +;;; Parameter Value Character Set and Language Information +;;; + +;; Content-Type: application/x-stuff; +;; title*=us-ascii'en-us'This%20is%20%2A%2A%2Afun%2A%2A%2A +(luna-define-method test-rfc2231-charset-language-1 ((case test-rfc2231)) + (lunit-assert + (string= + (mime-content-type-parameter + (mime-parse-Content-Type "application/x-stuff; + title*=us-ascii'en-us'This%20is%20%2A%2A%2Afun%2A%2A%2A") + "title") + "This is ***fun***"))) + +(luna-define-method test-rfc2231-charset-language-2 ((case test-rfc2231)) + (lunit-assert + (string= + (mime-content-type-parameter + (mime-parse-Content-Type "application/x-stuff; + title*=''This%20is%20%2A%2A%2Afun%2A%2A%2A") + "title") + "This is ***fun***"))) + +;;; +;;; Combining Character Set, Language, and Parameter Continuations +;;; + +;; Content-Type: application/x-stuff; +;; title*0*=us-ascii'en'This%20is%20even%20more%20; +;; title*1*=%2A%2A%2Afun%2A%2A%2A%20; +;; title*2="isn't it!" +(luna-define-method test-rfc2231-charset-language-continuation-1 ((case test-rfc2231)) + (lunit-assert + (string= + (mime-content-type-parameter + (mime-parse-Content-Type "application/x-stuff; + title*0*=us-ascii'en'This%20is%20even%20more%20; + title*1*=%2A%2A%2Afun%2A%2A%2A%20; + title*2=\"isn't it!\"") + "title") + "This is even more ***fun*** isn't it!"))) + +;; MIME states that parameters are not order sensitive. +(luna-define-method test-rfc2231-charset-language-continuation-2 ((case test-rfc2231)) + (lunit-assert + (string= + (mime-content-type-parameter + (mime-parse-Content-Type "application/x-stuff; + title*2=\"isn't it!\"; + title*1*=%2A%2A%2Afun%2A%2A%2A%20; + title*0*=us-ascii'en'This%20is%20even%20more%20") + "title") + "This is even more ***fun*** isn't it!"))) + +;; ABNF states that `ext-octet' is case-insensitive. +(luna-define-method test-rfc2231-charset-language-continuation-3 ((case test-rfc2231)) + (lunit-assert + (let ((case-fold-search nil)) + (string= + (mime-content-type-parameter + (mime-parse-Content-Type "application/x-stuff; + title*=us-ascii'en-us'This%20is%20%2a%2a%2afun%2a%2a%2a") + "title") + "This is ***fun***")))) + +;; unencoded segments MUST NOT be decoded. +(luna-define-method test-rfc2231-charset-language-continuation-4 ((case test-rfc2231)) + (lunit-assert + (string= + (mime-content-type-parameter + (mime-parse-Content-Type "application/x-stuff; + title*0*=us-ascii'en'This%20is%20even%20more%20; + title*1*=%2A%2A%2Afun%2A%2A%2A%20; + title*2=\"isn%27t%20it!\"") + "title") + "This is even more ***fun*** isn%27t%20it!"))) + +;;; +;;; Language specification in Encoded Words +;;; + +(luna-define-method test-rfc2231-encoded-word-1 ((case test-rfc2231)) + (lunit-assert + (string= + (eword-decode-encoded-word "=?US-ASCII?Q?Keith_Moore?=") + "Keith Moore"))) + +(luna-define-method test-rfc2231-encoded-word-2 ((case test-rfc2231)) + (lunit-assert + (string= + (eword-decode-encoded-word "=?US-ASCII*EN?Q?Keith_Moore?=") + "Keith Moore"))) + +(luna-define-method test-rfc2231-encoded-word-3 ((case test-rfc2231)) + (lunit-assert + (eq + (get-text-property + 0 'mime-language + (eword-decode-encoded-word "=?US-ASCII*EN?Q?Keith_Moore?=")) + 'en))) + +;;; +;;; Language specification in FLIM +;;; + +;; both flim-1_13-rfc2231 and flim-1_14-rfc2231 choose to put language +;; info to the `mime-language' text-property of the parameter value. + +(luna-define-method test-rfc2231-mime-language-1 ((case test-rfc2231)) + (lunit-assert + (eq + (get-text-property + 0 'mime-language + (mime-content-type-parameter + (mime-parse-Content-Type "application/x-stuff; + title*=us-ascii'en-us'This%20is%20%2A%2A%2Afun%2A%2A%2A") + "title")) + 'en-us))) + +(luna-define-method test-rfc2231-mime-language-2 ((case test-rfc2231)) + (lunit-assert + (eq + (get-text-property + 0 'mime-language + (mime-content-type-parameter + (mime-parse-Content-Type "application/x-stuff; + title*=US-ASCII'EN-US'This%20is%20%2A%2A%2Afun%2A%2A%2A") + "title")) + 'en-us))) + +(luna-define-method test-rfc2231-mime-language-3 ((case test-rfc2231)) + (lunit-assert + (null + (get-text-property + 0 'mime-language + (mime-content-type-parameter + (mime-parse-Content-Type "application/x-stuff; + title*=us-ascii''This%20is%20%2A%2A%2Afun%2A%2A%2A") + "title"))))) + +(luna-define-method test-rfc2231-mime-language-4 ((case test-rfc2231)) + (lunit-assert + (null + (get-text-property + 0 'mime-language + (mime-content-type-parameter + (mime-parse-Content-Type "application/x-stuff; + title*=''This%20is%20%2A%2A%2Afun%2A%2A%2A") + "title")))))