On Sat, Mar 17, 2012 at 08:46:54PM +0200, Eli Zaretskii wrote:
> The doc string of ispell-dictionary-alist says, inter alia:
> Each element of this list is also a list:
> (DICTIONARY-NAME CASECHARS NOT-CASECHARS OTHERCHARS MANY-OTHERCHARS-P
> ISPELL-ARGS EXTENDED-CHARACTER-MODE CHARACTER-SET)
> CASECHARS, NOT-CASECHARS, and OTHERCHARS must be unibyte strings
> containing bytes of CHARACTER-SET. In addition, if they contain
> a non-ASCII byte, the regular expression must be a single
> `character set' construct that doesn't specify a character range
> for non-ASCII bytes.
> Why the restriction to unibyte character sets? This is quite a
> serious limitation, given that the modern spellers (aspell and
> hunspell) use UTF-8 as their default encoding.
At least for aspell ispell.el already uses utf8 as default communication
encoding and [:alpha:] as CASECHARS (and ^[:alpha:] as NOT-CASECHARS).
OTHERCHARS is guessed from aspell .dat file for given dictionary.
Since currently it is not possible to ask hunspell for installed
dictionaries (hunspell -D does not return control to the console)
no one tried something similar for hunspell.
> The only reason for this limitation I could find is in
> ispell-process-line, which assumes that the byte offsets returned by
> the speller can be used to compute character position of the
> misspelled word in the buffer. Are there any other places in
> ispell.el that assume unibyte characters?
Not sure if using utf8 and [:alpha:] has caused some problem for aspell,
I do not remember reports about this.
> If ispell-process-line is the only place, then it should be easy to
> extend it so it handles correctly UTF-8 in addition to unibyte
> character sets.
> In any case, I see no reason to specify CASECHARS, NOT-CASECHARS, and
> OTHERCHARS as ugly unibyte escapes, since their usage is entirely
> consistent with multibyte characters: they are used to construct
> regular expressions and match buffer text against those regexps.
IIRC, the reason to use octal escapes is mostly that they are encoding
independent. Otherwise a .emacs file may have mixed unibyte/multibyte
Current limitation in docstring may be only something left from old times. I
will try to look with recent ispell american dict, which can be called in
utf8. Will let you know.