Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
hunspell(5)		      File Formats Manual		   hunspell(5)

NAME
       hunspell	- format of Hunspell dictionaries and affix files

DESCRIPTION
       Hunspell(1) Hunspell requires two files to define the way a language is
       being spell checked: a dictionary file containing words and  applicable
       flags,  and  an	affix file that	specifies how these flags will control
       spell checking.	An optional file is the	personal dictionary file.

Dictionary file
       A dictionary file (*.dic) contains a list of words, one per line.   The
       first  line of the dictionaries (except personal	dictionaries) contains
       the approximate word count (for optimal hash memory  size).  Each  word
       may  optionally	be  followed  by  a slash ("/")	and one	or more	flags,
       which represents	the word attributes, for example affixes.

       Note: Dictionary	words can contain also slashes when escaped  like   ""
       syntax.

       It's  worth  to add not only words, but word pairs to the dictionary to
       get correct suggestions for common misspellings with missing space,  as
       in  the	following  example, for	the bad	"alot" and "inspite" (see also
       "REP" and  field	 "ph:"	about  correct	suggestions  for  common  mis-
       spellings):

	      3
	      word
	      a	lot
	      in spite

Personal dictionary file
       Personal	 dictionaries  are  simple  word  lists. Asterisk at the first
       character position signs	prohibition.  A	second	word  separated	 by  a
       slash sets the affixation.

	      foo
	      Foo/Simpson
	      *bar

       In  this	 example, "foo"	and "Foo" are personal words, plus Foo will be
       recognized with affixes of Simpson (Foo's etc.) and bar is a  forbidden
       word.

Short example
       Dictionary file:

	      3
	      hello
	      try/B
	      work/AB

       The flags B and A specify attributes of these words.

       Affix file:

	      SET UTF-8
	      TRY esianrtolcdugmphbyfvkwzESIANRTOLCDUGMPHBYFVKWZ'

	      REP 2
	      REP f ph
	      REP ph f

	      PFX A Y 1
	      PFX A 0 re .

	      SFX B Y 2
	      SFX B 0 ed [^y]
	      SFX B y ied y

       In the affix file, prefix A and suffix B	have been defined.  Flag A de-
       fines a `re-' prefix. Class B defines two `-ed' suffixes. First B  suf-
       fix can be added	to a word if the last character	of the word isn't `y'.
       Second suffix can be added to the words terminated with an `y'.

       All accepted words with this  dictionary	 and  affix  combination  are:
       "hello",	"try", "tried",	"work",	"worked", "rework", "reworked".

AFFIX FILE GENERAL OPTIONS
       Hunspell	 source	distribution contains more than	80 examples for	option
       usage.

       SET encoding
	      Set character encoding of	words and morphemes in affix and  dic-
	      tionary  files.  Possible	values:	UTF-8, ISO8859-1 - ISO8859-10,
	      ISO8859-13 - ISO8859-15, KOI8-R, KOI8-U,	cp1251,	 ISCII-DEVANA-
	      GARI.

	      SET UTF-8

       FLAG value
	      Set  flag	type. Default type is the extended ASCII (8-bit) char-
	      acter.  `UTF-8' parameter	sets UTF-8 encoded  Unicode  character
	      flags.   The `long' value	sets the double	extended ASCII charac-
	      ter flag type, the `num' sets the	decimal	number flag type. Dec-
	      imal flags numbered from 1 to 65000, and in flag fields are sep-
	      arated by	comma.	BUG: UTF-8 flag	type doesn't work on ARM plat-
	      form.

	      FLAG long

       COMPLEXPREFIXES
	      Set  twofold  prefix stripping (but single suffix	stripping) eg.
	      for morphologically complex languages with right-to-left writing
	      system.

       LANG langcode
	      Set  language  code for language-specific	functions of Hunspell.
	      Use it to	activate special casing	of Azeri  (LANG	 az),  Turkish
	      (LANG  tr)  and  Crimean	Tatar (LANG crh), also not generalized
	      syllable-counting	compounding rules of Hungarian (LANG hu).

       IGNORE characters
	      Sets characters to ignore	dictionary words,  affixes  and	 input
	      words.   Useful  for optional characters,	as Arabic (harakat) or
	      Hebrew (niqqud) diacritical marks	(see tests/ignore.* test  dic-
	      tionary in Hunspell distribution).

       AF number_of_flag_vector_aliases

       AF flag_vector
	      Hunspell	can substitute affix flag sets with ordinal numbers in
	      affix rules (alias compression, see makealias tool). First exam-
	      ple with alias compression:

	      3
	      hello
	      try/1
	      work/2

       AF definitions in the affix file:

	      AF 2
	      AF A
	      AF AB

       It is equivalent	of the following dic file:

	      3
	      hello
	      try/A
	      work/AB

       See also	tests/alias* examples of the source distribution.

       Note I: If affix	file contains the FLAG parameter, define it before the
       AF definitions.

       Note II:	Use makealias utility in Hunspell distribution to compress aff
       and dic files.

       AM number_of_morphological_aliases

       AM morphological_fields
	      Hunspell	can  substitute	 also  morphological data with ordinal
	      numbers in affix rules (alias  compression).   See  tests/alias*
	      examples.

AFFIX FILE OPTIONS FOR SUGGESTION
       Suggestion  parameters  can  optimize  the  default  n-gram (similarity
       search in the dictionary	words based on the common 1, 2,	3, 4-character
       length common character-sequences), character swap and deletion sugges-
       tions of	Hunspell.  REP is suggested to fix the typical and  especially
       bad  language specific bugs, because the	REP suggestions	have the high-
       est priority in the suggestion list.  PHONE is for languages  with  not
       pronunciation based orthography.

       For short common	misspellings, it's important to	use the	ph: field (see
       later) to give the best suggestions.

       KEY characters_separated_by_vertical_line_optionally
	      Hunspell searches	and suggests words with	one different  charac-
	      ter  replaced  by	a neighbor KEY character. Not neighbor charac-
	      ters in KEY string separated by vertical line characters.	  Sug-
	      gested KEY parameters for	QWERTY and Dvorak keyboard layouts:

	      KEY qwertyuiop|asdfghjkl|zxcvbnm
	      KEY pyfgcrl|aeouidhtns|qjkxbmwvz

       Using  the first	QWERTY layout, Hunspell	suggests "nude"	and "node" for
       "*nide".	A character may	have more neighbors, too:

	      KEY qwertzuop|yxcvbnm|qaw|say|wse|dsx|sy|edr|fdc|dx|rft|gfv|fc|tgz|hgb|gv|zhu|jhn|hb|uji|kjm|jn|iko|lkm

       TRY characters
	      Hunspell can suggest right word forms, when they differ from the
	      bad  input  word	by  one	TRY character. The parameter of	TRY is
	      case sensitive.

       NOSUGGEST flag
	      Words signed with	NOSUGGEST flag are not	suggested  (but	 still
	      accepted when typed correctly). Proposed flag for	vulgar and ob-
	      scene words (see also SUBSTANDARD).

       MAXCPDSUGS num
	      Set max. number of suggested compound words  generated  by  com-
	      pound  rules.  The number	of the suggested compound words	may be
	      greater from the same 1-character	distance type.

       MAXNGRAMSUGS num
	      Set max. number of n-gram	suggestions. Value 0 switches off  the
	      n-gram suggestions (see also MAXDIFF).

       MAXDIFF [0-10]
	      Set  the similarity factor for the n-gram	based suggestions (5 =
	      default value; 0 = fewer n-gram suggestions, but min.  1;	 10  =
	      MAXNGRAMSUGS n-gram suggestions).

       ONLYMAXDIFF
	      Remove  all  bad n-gram suggestions (default mode	keeps one, see
	      MAXDIFF).

       NOSPLITSUGS
	      Disable word suggestions with spaces.

       SUGSWITHDOTS
	      Add dot(s) to suggestions, if input word terminates  in  dot(s).
	      (Not  for	 LibreOffice  dictionaries, because LibreOffice	has an
	      automatic	dot expansion mechanism.)

       REP number_of_replacement_definitions

       REP what	replacement
	      This table specifies modifications to try	first.	First  REP  is
	      the  header of this table	and one	or more	REP data line are fol-
	      lowing it.  With this table,  Hunspell  can  suggest  the	 right
	      forms  for the typical spelling mistakes when the	incorrect form
	      differs by more than 1 letter from  the  right  form  (see  also
	      "ph:").	The search string supports the regex boundary signs (^
	      and $).  For example a possible English replacement table	 defi-
	      nition to	handle misspelled consonants:

	      REP 5
	      REP f ph
	      REP ph f
	      REP tion$	shun
	      REP ^cooccurr co-occurr
	      REP ^alot$ a_lot

       Note  I:	 It's  very useful to define replacements for the most typical
       one-character mistakes, too: with REP you can add higher	priority to  a
       subset of the TRY suggestions (suggestion list begins with the REP sug-
       gestions).

       Note II:	Suggesting separated words, specify spaces with	underlines:

	      REP 1
	      REP onetwothree one_two_three

       Note III: Replacement table can be used for a  stricter	compound  word
       checking	with the option	CHECKCOMPOUNDREP.

       MAP number_of_map_definitions

       MAP string_of_related_chars_or_parenthesized_character_sequences
	      We  can  define language-dependent information on	characters and
	      character	sequences that	should	be  considered	related	 (i.e.
	      nearer than other	chars not in the set) in the affix file	(.aff)
	      by a map table.  With this table,	Hunspell can suggest the right
	      forms  for  words,  which	incorrectly choose the wrong letter or
	      letter groups from a related set more than once in a  word  (see
	      REP).

	      For  example a possible mapping could be for the German umlauted
	      A1/4 versus the  regular	u;  the	 word  FrA1/4hstA1/4ck	really
	      should be	written	with umlauted u's and not regular ones

	      MAP 1
	      MAP uA1/4

       Use parenthesized groups	for character sequences	(eg. for composed Uni-
       code characters):

	      MAP 3
	      MAP A(ss)	 (character sequence)
	      MAP i~(fi)  ("fi"	compatibility characters for Unicode fi	ligature)
	      MAP (A^3IL)o   (composed Unicode character: A^3 with bottom dot)

       PHONE number_of_phone_definitions

       PHONE what replacement
	      PHONE uses a table-driven	phonetic transcription algorithm  bor-
	      rowed from Aspell. It is useful for languages with not pronunci-
	      ation based orthography. You can add a full alphabet  conversion
	      and  other rules for conversion of special letter	sequences. For
	      detailed documentation see  http://aspell.net/man-html/Phonetic-
	      Code.html.   Note:  Multibyte  UTF-8  characters have not	worked
	      with bracket expression yet. Dash	expression  has	 signed	 bytes
	      and not UTF-8 characters yet.

       WARN flag
	      This  flag is for	rare words, which are also often spelling mis-
	      takes, see option	-r of command line Hunspell and	FORBIDWARN.

       FORBIDWARN
	      Words with flag WARN aren't accepted by the spell	checker	 using
	      this parameter.

OPTIONS	FOR COMPOUNDING
       BREAK number_of_break_definitions

       BREAK character_or_character_sequence
	      Define  new  break  points  for breaking words and checking word
	      parts separately.	Use ^ and $ to delete characters  at  end  and
	      start  of	the word. Rationale: useful for	compounding with join-
	      ing character or strings (for example,  hyphen  in  English  and
	      German  or hyphen	and n-dash in Hungarian). Dashes are often bad
	      break points for tokenization, because compounds with dashes may
	      contain  not  valid parts, too.)	With BREAK, Hunspell can check
	      both side	of these compounds, breaking the words at  dashes  and
	      n-dashes:

	      BREAK 2
	      BREAK -
	      BREAK --	  # n-dash

       Breaking	 are recursive,	so foo-bar, bar-foo and	foo-foo--bar-bar would
       be valid	compounds.  Note: The default word break of Hunspell is	equiv-
       alent of	the following BREAK definition:

	      BREAK 3
	      BREAK -
	      BREAK ^-
	      BREAK -$

       Hunspell	 doesn't  accept  the  "-word" and "word-" forms by this BREAK
       definition:

	      BREAK 1
	      BREAK -

       Switching off the default values:

	      BREAK 0

       Note II:	COMPOUNDRULE is	better for handling dashes and other  compound
       joining	characters  or	character  strings.  Use BREAK,	if you want to
       check words with	dashes or other	joining	characters  and	 there	is  no
       time  or	 possibility  to  describe  precise  compound  rules with COM-
       POUNDRULE (COMPOUNDRULE handles only the	suffixation of the  last  word
       part of a compound word).

       Note  III:  For command line spell checking of words with extra charac-
       ters, set WORDCHARS parameters: WORDCHARS --- (see tests/break.*) exam-
       ple

       COMPOUNDRULE number_of_compound_definitions

       COMPOUNDRULE compound_pattern
	      Define  custom  compound patterns	with a regex-like syntax.  The
	      first COMPOUNDRULE is a header with the number of	the  following
	      COMPOUNDRULE  definitions.  Compound  patterns  consist compound
	      flags, parentheses, star and question mark  meta	characters.  A
	      flag  followed  by  a  `*'  matches a word sequence of 0 or more
	      matches of words signed with this	compound flag.	 A  flag  fol-
	      lowed  by	 a  `?'	matches	a word sequence	of 0 or	1 matches of a
	      word signed with this compound flag.  See	tests/compound*.*  ex-
	      amples.

	      Note:  en_US  dictionary of OpenOffice.org uses COMPOUNDRULE for
	      ordinal number recognition (1st, 2nd, 11th, 12th,	 22nd,	112th,
	      1000122nd	etc.).

	      Note  II:	 In the	case of	long and numerical flag	types use only
	      parenthesized flags: (1500)*(2000)?

	      Note III:	COMPOUNDRULE flags work	completely separately from the
	      compounding  mechanisms  using COMPOUNDFLAG, COMPOUNDBEGIN, etc.
	      compound flags.  (Use  these  flags  on  different  entries  for
	      words).

       COMPOUNDMIN num
	      Minimum  length of words used for	compounding.  Default value is
	      3	letters.

       COMPOUNDFLAG flag
	      Words signed with	COMPOUNDFLAG may be in compound	words  (except
	      when  word  shorter than COMPOUNDMIN). Affixes with COMPOUNDFLAG
	      also permits compounding of affixed words.

       COMPOUNDBEGIN flag
	      Words signed with	COMPOUNDBEGIN (or with a signed	affix) may  be
	      first elements in	compound words.

       COMPOUNDLAST flag
	      Words  signed  with COMPOUNDLAST (or with	a signed affix)	may be
	      last elements in compound	words.

       COMPOUNDMIDDLE flag
	      Words signed with	COMPOUNDMIDDLE (or with	a signed affix)	may be
	      middle elements in compound words.

       ONLYINCOMPOUND flag
	      Suffixes	signed	with ONLYINCOMPOUND flag may be	only inside of
	      compounds	(Fuge-elements in German, fogemorphemes	 in  Swedish).
	      ONLYINCOMPOUND  flag works also with words (see tests/onlyincom-
	      pound.*).	 Note: also valuable to	flag compounding  parts	 which
	      are not correct as a word	by itself.

       COMPOUNDPERMITFLAG flag
	      Prefixes are allowed at the beginning of compounds, suffixes are
	      allowed at the end of compounds by default.  Affixes  with  COM-
	      POUNDPERMITFLAG may be inside of compounds.

       COMPOUNDFORBIDFLAG flag
	      Suffixes	with this flag forbid compounding of the affixed word.
	      Dictionary words with this flag are removed from	the  beginning
	      and middle of compound words, overriding the effect of COMPOUND-
	      PERMITFLAG.

       COMPOUNDMORESUFFIXES
	      Allow twofold suffixes within compounds.

       COMPOUNDROOT flag
	      COMPOUNDROOT flag	signs the compounds in the dictionary (Now  it
	      is used only in the Hungarian language specific code).

       COMPOUNDWORDMAX number
	      Set  maximum  word  count	in a compound word. (Default is	unlim-
	      ited.)

       CHECKCOMPOUNDDUP
	      Forbid word duplication in compounds (e.g. foofoo).

       CHECKCOMPOUNDREP
	      Forbid compounding, if the (usually bad) compound	word may be  a
	      non-compound  word  with	a REP fault. Useful for	languages with
	      `compound	friendly' orthography.

       CHECKCOMPOUNDCASE
	      Forbid upper case	characters at word boundaries in compounds.

       CHECKCOMPOUNDTRIPLE
	      Forbid compounding, if compound word contains  triple  repeating
	      letters (e.g. foo|ox or xo|oof). Bug: missing multi-byte charac-
	      ter support in UTF-8 encoding (works only	for 7-bit ASCII	 char-
	      acters).

       SIMPLIFIEDTRIPLE
	      Allow  simplified	 2-letter  forms of the	compounds forbidden by
	      CHECKCOMPOUNDTRIPLE.  It's useful	for Swedish and	Norwegian (and
	      for the old German orthography: Schiff|fahrt -> Schiffahrt).

       CHECKCOMPOUNDPATTERN number_of_checkcompoundpattern_definitions

       CHECKCOMPOUNDPATTERN endchars[/flag] beginchars[/flag] [replacement]
	      Forbid  compounding, if the first	word in	the compound ends with
	      endchars,	and next word begins with beginchars and  (optionally)
	      they have	the requested flags.  The optional replacement parame-
	      ter allows simplified compound form.

	      The special "endchars" pattern 0 (zero) limits the rule  to  the
	      unmodified stems (stems and stems	with zero affixes):

	      CHECKCOMPOUNDPATTERN 0/x /y

       Note:  COMPOUNDMIN doesn't work correctly with the compound word	alter-
       nation, so it may need to set COMPOUNDMIN to lower value.

       FORCEUCASE flag
	      Last word	part of	a compound with	flag FORCEUCASE	 forces	 capi-
	      talization  of  the whole	compound word. Eg. Dutch word "straat"
	      (street) with FORCEUCASE flags will allowed only in  capitalized
	      compound forms, according	to the Dutch spelling rules for	proper
	      names.

       COMPOUNDSYLLABLE	max_syllable vowels
	      Need for special compounding rules in Hungarian.	First  parame-
	      ter  is  the maximum syllable number, that may be	in a compound,
	      if words in compounds are	more than COMPOUNDWORDMAX.  Second pa-
	      rameter is the list of vowels (for calculating syllables).

       SYLLABLENUM flags
	      Need for special compounding rules in Hungarian.

AFFIX FILE OPTIONS FOR AFFIX CREATION
       PFX flag	cross_product number

       PFX flag	stripping prefix [condition [morphological_fields...]]

       SFX flag	cross_product number

       SFX flag	stripping suffix [condition [morphological_fields...]]
	      An  affix	 is either a prefix or a suffix	attached to root words
	      to make other words. We can define affix classes with  arbitrary
	      number  affix rules.  Affix classes are signed with affix	flags.
	      The first	line of	an affix class definition is the  header.  The
	      fields of	an affix class header:

	      (0) Option name (PFX or SFX)

	      (1) Flag (name of	the affix class)

	      (2) Cross	product	(permission to combine prefixes	and suffixes).
	      Possible values: Y (yes) or N (no)

	      (3) Line count of	the following rules.

	      Fields of	an affix rules:

	      (0) Option name

	      (1) Flag

	      (2) stripping characters from beginning (at prefix rules)	or end
	      (at suffix rules)	of the word

	      (3)  affix (optionally with flags	of continuation	classes, sepa-
	      rated by a slash)

	      (4) condition.

	      Zero stripping or	affix are indicated by zero. Zero condition is
	      indicated	 by  dot.   Condition is a simplified, regular expres-
	      sion-like	pattern, which must be met before the affix can	be ap-
	      plied.  (Dot  signs an arbitrary character. Characters in	braces
	      sign an arbitrary	character  from	 the  character	 subset.  Dash
	      hasn't  got  special  meaning, but circumflex (^)	next the first
	      brace sets the complementer character set.)

	      (5) Optional morphological fields	separated by spaces or tabula-
	      tors.

AFFIX FILE OTHER OPTIONS
       CIRCUMFIX flag
	      Affixes  signed  with  CIRCUMFIX flag may	be on a	word when this
	      word also	has a prefix with CIRCUMFIX flag and vice  versa  (see
	      circumfix.* test files in	the source distribution).

       FORBIDDENWORD flag
	      This  flag  signs	forbidden word form. Because affixed forms are
	      also forbidden, we can subtract a	subset from  set  of  the  ac-
	      cepted  affixed and compound words.  Note: usefull to forbid er-
	      roneous words, generated by the compounding mechanism.

       FULLSTRIP
	      With FULLSTRIP, affix rules can strip full words,	not  only  one
	      less characters, before adding the affixes, see fullstrip.* test
	      files in the source distribution).  Note:	conditions may be word
	      length without FULLSTRIP,	too.

       KEEPCASE	flag
	      Forbid  uppercased  and  capitalized  forms of words signed with
	      KEEPCASE flags. Useful for special  orthographies	 (measurements
	      and  currency  often  keep  their	 case in uppercased texts) and
	      writing systems (e.g. keeping lower  case	 of  IPA  characters).
	      Also valuable for	words erroneously written in the wrong case.

	      Note: With CHECKSHARPS declaration, words	with sharp s and KEEP-
	      CASE flag	may be	capitalized  and  uppercased,  but  uppercased
	      forms  of	these words may	not contain sharp s, only SS. See ger-
	      mancompounding example in	the tests directory  of	 the  Hunspell
	      distribution.

       ICONV number_of_ICONV_definitions

       ICONV pattern pattern2
	      Define input conversion table.  Note: useful to convert one type
	      of quote to another one, or change ligature.

       OCONV number_of_OCONV_definitions

       OCONV pattern pattern2
	      Define output conversion table.

       LEMMA_PRESENT flag
	      Deprecated. Use "st:" field instead of LEMMA_PRESENT.

       NEEDAFFIX flag
	      This flag	signs virtual stems  in	 the  dictionary,  words  only
	      valid  when  affixed.   Except,  if  the	dictionary  word has a
	      homonym or a zero	affix.	NEEDAFFIX works	also with prefixes and
	      prefix + suffix combinations (see	tests/needaffix5.*).

       PSEUDOROOT flag
	      Deprecated. (Former name of the NEEDAFFIX	option.)

       SUBSTANDARD flag
	      SUBSTANDARD  flag	 signs affix rules and dictionary words	(allo-
	      morphs) not used in morphological	generation and root words  re-
	      moved from suggestion. See also NOSUGGEST.

       WORDCHARS characters
	      WORDCHARS	 extends  tokenizer of Hunspell	command	line interface
	      with additional word character. For example, dot,	dash,  n-dash,
	      numbers, percent sign are	word character in Hungarian.

       CHECKSHARPS
	      SS  letter  pair	in uppercased (German) words may be upper case
	      sharp s (A).  Hunspell can handle	this special casing  with  the
	      CHECKSHARPS  declaration	(see also KEEPCASE flag	and tests/ger-
	      mancompounding example) in both spelling and suggestion.

Morphological analysis
       Hunspell's dictionary items and affix rules may have optional space  or
       tabulator  separated  morphological  description	 fields,  started with
       3-character (two	letters	and a colon) field IDs:

	       word/flags po:noun is:nom

       Example:	We define a simple resource with morphological informations, a
       derivative suffix (ds:) and a part of speech category (po:):

       Affix file:

	       SFX X Y 1
	       SFX X 0 able . ds:able

       Dictionary file:

	       drink/X po:verb

       Test file:

	       drink
	       drinkable

       Test:

	       $ analyze test.aff test.dic test.txt
	       > drink
	       analyze(drink) =	po:verb
	       stem(drink) = po:verb
	       > drinkable
	       analyze(drinkable) = po:verb ds:able
	       stem(drinkable) = drinkable

       You  can	see in the example, that the analyzer concatenates the morpho-
       logical fields in item and arrangement style.

Optional data fields
       Default morphological and other IDs (used in suggestion,	 stemming  and
       morphological generation):

       ph:    Alternative  transliteration  for	 better	suggestions, ie.  mis-
	      spellings	related	to the special orthography  and	 pronunciation
	      of the word. The best way	to handle common misspellings, so it's
	      worth to add ph: field to	the most affected few thousand dictio-
	      nary  words  (or word pairs etc.)	to get correct suggestions for
	      their misspellings.

	      For example:

	      Wednesday	ph:wendsay ph:wensday
	      Marseille	ph:maarsayl

       Hunspell	adds all ph: transliterations to the inner REP	table,	so  it
       will  always  suggest  the  correct word	for the	specified misspellings
       with the	highest	priority.

       The previous example is equivalent of the following REP definition:

	      REP 6
	      REP wendsay Wednesday
	      REP Wendsay Wednesday
	      REP wensday Wednesday
	      REP Wensday Wednesday
	      REP maarsayl Marseille
	      REP Maarsayl Marseille

       The asterisk at the end of the ph: pattern means	stripping  the	termi-
       nating  character  both from the	pattern	and the	word in	the associated
       REP rule:

	      pretty ph:prity*

       will result

	      REP 1
	      REP prit prett

       REP rule, resulting the following correct suggestions

	      *prity ->	pretty
	      *pritier -> prettier
	      *pritiest	-> prettiest

       Moreover, ph: fields can	handle suggestions with	more than  two	words,
       also different suggestions for the same misspelling:

	      do not know ph:dunno
	      don't know ph:dunno

       results

	      *dunno ->	do not know, don't know

       Note: if	available, ph: is used in n-gram similarity, too.

       The  ASCII arrow	"->" in	a ph: pattern means a REP rule (see REP), cre-
       ating arbitrary replacement rule	associated to the dictionary item:

	      happy/B ph:hepy ph:hepi->happi

       results

	      *hepy -> happy
	      *hepiest -> happiest

       st:    Stem. Optional: default stem is the dictionary item  in  morpho-
	      logical  analysis.  Stem field is	useful for virtual stems (dic-
	      tionary words with NEEDAFFIX flag) and morphological  exceptions
	      instead of new, single used morphological	rules.

	      feet  st:foot  is:plural
	      mice  st:mouse is:plural
	      teeth st:tooth is:plural

       Word forms with multiple	stems need multiple dictionary items:

	      lay po:verb st:lie is:past_2
	      lay po:verb is:present
	      lay po:noun

       al:    Allomorph(s).  A	dictionary item	is the stem of its allomorphs.
	      Morphological generation needs stem, allomorph and affix fields.

	      sing al:sang al:sung
	      sang st:sing
	      sung st:sing

       po:    Part of speech category.

       ds:    Derivational suffix(es).	Stemming doesn't  remove  derivational
	      suffixes.	  Morphological	generation depends on the order	of the
	      suffix fields.

	      In affix rules:

	      SFX Y Y 1
	      SFX Y 0 ly . ds:ly_adj

       In the dictionary:

	      ably st:able ds:ly_adj
	      able al:ably

       is:    Inflectional suffix(es).	All inflectional suffixes are  removed
	      by  stemming.   Morphological generation depends on the order of
	      the suffix fields.

	      feet st:foot is:plural

       ts:    Terminal suffix(es).  Terminal suffix  fields  are  inflectional
	      suffix fields "removed" by additional (not terminal) suffixes.

	      Useful  for  zero	 morphemes  and	 affixes  removed by splitting
	      rules.

	      work/D ts:present

	      SFX D Y 2
	      SFX D   0	ed . is:past_1
	      SFX D   0	ed . is:past_2

       Typical example of the terminal suffix is the zero morpheme of the nom-
       inative case.

       sp:    Surface  prefix.	Temporary  solution for	adding prefixes	to the
	      stems and	generated word forms. See tests/morph.*	example.

       pa:    Parts of the compound  words.  Output  fields  of	 morphological
	      analysis for stemming.

       dp:    Planned: derivational prefix.

       ip:    Planned: inflectional prefix.

       tp:    Planned: terminal	prefix.

Twofold	suffix stripping
       Ispell's	 original algorithm strips only	one suffix. Hunspell can strip
       another one yet (or a plus prefix in COMPLEXPREFIXES mode).

       The twofold suffix stripping is a significant improvement  in  handling
       of  immense  number  of	suffixes, that characterize agglutinative lan-
       guages.

       A second	`s' suffix (affix class	Y) will	be the continuation  class  of
       the suffix `able' in the	following example:

	       SFX Y Y 1
	       SFX Y 0 s .

	       SFX X Y 1
	       SFX X 0 able/Y .

       Dictionary file:

	       drink/X

       Test file:

	       drink
	       drinkable
	       drinkables

       Test:

	       $ hunspell -m -d	test <test.txt
	       drink st:drink
	       drinkable st:drink fl:X
	       drinkables st:drink fl:X	fl:Y

       Theoretically  with  the	twofold	suffix stripping needs only the	square
       root of the number of suffix rules, compared with a Hunspell  implemen-
       tation. In our practice,	we could have elaborated the Hungarian inflec-
       tional morphology with twofold suffix stripping.

Extended affix classes
       Hunspell	can handle more	than 65000 affix classes.  There are three new
       syntax for giving flags in affix	and dictionary files.

       FLAG long command sets 2-character flags:

		FLAG long
		SFX Y1 Y 1
		SFX Y1 0 s 1

       Dictionary record with the Y1, Z3, F? flags:

		foo/Y1Z3F?

       FLAG num	command	sets numerical flags separated by comma:

		FLAG num
		SFX 65000 Y 1
		SFX 65000 0 s 1

       Dictionary example:

		foo/65000,12,2756

       The third one is	the Unicode character flags.

Homonyms
       Hunspell's dictionary can contain repeating elements that are homonyms:

	       work/A	 po:verb
	       work/B	 po:noun

       An affix	file:

	       SFX A Y 1
	       SFX A 0 s . sf:sg3

	       SFX B Y 1
	       SFX B 0 s . is:plur

       Test file:

	       works

       Test:

	       $ hunspell -d test -m <testwords
	       work st:work po:verb is:sg3
	       work st:work po:noun is:plur

       This  feature also gives	a way to forbid	illegal	prefix/suffix combina-
       tions.

Prefix--suffix dependencies
       An interesting side-effect of multi-step	stripping is, that the	appro-
       priate  treatment  of circumfixes now comes for free.  For instance, in
       Hungarian, superlatives are formed by simultaneous prefixation of  leg-
       and  suffixation	of -bb to the adjective	base.  A problem with the one-
       level architecture is that there	is no way to render lexical  licensing
       of  particular  prefixes	and suffixes interdependent, and therefore in-
       correct forms are recognized as valid, i.e. *legv_A(C)n =	leg  +	v_A(C)n
       `old'.  Until  the introduction of clusters, a special treatment	of the
       superlative had to be hardwired in the earlier HunSpell code. This  may
       have  been legitimate for a single case,	but in fact prefix--suffix de-
       pendences are ubiquitous	 in  category-changing	derivational  patterns
       (cf.  English  payable, non-payable but *non-pay	or drinkable, undrink-
       able but	*undrink). In simple words, here, the prefix un- is legitimate
       only  if	 the  base drink is suffixed with -able. If both these patters
       are handled by on-line affix rules and affix rules are checked  against
       the  base only, there is	no way to express this dependency and the sys-
       tem will	necessarily over- or undergenerate.

       In next example,	suffix class R have got	a prefix `continuation'	 class
       (class P).

	      PFX P Y 1
	      PFX P   0	un . [prefix_un]+

	      SFX S Y 1
	      SFX S   0	s . +PL

	      SFX Q Y 1
	      SFX Q   0	s . +3SGV

	      SFX R Y 1
	      SFX R   0	able/PS	. +DER_V_ADJ_ABLE

       Dictionary:

	      2
	      drink/RQ	[verb]
	      drink/S	[noun]

       Morphological analysis:

	      >	drink
	      drink[verb]
	      drink[noun]
	      >	drinks
	      drink[verb]+3SGV
	      drink[noun]+PL
	      >	drinkable
	      drink[verb]+DER_V_ADJ_ABLE
	      >	drinkables
	      drink[verb]+DER_V_ADJ_ABLE+PL
	      >	undrinkable
	      [prefix_un]+drink[verb]+DER_V_ADJ_ABLE
	      >	undrinkables
	      [prefix_un]+drink[verb]+DER_V_ADJ_ABLE+PL
	      >	undrink
	      Unknown word.
	      >	undrinks
	      Unknown word.

Circumfix
       Conditional  affixes implemented	by a continuation class	are not	enough
       for circumfixes,	because	a circumfix is one  affix  in  morphology.  We
       also need CIRCUMFIX option for correct morphological analysis.

	      #	circumfixes: ~ obligate	prefix/suffix combinations
	      #	superlative in Hungarian: leg- (prefix)	AND -bb	(suffix)
	      #	nagy, nagyobb, legnagyobb, legeslegnagyobb
	      #	(great,	greater, greatest, most	greatest)

	      CIRCUMFIX	X

	      PFX A Y 1
	      PFX A 0 leg/X .

	      PFX B Y 1
	      PFX B 0 legesleg/X .

	      SFX C Y 3
	      SFX C 0 obb . +COMPARATIVE
	      SFX C 0 obb/AX . +SUPERLATIVE
	      SFX C 0 obb/BX . +SUPERSUPERLATIVE

       Dictionary:

	      1
	      nagy/C	[MN]

       Analysis:

	      >	nagy
	      nagy[MN]
	      >	nagyobb
	      nagy[MN]+COMPARATIVE
	      >	legnagyobb
	      nagy[MN]+SUPERLATIVE
	      >	legeslegnagyobb
	      nagy[MN]+SUPERSUPERLATIVE

Compounds
       Allowing	 free compounding yields decrease in precision of recognition,
       not to mention stemming and morphological analysis.   Although  lexical
       switches	are introduced to license compounding of bases by Ispell, this
       proves not to be	restrictive enough. For	example:

	      #	affix file
	      COMPOUNDFLAG X

	      2
	      foo/X
	      bar/X

       With this resource, foobar and barfoo also are accepted words.

       This has	been improved upon with	the introduction  of  direction-sensi-
       tive compounding, i.e., lexical features	can specify separately whether
       a base can occur	as leftmost or	rightmost  constituent	in  compounds.
       This,  however,	is still insufficient to handle	the intricate patterns
       of compounding, not to mention idiosyncratic  (and  language  specific)
       norms of	hyphenation.

       The  Hunspell  algorithm	 currently  allows  any	affixed	form of	words,
       which are lexically marked as potential members of compounds.  Hunspell
       improved	 this, and its recursive compound checking rules makes it pos-
       sible to	implement the intricate	spelling conventions of	Hungarian com-
       pounds. For example, using COMPOUNDWORDMAX, COMPOUNDSYLLABLE, COMPOUND-
       ROOT, SYLLABLENUM options can be	set  the  noteworthy  Hungarian	 `6-3'
       rule.   Further	example	 in  Hungarian,	derivate suffixes often	modify
       compounding properties. Hunspell	allows the compounding	flags  on  the
       affixes,	 and there are two special flags (COMPOUNDPERMITFLAG and (COM-
       POUNDFORBIDFLAG)	to permit or prohibit compounding of the derivations.

       Suffixes	with this flag forbid compounding of the affixed word.

       We also need several Hunspell features for handling German compounding:

	      #	German compounding

	      #	set language to	handle special casing of German	sharp s

	      LANG de_DE

	      #	compound flags

	      COMPOUNDBEGIN U
	      COMPOUNDMIDDLE V
	      COMPOUNDEND W

	      #	Prefixes are allowed at	the beginning of compounds,
	      #	suffixes are allowed at	the end	of compounds by	default:
	      #	(prefix)?(root)+(affix)?
	      #	Affixes	with COMPOUNDPERMITFLAG	may be inside of compounds.
	      COMPOUNDPERMITFLAG P

	      #	for German fogemorphemes (Fuge-element)
	      #	Hint: ONLYINCOMPOUND is	not required everywhere, but the
	      #	checking will be a little faster with it.

	      ONLYINCOMPOUND X

	      #	forbid uppercase characters at compound	word bounds
	      CHECKCOMPOUNDCASE

	      #	for handling Fuge-elements with	dashes (Arbeits-)
	      #	dash will be a special word

	      COMPOUNDMIN 1
	      WORDCHARS	-

	      #	compound settings and fogemorpheme for `Arbeit'

	      SFX A Y 3
	      SFX A 0 s/UPX .
	      SFX A 0 s/VPDX .
	      SFX A 0 0/WXD .

	      SFX B Y 2
	      SFX B 0 0/UPX .
	      SFX B 0 0/VWXDP .

	      #	a suffix for `Computer'

	      SFX C Y 1
	      SFX C 0 n/WD .

	      #	for forbid exceptions (*Arbeitsnehmer)

	      FORBIDDENWORD Z

	      #	dash prefix for	compounds with dash (Arbeits-Computer)

	      PFX - Y 1
	      PFX - 0 -/P .

	      #	decapitalizing prefix
	      #	circumfix for positioning in compounds

	      PFX D Y 29
	      PFX D A a/PX A
	      PFX D A Ax/PX A
	       .
	       .
	      PFX D Y y/PX Y
	      PFX D Z z/PX Z

       Example dictionary:

	      4
	      Arbeit/A-
	      Computer/BC-
	      -/W
	      Arbeitsnehmer/Z

       Accepted	compound compound words	with the previous resource:

	      Computer
	      Computern
	      Arbeit
	      Arbeits-
	      Computerarbeit
	      Computerarbeits-
	      Arbeitscomputer
	      Arbeitscomputern
	      Computerarbeitscomputer
	      Computerarbeitscomputern
	      Arbeitscomputerarbeit
	      Computerarbeits-Computer
	      Computerarbeits-Computern

       Not accepted compoundings:

	      computer
	      arbeit
	      Arbeits
	      arbeits
	      ComputerArbeit
	      ComputerArbeits
	      Arbeitcomputer
	      ArbeitsComputer
	      Computerarbeitcomputer
	      ComputerArbeitcomputer
	      ComputerArbeitscomputer
	      Arbeitscomputerarbeits
	      Computerarbeits-computer
	      Arbeitsnehmer

       This solution is	still not ideal, however, and will be  replaced	 by  a
       pattern-based  compound-checking	 algorithm which is closely integrated
       with input buffer tokenization. Patterns	describing compounds come as a
       separate	input resource that can	refer to high-level properties of con-
       stituent	parts (e.g. the	number of syllables, affix flags, and contain-
       ment  of	hyphens). The patterns are matched against potential segmenta-
       tions of	compounds to assess wellformedness.

Unicode	character encoding
       Both Ispell and Myspell use 8-bit ASCII character encoding, which is  a
       major  deficiency  when	it  comes to scalability.  Although a language
       like Hungarian has a standard ASCII  character  set  (ISO  8859-2),  it
       fails  to allow a full implementation of	Hungarian orthographic conven-
       tions.  For instance, the '--' symbol (n-dash)  is  missing  from  this
       character  set  contrary	 to  the fact that it is not only the official
       symbol to delimit parenthetic clauses in	the language, but it can be in
       compound	words as a special 'big' hyphen.

       MySpell	has  got  some	8-bit encoding tables, but there are languages
       without standard	8-bit encoding,	too. For example,  a  lot  of  African
       languages have non-latin	or extended latin characters.

       Similarly,  using  the  original	spelling of certain foreign names like
       _Angstr_A_paragraph_m or Moli_A"re is encouraged by	the Hungarian spelling
       norm,  and,  since  characters 'A' and 'A"' are not part	of ISO 8859-2,
       when they  combine  with	 inflections  containing  characters  only  in
       ISO 8859-2 (like	elative	-b_Al, allative -t_Al or delative	-r_Al with dou-
       ble acute), these  result  in  words  (like  _Angstr_A_paragraph_mr_Al  or
       Moli_A"re-t_Al.)  that can	not be encoded using any single	ASCII encoding
       scheme.

       The problems raised in relation to 8-bit	ASCII encoding have long  been
       recognized  by  proponents  of  Unicode.	It is clear that trading effi-
       ciency for encoding-independence	has its	advantages  when  it  comes  a
       truly multi-lingual application.	There is implemented a memory and time
       efficient Unicode handling in Hunspell. In non-UTF-8  character	encod-
       ings Hunspell works with	the original 8-bit strings. In UTF-8 encoding,
       affixes and words are stored in UTF-8, during the analysis are  handled
       in  mostly UTF-8, under condition checking and suggestion are converted
       to UTF-16. Unicode text analysis	and  spell  checking  have  a  minimal
       (0-20%) time overhead and minimal or reasonable memory overhead depends
       from the	language (its UTF-8 encoding and affixation).

Conversion of aspell dictionaries
       Aspell dictionaries can be easily converted into	 hunspell.  Conversion
       steps:

       dictionary (xx.cwl -> xx.wl):

       preunzip	xx.cwl
       wc -l < xx.wl > xx.dic
       cat xx.wl >> xx.dic

       affix file

       If the affix file exists, copy it:
       cp xx_affix.dat xx.aff
       If not, create it with the suitable character encoding (see xx.dat)
       echo "SET ISO8859-x" > xx.aff
       or
       echo "SET UTF-8"	> xx.aff

       It's  useful  to	add a TRY option with the characters of	the dictionary
       with frequency order to set edit	distance suggestions:
       echo "TRY qwertzuiopasdfghjklyxcvbnmQWERTZUIOPASDFGHJKLYXCVBNM" >>xx.aff

SEE ALSO
       hunspell	(1), ispell (1), ispell	(4)

				  2017-09-20			   hunspell(5)

NAME | DESCRIPTION | Dictionary file | Personal dictionary file | Short example | AFFIX FILE GENERAL OPTIONS | AFFIX FILE OPTIONS FOR SUGGESTION | OPTIONS FOR COMPOUNDING | AFFIX FILE OPTIONS FOR AFFIX CREATION | AFFIX FILE OTHER OPTIONS | Morphological analysis | Optional data fields | Twofold suffix stripping | Extended affix classes | Homonyms | Prefix--suffix dependencies | Circumfix | Compounds | Unicode character encoding | Conversion of aspell dictionaries | SEE ALSO

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=hunspell&sektion=5&manpath=FreeBSD+13.0-RELEASE+and+Ports>

home | help