Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
YAZ-ICU(1)			   Commands			    YAZ-ICU(1)

NAME
       yaz-icu - YAZ ICU utility

SYNOPSIS
       yaz-icu [-c config] [-p opt] [-s] [-x] [infile]

DESCRIPTION
       yaz-icu is a utility which demonstrates the ICU chain module of yaz.
       (yaz/icu.h).

       The utility can be used in two ways. It may read	some text using	an XML
       configuration for configuring ICU and show text analysis. This mode is
       triggered by option -c which specifies the configuration	to be used.
       The input file is read from standard input or from a file if infile is
       specified.

       The utility may also show ICU information. This is triggered by option
       -p.

OPTIONS
       -c config
	   Specifies the file containing ICU chain configuration which is XML
	   based.

       -p type
	   Specifies extra information to be printed about the ICU system. If
	   type	is c then ICU converters are printed. If type is l, then
	   available locales are printed. If type is t,	then available
	   transliterators are printed.

       -s
	   Specifies that output should	include	sort key as well. Note that
	   sort	key differs between ICU	versions.

       -x
	   Specifies that output should	be XML based rather than "text"	based.

ICU CHAIN CONFIGURATION
       The ICU chain configuration specifies one or more rules to convert text
       data into tokens. The configuration format is XML based.

       The toplevel element must be named icu_chain. The icu_chain element has
       one required attribute locale which specifies the ICU locale to be used
       in the conversion steps.

       The icu_chain element must include elements where each element
       specifies a conversion step. The	conversion is performed	in the order
       in which	the conversion steps are specified. Each conversion element
       takes one attribute: rule which serves as argument to the conversion
       step.

       The following conversion	elements are available:

       casemap
	   Converts case (and rule specifies how):

	   l
	       Lower case using	ICU function u_strToLower.

	   u
	       Upper case using	ICU function u_strToUpper.

	   t
	       To title	using ICU function u_strToTitle.

	   f
	       Fold case using ICU function u_strFoldCase.

       display
	   This	is a meta step which specifies that a term/token is to be
	   displayed. This term	is retrieved in	an application using function
	   icu_chain_token_display (yaz/icu.h).

       transform
	   Specifies an	ICU transform rule using a transliterator Identifier.
	   The rule attribute is the transliterator Identifier.	See ICU
	   Transforms[1] for more information.

       transliterate
	   Specifies a rule-based transliterator. The rule attribute is	the
	   custom transformation rule to be used. See ICU Transforms[1]	for
	   more	information.

       tokenize
	   Breaks / tokenizes a	string into components using ICU functions
	   ubrk_open, ubrk_setText, .. . The rule is one of:

	   l
	       Line. ICU: UBRK_LINE.

	   s
	       Sentence. ICU: UBRK_SENTENCE.

	   w
	       Word. ICU: UBRK_WORD.

	   c
	       Character. ICU: UBRK_CHARACTER.

	   t
	       Title. ICU: UBRK_TITLE.

       join
	   Joins tokens	into one string. The rule attribute is the joining
	   string, which may be	empty. The join	conversion element was added
	   in YAZ 4.2.49.

EXAMPLES
       The following command analyzes text in file text	using ICU chain
       configuration chain.xml:

	       cat text	| yaz-icu -c chain.xml

       The chain.xml might look	as follows:

	   <icu_chain locale="en">
	     <transform	rule="[:Control:] Any-Remove"/>
	     <tokenize rule="w"/>
	     <transform	rule="[[:WhiteSpace:][:Punctuation:]] Remove"/>
	     <transliterate rule="xy > z;"/>
	     <display/>
	     <casemap rule="l"/>
	   </icu_chain>

SEE ALSO
       yaz(7)

       ICU Home[2]

       ICU Transforms[1]

AUTHORS
       Index Data

NOTES
	1. ICU Transforms
	   http://userguide.icu-project.org/transforms/general

	2. ICU Home
	   http://www.icu-project.org/

YAZ 5.30.3			  06/10/2020			    YAZ-ICU(1)

NAME | SYNOPSIS | DESCRIPTION | OPTIONS | ICU CHAIN CONFIGURATION | EXAMPLES | SEE ALSO | AUTHORS | NOTES

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=yaz-icu&sektion=1&manpath=FreeBSD+13.0-RELEASE+and+Ports>

home | help