Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
re_syntax(n)		     Tcl Built-In Commands		  re_syntax(n)


       re_syntax - Syntax of Tcl regular expressions

       A  regular  expression describes	strings	of characters.	It's a pattern
       that matches certain strings and	does not match others.

       Regular expressions ("RE"s), as defined by POSIX, come in two  flavors:
       extended	 REs  ("ERE"s) and basic REs ("BRE"s).	EREs are roughly those
       of the traditional egrep, while BREs are	roughly	those  of  the	tradi-
       tional  ed.  This  implementation  adds	a  third  flavor, advanced REs
       ("ARE"s), basically EREs	with some significant extensions.

       This manual page	primarily describes AREs. BREs mostly exist for	 back-
       ward  compatibility in some old programs; they will be discussed	at the
       end. POSIX EREs are almost an exact subset of AREs.  Features  of  AREs
       that are	not present in EREs will be indicated.

       Tcl  regular  expressions  are implemented using	the package written by
       Henry Spencer, based on the 1003.2 spec and some	(not quite all)	of the
       Perl5  extensions  (thanks, Henry!). Much of the	description of regular
       expressions below is copied verbatim from his manual entry.

       An ARE is one or	more branches, separated  by  "|",  matching  anything
       that matches any	of the branches.

       A branch	is zero	or more	constraints or quantified atoms, concatenated.
       It matches a match for the first, followed by a match for  the  second,
       etc; an empty branch matches the	empty string.

       A  quantified atom is an	atom possibly followed by a single quantifier.
       Without a quantifier, it	matches	a single  match	 for  the  atom.   The
       quantifiers, and	what a so-quantified atom matches, are:

	 *     a sequence of 0 or more matches of the atom

	 +     a sequence of 1 or more matches of the atom

	 ?     a sequence of 0 or 1 matches of the atom

	 {m}   a sequence of exactly m matches of the atom

	 {m,}  a sequence of m or more matches of the atom

	 {m,n} a  sequence  of	m through n (inclusive)	matches	of the atom; m
	       may not exceed n

	 *?  +?	 ??  {m}?  {m,}?  {m,n}?
	       non-greedy quantifiers, which match the same possibilities, but
	       prefer  the  smallest  number rather than the largest number of
	       matches (see MATCHING)

       The forms using { and } are known as bounds. The	numbers	m  and	n  are
       unsigned	 decimal integers with permissible values from 0 to 255	inclu-

       An atom is one of:

	 (re)  matches a match for re (re is any regular expression) with  the
	       match noted for possible	reporting

	       as  previous,  but  does	no reporting (a	"non-capturing"	set of

	 ()    matches an empty	string,	noted for possible reporting

	 (?:)  matches an empty	string,	without	reporting

	       a bracket expression,  matching	any  one  of  the  chars  (see
	       BRACKET EXPRESSIONS for more detail)

	 .     matches any single character

	 \k    matches	the  non-alphanumeric character	k taken	as an ordinary
	       character, e.g. \\ matches a backslash character

	 \c    where c is alphanumeric (possibly  followed  by	other  charac-
	       ters), an escape	(AREs only), see ESCAPES below

	 {     when  followed  by  a character other than a digit, matches the
	       left-brace character "{"; when followed by a digit, it  is  the
	       beginning of a bound (see above)

	 x     where  x	 is  a	single	character  with	no other significance,
	       matches that character.

       A constraint matches an empty string when specific conditions are  met.
       A  constraint  may  not	be  followed  by a quantifier. The simple con-
       straints	are as follows;	some more constraints are described later, un-
       der ESCAPES.

	 ^	 matches  at  the beginning of the string or a line (according
		 to whether matching is	newline-sensitive or not, as described
		 in MATCHING, below).

	 $	 matches  at  the  end	of  the	string or a line (according to
		 whether matching is newline-sensitive or not, as described in
		 MATCHING, below).

		 The  difference between string	and line matching modes	is im-
		 material when the string does not contain a  newline  charac-
		 ter.  The \A and \Z constraint	escapes	have a similar purpose
		 but are always	constraints for	the overall string.

		 The default newline-sensitivity depends on the	 command  that
		 uses  the  regular  expression,  and can be overridden	as de-
		 scribed in METASYNTAX,	below.

	 (?=re)	 positive lookahead (AREs only), matches at any	point where  a
		 substring matching re begins

	 (?!re)	 negative lookahead (AREs only), matches at any	point where no
		 substring matching re begins

       The lookahead constraints may not contain back references (see  later),
       and all parentheses within them are considered non-capturing.

       An RE may not end with "\".

       A bracket expression is a list of characters enclosed in	"[]".  It nor-
       mally matches any single	character from the list	(but  see  below).  If
       the  list begins	with "^", it matches any single	character (but see be-
       low) not	from the rest of the list.

       If two characters in the	list are separated by "-", this	 is  shorthand
       for  the	 full range of characters between those	two (inclusive)	in the
       collating sequence, e.g.	 "[0-9]" in Unicode matches  any  conventional
       decimal	digit.	Two ranges may not share an endpoint, so e.g.  "a-c-e"
       is illegal. Ranges in Tcl always	use the	 Unicode  collating  sequence,
       but  other programs may use other collating sequences and this can be a
       source of incompatibility between programs.

       To include a literal ] or - in the list,	the simplest method is to  en-
       close  it  in [.	and .] to make it a collating element (see below). Al-
       ternatively, make it the	first character	(following a possible "^"), or
       (AREs  only)  precede it	with "\".  Alternatively, for "-", make	it the
       last character, or the second endpoint of a range. To use a  literal  -
       as  the first endpoint of a range, make it a collating element or (AREs
       only) precede it	with "\".  With	the exception of these,	some  combina-
       tions  using  [	(see  next paragraphs),	and escapes, all other special
       characters lose their special significance within a bracket expression.

       Within a	bracket	expression, the	name of	a character class enclosed  in
       [: and :] stands	for the	list of	all characters (not all	collating ele-
       ments!) belonging to that class.	 Standard character classes are:

       alpha   A letter.

       upper   An upper-case letter.

       lower   A lower-case letter.

       digit   A decimal digit.

       xdigit  A hexadecimal digit.

       alnum   An alphanumeric (letter or digit).

       print   A "printable" (same as graph, except also including space).

       blank   A space or tab character.

       space   A character producing white space in displayed text.

       punct   A punctuation character.

       graph   A character with	a visible representation (includes both	 alnum
	       and punct).

       cntrl   A control character.

       A  locale  may  provide others. A character class may not be used as an
       endpoint	of a range.

	      (Note: the current Tcl implementation has	only one  locale,  the
	      Unicode locale, which supports exactly the above classes.)

       There are two special cases of bracket expressions: the bracket expres-
       sions "[[:<:]]" and "[[:>:]]" are constraints, matching	empty  strings
       at  the beginning and end of a word respectively.  A word is defined as
       a sequence of word characters that is neither preceded nor followed  by
       word  characters.  A  word character is an alnum	character or an	under-
       score ("_").  These special bracket expressions are  deprecated;	 users
       of AREs should use constraint escapes instead (see below).

       Within a	bracket	expression, a collating	element	(a character, a	multi-
       character sequence that collates	as if it were a	single character, or a
       collating-sequence  name	 for  either) enclosed in [. and .] stands for
       the sequence of characters of that collating element. The sequence is a
       single  element	of the bracket expression's list. A bracket expression
       in a locale that	has multi-character collating elements can thus	 match
       more  than  one	character. So (insidiously), a bracket expression that
       starts with ^ can match multi-character collating elements even if none
       of them appear in the bracket expression!

	      (Note:  Tcl  has no multi-character collating elements. This in-
	      formation	is only	for illustration.)

       For example, assume the collating sequence includes a ch	 multi-charac-
       ter  collating  element.	 Then  the RE "[[.ch.]]*c" (zero or more "chs"
       followed	by "c")	matches	the first five characters of "chchcc".	 Also,
       the  RE "[^c]b" matches all of "chb" (because "[^c]" matches the	multi-
       character "ch").

       Within a	bracket	expression, a collating	element	enclosed in [= and  =]
       is  an  equivalence  class, standing for	the sequences of characters of
       all collating elements equivalent to that one,  including  itself.  (If
       there  are  no other equivalent collating elements, the treatment is as
       if the enclosing	delimiters were	"[." and ".]".)	 For example, if o and
       ^o  are  the members of an equivalence class, then "[[=o=]]", "[[=^o=]]",
       and "[o^o]" are all synonymous. An equivalence class may not be an  end-
       point of	a range.

	      (Note:  Tcl  implements only the Unicode locale. It does not de-
	      fine any equivalence classes. The	examples above are just	illus-

       Escapes	(AREs  only), which begin with a \ followed by an alphanumeric
       character, come in several varieties:  character	 entry,	 class	short-
       hands,  constraint escapes, and back references.	A \ followed by	an al-
       phanumeric character but	not constituting a valid escape	is illegal  in
       AREs.  In EREs, there are no escapes: outside a bracket expression, a \
       followed	by an alphanumeric character merely stands for that  character
       as  an ordinary character, and inside a bracket expression, \ is	an or-
       dinary character. (The latter is	the one	actual incompatibility between
       EREs and	AREs.)

       Character-entry	escapes	(AREs only) exist to make it easier to specify
       non-printing and	otherwise inconvenient characters in REs:

	 \a   alert (bell) character, as in C

	 \b   backspace, as in C

	 \B   synonym for \ to help reduce backslash doubling in some applica-
	      tions where there	are multiple levels of backslash processing

	 \cX  (where  X	is any character) the character	whose low-order	5 bits
	      are the same as those of X, and whose other bits are all zero

	 \e   the character whose collating-sequence name is "ESC", or failing
	      that, the	character with octal value 033

	 \f   formfeed,	as in C

	 \n   newline, as in C

	 \r   carriage return, as in C

	 \t   horizontal tab, as in C

	      (where  wxyz  is	one up to four hexadecimal digits) the Unicode
	      character	U+wxyz in the local byte ordering

	      (where stuvwxyz is one up	to eight hexadecimal digits)  reserved
	      for a Unicode extension up to 21 bits. The digits	are parsed un-
	      til the first non-hexadecimal character is encountered, the max-
	      imun  of	eight  hexadecimal  digits are reached,	or an overflow
	      would occur in the maximum value of U+10ffff.

	 \v   vertical tab, as in C are	all available.

	 \xhh (where hh	is one or two hexadecimal digits) the character	 whose
	      hexadecimal value	is 0xhh.

	 \0   the character whose value	is 0

	 \xyz (where xyz is exactly three octal	digits,	and is not a back ref-
	      erence (see below)) the character	whose octal value is 0xyz. The
	      first  digit  must  be in	the range 0-3, otherwise the two-digit
	      form is assumed.

	 \xy  (where xy	is exactly two octal digits, and is not	a back	refer-
	      ence (see	below))	the character whose octal value	is 0xy

       Hexadecimal digits are "0"-"9", "a"-"f",	and "A"-"F".  Octal digits are

       The character-entry escapes are always taken  as	 ordinary  characters.
       For  example,  \135  is	]  in  Unicode,	 but \135 does not terminate a
       bracket expression. Beware, however, that some  applications  (e.g.,  C
       compilers  and  the  Tcl	 interpreter  if the regular expression	is not
       quoted with braces) interpret such sequences themselves before the reg-
       ular-expression	package	 gets  to see them, which may require doubling
       (quadrupling, etc.) the "\".

       Class-shorthand escapes (AREs only) provide shorthands for certain com-
       monly-used character classes:

	 \d	   [[:digit:]]

	 \s	   [[:space:]]

	 \w	   [[:al-
		   (including punctuation connector characters)

	 \D	   [^[:digit:]]

	 \S	   [^[:space:]]

	 \W	   [^[:al-
		   (including punctuation connector characters)

       Within  bracket	expressions,  "\d",  "\s",  and	 "\w" lose their outer
       brackets, and "\D", "\S", and "\W" are illegal. (So, for	example,  "[a-
       c\d]"  is  equivalent  to  "[a-c[:digit:]]".  Also, "[a-c\D]", which is
       equivalent to "[a-c^[:digit:]]",	is illegal.)

       A constraint escape (AREs only) is a  constraint,  matching  the	 empty
       string if specific conditions are met, written as an escape:

	 \A    matches	only at	the beginning of the string (see MATCHING, be-
	       low, for	how this differs from "^")

	 \m    matches only at the beginning of	a word

	 \M    matches only at the end of a word

	 \y    matches only at the beginning or	end of a word

	 \Y    matches only at a point that is not the beginning or end	 of  a

	 \Z    matches only at the end of the string (see MATCHING, below, for
	       how this	differs	from "$")

	 \m    (where m	is a nonzero digit) a back reference, see below

	 \mnn  (where m	is a nonzero digit, and	nn is some  more  digits,  and
	       the decimal value mnn is	not greater than the number of closing
	       capturing parentheses seen so far) a back reference, see	below

       A word is defined as in the specification of  "[[:<:]]"	and  "[[:>:]]"
       above. Constraint escapes are illegal within bracket expressions.

       A  back	reference  (AREs  only)	matches	the same string	matched	by the
       parenthesized subexpression specified by	the  number,  so  that	(e.g.)
       "([bc])\1"  matches  "bb" or "cc" but not "bc".	The subexpression must
       entirely	precede	the back reference in the RE.  Subexpressions are num-
       bered  in the order of their leading parentheses.  Non-capturing	paren-
       theses do not define subexpressions.

       There is	an inherent historical ambiguity between octal character-entry
       escapes and back	references, which is resolved by heuristics, as	hinted
       at above. A leading zero	always indicates an  octal  escape.  A	single
       non-zero	 digit,	 not  followed	by another digit, is always taken as a
       back reference. A multi-digit sequence not  starting  with  a  zero  is
       taken  as  a  back reference if it comes	after a	suitable subexpression
       (i.e. the number	is in the legal	range for a back reference), and  oth-
       erwise is taken as octal.

       In  addition to the main	syntax described above,	there are some special
       forms and miscellaneous syntactic facilities available.

       Normally	the flavor of RE being used is specified by application-depen-
       dent  means. However, this can be overridden by a director. If an RE of
       any flavor begins with "***:", the rest of the RE is an ARE. If	an  RE
       of  any	flavor begins with "***=", the rest of the RE is taken to be a
       literal string, with all	characters considered ordinary characters.

       An ARE may begin	with embedded options: a sequence (?xyz) (where	xyz is
       one or more alphabetic characters) specifies options affecting the rest
       of the RE. These	supplement, and	can override, any options specified by
       the application.	The available option letters are:

	 b  rest of RE is a BRE

	 c  case-sensitive matching (usual default)

	 e  rest of RE is an ERE

	 i  case-insensitive matching (see MATCHING, below)

	 m  historical synonym for n

	 n  newline-sensitive matching (see MATCHING, below)

	 p  partial newline-sensitive matching (see MATCHING, below)

	 q  rest of RE is a literal ("quoted") string, all ordinary characters

	 s  non-newline-sensitive matching (usual default)

	 t  tight syntax (usual	default; see below)

	 w  inverse  partial  newline-sensitive	("weird") matching (see	MATCH-
	    ING, below)

	 x  expanded syntax (see below)

       Embedded	options	take effect at the ) terminating the  sequence.	  They
       are  available  only  at	the start of an	ARE, and may not be used later
       within it.

       In addition to the usual	(tight)	RE syntax, in which all	characters are
       significant,  there  is an expanded syntax, available in	all flavors of
       RE with the -expanded switch, or	in AREs	with the embedded x option. In
       the expanded syntax, white-space	characters are ignored and all charac-
       ters between a #	and the	following newline (or the end of the  RE)  are
       ignored,	permitting paragraphing	and commenting a complex RE. There are
       three exceptions	to that	basic rule:

       o  a white-space	character or "#" preceded by "\" is retained

       o  white	space or "#" within a bracket expression is retained

       o  white	space and comments are illegal within multi-character  symbols
	  like the ARE "(?:" or	the BRE	"\("

       Expanded-syntax white-space characters are blank, tab, newline, and any
       character that belongs to the space character class.

       Finally,	in an ARE, outside bracket expressions,	the sequence "(?#ttt)"
       (where  ttt  is any text	not containing a ")") is a comment, completely
       ignored.	Again, this is not allowed between the	characters  of	multi-
       character  symbols like "(?:".  Such comments are more a	historical ar-
       tifact than a useful facility, and their	use is deprecated; use the ex-
       panded syntax instead.

       None of these metasyntax	extensions is available	if the application (or
       an initial "***=" director) has specified  that	the  user's  input  be
       treated as a literal string rather than as an RE.

       In  the event that an RE	could match more than one substring of a given
       string, the RE matches the one starting earliest	in the string. If  the
       RE  could  match	 more  than  one substring starting at that point, its
       choice is determined by its preference: either the  longest  substring,
       or the shortest.

       Most atoms, and all constraints,	have no	preference. A parenthesized RE
       has the same preference (possibly none) as the RE.  A  quantified  atom
       with  quantifier	{m} or {m}? has	the same preference (possibly none) as
       the atom	itself.	A quantified atom with other normal  quantifiers  (in-
       cluding	{m,n}  with  m equal to	n) prefers longest match. A quantified
       atom with other non-greedy quantifiers (including {m,n}?	 with m	 equal
       to  n)  prefers shortest	match. A branch	has the	same preference	as the
       first quantified	atom in	it which has a preference. An RE consisting of
       two or more branches connected by the | operator	prefers	longest	match.

       Subject	to the constraints imposed by the rules	for matching the whole
       RE, subexpressions also match the longest  or  shortest	possible  sub-
       strings,	 based on their	preferences, with subexpressions starting ear-
       lier in the RE taking priority over  ones  starting  later.  Note  that
       outer subexpressions thus take priority over their component subexpres-

       The quantifiers {1,1} and {1,1}?	can  be	 used  to  force  longest  and
       shortest	preference, respectively, on a subexpression or	a whole	RE.

	      NOTE:  This  means  that you can usually make a RE be non-greedy
	      overall by putting {1,1}?	after one of the first	non-constraint
	      atoms or parenthesized sub-expressions in	it. It pays to experi-
	      ment with	the placing of this non-greediness override on a suit-
	      able  range  of input texts when you are writing a RE if you are
	      using this level of complexity.

	      For example, this	regular	expression  is	non-greedy,  and  will
	      match  the  shortest substring possible given that "abc" will be
	      matched as early as possible (the	 quantifier  does  not	change


	      The  atom	 "a"  has no greediness	preference, we explicitly give
	      one for "b", and the remaining quantifiers are overridden	to  be
	      non-greedy by the	preceding non-greedy quantifier.

       Match  lengths  are  measured in	characters, not	collating elements. An
       empty string is considered longer than no match at  all.	 For  example,
       "bb*"	matches	   the	  three	   middle   characters	 of   "abbbc",
       "(week|wee)(night|knights)" matches all ten characters of "weeknights",
       when "(.*).*"  is matched against "abc" the parenthesized subexpression
       matches all three characters, and when "(a*)*" is matched against  "bc"
       both  the  whole	 RE and	the parenthesized subexpression	match an empty

       If case-independent matching is specified, the effect is	much as	if all
       case  distinctions  had	vanished from the alphabet. When an alphabetic
       that exists in multiple cases appears as	an ordinary character  outside
       a  bracket expression, it is effectively	transformed into a bracket ex-
       pression	containing both	cases, so that x becomes "[xX]".  When it  ap-
       pears  inside  a	 bracket  expression,  all case	counterparts of	it are
       added to	the bracket expression,	 so  that  "[x]"  becomes  "[xX]"  and
       "[^x]" becomes "[^xX]".

       If  newline-sensitive  matching is specified, . and bracket expressions
       using ^ will never match	the newline character (so  that	 matches  will
       never  cross newlines unless the	RE explicitly arranges it) and ^ and $
       will match the empty string after and before a newline respectively, in
       addition	 to  matching at beginning and end of string respectively. ARE
       \A and \Z continue to match beginning or	end of string only.

       If partial newline-sensitive matching is	specified, this	affects	.  and
       bracket	expressions  as	with newline-sensitive matching, but not ^ and

       If inverse partial newline-sensitive matching is	 specified,  this  af-
       fects ^ and $ as	with newline-sensitive matching, but not . and bracket
       expressions. This is not	very useful but	is provided for	symmetry.

       No particular limit is imposed on the length of REs. Programs  intended
       to be highly portable should not	employ REs longer than 256 bytes, as a
       POSIX-compliant implementation can refuse to accept such	REs.

       The only	feature	of AREs	that is	actually incompatible with POSIX  EREs
       is that \ does not lose its special significance	inside bracket expres-
       sions. All other	ARE features use syntax	which is illegal or has	 unde-
       fined or	unspecified effects in POSIX EREs; the *** syntax of directors
       likewise	is outside the POSIX syntax for	both BREs and EREs.

       Many of the ARE extensions are borrowed from Perl, but some  have  been
       changed	to  clean  them	up, and	a few Perl extensions are not present.
       Incompatibilities of note include  "\b",	 "\B",	the  lack  of  special
       treatment  for a	trailing newline, the addition of complemented bracket
       expressions to the things affected by newline-sensitive	matching,  the
       restrictions  on	 parentheses  and  back	 references  in	lookahead con-
       straints, and  the  longest/shortest-match  (rather  than  first-match)
       matching	semantics.

       The  matching rules for REs containing both normal and non-greedy quan-
       tifiers have changed since early	beta-test versions  of	this  package.
       (The new	rules are much simpler and cleaner, but	do not work as hard at
       guessing	the user's real	intentions.)

       Henry Spencer's original	1986 regexp package, still in  widespread  use
       (e.g., in pre-8.1 releases of Tcl), implemented an early	version	of to-
       day's EREs. There are four incompatibilities between regexp's near-EREs
       ("RREs"	for  short)  and AREs. In roughly increasing order of signifi-

       o  In AREs, \ followed by an alphanumeric character is either an	escape
	  or  an  error, while in RREs,	it was just another way	of writing the
	  alphanumeric.	This should not	be a problem because there was no rea-
	  son to write such a sequence in RREs.

       o  {  followed  by a digit in an	ARE is the beginning of	a bound, while
	  in RREs, { was always	an ordinary character. Such  sequences	should
	  be rare, and will often result in an error because following charac-
	  ters will not	look like a valid bound.

       o  In AREs, \ remains a special character within	"[]", so a  literal  \
	  within [] must be written "\\".  \\ also gives a literal \ within []
	  in RREs, but only truly paranoid programmers routinely  doubled  the

       o  AREs	report	the longest/shortest match for the RE, rather than the
	  first	found in a specified search order. This	may affect  some  RREs
	  which	 were written in the expectation that the first	match would be
	  reported. (The careful crafting of RREs to optimize the search order
	  for  fast matching is	obsolete (AREs examine all possible matches in
	  parallel, and	their performance is largely insensitive to their com-
	  plexity)  but	cases where the	search order was exploited to deliber-
	  ately	find a match which was	not  the  longest/shortest  will  need

       BREs  differ  from EREs in several respects.  "|", "+", and ? are ordi-
       nary characters and there is no equivalent for their functionality. The
       delimiters for bounds are \{ and	"\}", with { and } by themselves ordi-
       nary characters.	The parentheses	for nested subexpressions are  \(  and
       "\)",  with ( and ) by themselves ordinary characters. ^	is an ordinary
       character except	at the beginning of the	 RE  or	 the  beginning	 of  a
       parenthesized  subexpression,  $	is an ordinary character except	at the
       end of the RE or	the end	of a parenthesized subexpression, and *	is  an
       ordinary	 character if it appears at the	beginning of the RE or the be-
       ginning of a parenthesized  subexpression  (after  a  possible  leading
       "^").   Finally,	single-digit back references are available, and	\< and
       \> are synonyms for "[[:<:]]" and "[[:>:]]" respectively; no other  es-
       capes are available.

       RegExp(3), regexp(n), regsub(n),	lsearch(n), switch(n), text(n)

       match, regular expression, string

Tcl				      8.1			  re_syntax(n)


Want to link to this manual page? Use this URL:

home | help