Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help

       ctags-client-tools  -  Hints  for developing a tool using ctags command
       and tags	output

       ctags [options] [file(s)]
       etags [options] [file(s)]

       Client tool means a tool	running	the ctags  command  and/or  reading  a
       tags  file generated by ctags command.  This man	page gathers hints for
       people who develop client tools.

       Pseudo-tags, stored in a	tag file, indicate  how	 ctags	generated  the
       tags  file:  whether  the  tags file is sorted or not, which version of
       tags file format	is used, the name of tags generator, and  so  on.  The
       opposite	 term  for pseudo-tags is regular-tags.	A regular-tag is for a
       language	object in an input file. A pseudo-tag is for the tags file it-
       self. Client tools may use pseudo-tags as reference for processing reg-

       A pseudo-tag is stored in a tags	file  in  the  same  format  as	 regu-
       lar-tags	as described in	tags(5), except	that pseudo-tag	names are pre-
       fixed with "!_".	For the	general	 information  about  pseudo-tags,  see
       "TAG FILE INFORMATION" in tags(5).

       An example of a pseudo tag:

	  !_TAG_PROGRAM_NAME	  Universal Ctags /Derived from	Exuberant Ctags/

       The  value,  "2", associated with the pseudo tag	"TAG_PROGRAM_NAME", is
       used in the field for input file. The description, "Derived  from  Exu-
       berant Ctags", is used in the field for pattern.

       Universal  Ctags	extends	the naming scheme of the classical pseudo-tags
       available in Exuberant Ctags for	emitting language specific information
       as pseudo tags:

	  !_{pseudo-tag-name}!{language-name}	  {associated-value}	  /{description}/

       The  language-name is appended to the pseudo-tag	name with a separator,

       An example of pseudo tag	with a language	suffix:

	  !_TAG_KIND_DESCRIPTION!C	  f,function	  /function definitions/

       This pseudo-tag says "the function kind of C language is	 enabled  when
       generating  this	 tags  file."  --pseudo-tags  is  the  option  for en-
       abling/disabling	 individual  pseudo-tags.  When	 enabling/disabling  a
       pseudo  tag  with  the  option, specify the tag name only "TAG_KIND_DE-
       SCRIPTION", without the prefix ("!_") or	the suffix ("!C").

   Options for Pseudo-tags
       --extras=+p (or --extras=+{pseudo})
	      Forces writing pseudo-tags.

	      ctags emits pseudo-tags by default when writing tags to a	 regu-
	      lar  file	 (e.g.	"tags'.) However, when specifying -o - or -f -
	      for  writing  tags  to  standard	output,	 ctags	doesn't	  emit
	      pseudo-tags.   --extras=+p   or  --extras=+{pseudo}  will	 force
	      pseudo-tags to be	written.

	      Lists available types of pseudo-tags and shows whether they  are
	      enabled or disabled.

	      Running  ctags  with  --list-pseudo-tags	option lists available
	      pseudo-tags. Some	of pseudo-tags newly introduced	 in  Universal
	      Ctags  project are disabled by default. Use --pseudo-tags=... to
	      enable them.

	      Specifies	a list of pseudo-tag types to include in the output.

	      The parameters are a set of pseudo tag names. Valid  pseudo  tag
	      names  can be listed with	--list-pseudo-tags. Surround each name
	      in the set with braces, like "{TAG_PROGRAM_AUTHOR}".  You	 don't
	      have  to	include	 the  "!_" pseudo tag prefix when specifying a
	      name in the option argument for --pseudo-tags= option.

	      pseudo-tags don't	have a notation	using one-letter flags.

	      If a name	is preceded by either the '+' or '-' characters,  that
	      tags's effect has	been added or removed. Otherwise the names re-
	      place any	current	settings. All entries are included if  '*'  is

       --fields=+E (or --fields=+{extras})
	      Attach "extras:pseudo" field to pseudo-tags.

	      An example of pseudo tags	with the field:

		 !_TAG_PROGRAM_NAME	 Universal Ctags /Derived from Exuberant Ctags/	 extras:pseudo

	      If  the  name  of	a normal tag in	a tag file starts with "!_", a
	      client tool cannot distinguish whether the tag is	a  regular-tag
	      or  pseudo-tag.	The  fields attached with this option help the
	      tool distinguish them.

   List	of notable pseudo-tags
       Running ctags with --list-pseudo-tags option lists available  types  of
       pseudo-tags  with  short	 descriptions. This subsection shows hints for
       using notable ones.

       TAG_EXTRA_DESCRIPTION (new in Universal Ctags)
	      Indicates	the names and descriptions of enabled extras:

		 !_TAG_EXTRA_DESCRIPTION       {extra-name}    /description/
		 !_TAG_EXTRA_DESCRIPTION!{language-name}       {extra-name}    /description/

	      If your tool relies on some extra	tags (extras),	refer  to  the
	      pseudo-tags  of  this type. A tool can reject the	tags file that
	      doesn't include expected extras, and raise an error in an	 early
	      stage of processing.

	      An example of the	pseudo-tags:

		 $ ctags --extras=+p --pseudo-tags='{TAG_EXTRA_DESCRIPTION}' -o	- input.c
		 !_TAG_EXTRA_DESCRIPTION       anonymous       /Include	tags for non-named objects like	lambda/
		 !_TAG_EXTRA_DESCRIPTION       fileScope       /Include	tags of	file scope/
		 !_TAG_EXTRA_DESCRIPTION       pseudo  /Include	pseudo tags/
		 !_TAG_EXTRA_DESCRIPTION       subparser       /Include	tags generated by subparsers/

	      A	client tool can	know "{anonymous}", "{fileScope}", "{pseudo}",
	      and "{subparser}"	extras are enabled from	the output.

       TAG_FIELD_DESCRIPTION (new in Universal Ctags)
	      Indicates	the names and descriptions of enabled fields:

		 !_TAG_FIELD_DESCRIPTION       {field-name}    /description/
		 !_TAG_FIELD_DESCRIPTION!{language-name}       {field-name}    /description/

	      If your tool relies on some fields, refer	to the pseudo-tags  of
	      this  type.   A tool can reject a	tags file that doesn't include
	      expected fields, and raise an error in an	early  stage  of  pro-

	      An example of the	pseudo-tags:

		 $ ctags --fields-C=+'{macrodef}' --extras=+p --pseudo-tags='{TAG_FIELD_DESCRIPTION}' -o - input.c
		 !_TAG_FIELD_DESCRIPTION       file    /File-restricted	scoping/
		 !_TAG_FIELD_DESCRIPTION       input   /input file/
		 !_TAG_FIELD_DESCRIPTION       name    /tag name/
		 !_TAG_FIELD_DESCRIPTION       pattern /pattern/
		 !_TAG_FIELD_DESCRIPTION       typeref /Type and name of a variable or typedef/
		 !_TAG_FIELD_DESCRIPTION!C     macrodef	       /macro definition/

	      A	 client	 tool  can  know "{file}", "{input}", "{name}",	"{pat-
	      tern}", and "{typeref}" fields are enabled from the output.  The
	      fields  are  common  in  languages.  In  addition	 to the	common
	      fields, the tool can known "{macrodef}" field of C  language  is
	      also enabled.

       TAG_FILE_ENCODING (new in Universal Ctags)

	      See also tags(5).

	      See also tags(5).

       TAG_KIND_DESCRIPTION (new in Universal Ctags)
	      Indicates	the names and descriptions of enabled kinds:

		 !_TAG_KIND_DESCRIPTION!{language-name}	       {kind-letter},{kind-name}       /description/

	      If  your	tool relies on some kinds, refer to the	pseudo-tags of
	      this type.  A tool can reject the	tags file that doesn't include
	      expected kinds, and raise	an error in an early stage of process-

	      Kinds are	language specific, so a	language name is   always  ap-
	      pended to	the tag	name as	suffix.

	      An example of the	pseudo-tags:

		 $ ctags --extras=+p --kinds-C=vfm --pseudo-tags='{TAG_KIND_DESCRIPTION}' -o - input.c
		 !_TAG_KIND_DESCRIPTION!C      f,function      /function definitions/
		 !_TAG_KIND_DESCRIPTION!C      m,member	       /struct,	and union members/
		 !_TAG_KIND_DESCRIPTION!C      v,variable      /variable definitions/

	      A	 client	 tool  can  know "{function}", "{member}", and "{vari-
	      able}" kinds of C	language are enabled from the output.

       TAG_KIND_SEPARATOR (new in Universal Ctags)

       TAG_OUTPUT_EXCMD	(new in	Universal Ctags)
	      Indicates	the specified type of EX command with --excmd option.

       TAG_OUTPUT_FILESEP (new in Universal Ctags)

       TAG_OUTPUT_MODE (new in Universal Ctags)

       TAG_PATTERN_LENGTH_LIMIT	(new in	Universal Ctags)

       TAG_PROC_CWD (new in Universal Ctags)
	      Indicates	the working directory of ctags during processing.

	      This pseudo-tag helps a client tool solve	the absolute paths for
	      the  input  files	for tag	entries	even when they are tagged with
	      relative paths.

	      An example of the	pseudo-tags:

		 $ cat tags
		 !_TAG_PROC_CWD	       /tmp/   //
		 main  input.c /^int main (void) { return 0; }$/;"     f       typeref:typename:int

	      From the regular tag for "main", the client tool	can  know  the
	      "main"  is  at "input.c".	 However, it is	a relative path. So if
	      the directory where ctags	run and	the directory where the	client
	      tool  runs  are different, the client tool cannot	find "input.c"
	      from the file system. In that case, TAG_PROC_CWD gives the  tool
	      a	hint; "input.c"	may be at "/tmp".


       TAG_ROLE_DESCRIPTION (new in Universal Ctags)
	      Indicates	the names and descriptions of enabled roles:

		 !_TAG_ROLE_DESCRIPTION!{language-name}!{kind-name}    {role-name}     /description/

	      If  your	tool relies on some roles, refer to the	pseudo-tags of
	      this type. Note that a role owned	by  a  disabled	 kind  is  not
	      listed even if the role itself is	enabled.


       Universal  ctags	 can run multiple parsers.  That means a parser, which
       supports	multiple parsers, may output  tags  for	 different  languages.
       language/l field	can be used to show the	language for each tag.

	  $ cat	/tmp/foo.html
	  <script>var x	= 1</script>
	  $ ./ctags -o - --extras=+g /tmp/foo.html
	  title	  /tmp/foo.html	  /^  <h1>title<\/h1>$/;" h
	  x	  /tmp/foo.html	  /var x = 1/;"	  v
	  $ ./ctags -o - --extras=+g --fields=+l /tmp/foo.html
	  title	  /tmp/foo.html	  /^  <h1>title<\/h1>$/;" h	  language:HTML
	  x	  /tmp/foo.html	  /var x = 1/;"	  v	  language:JavaScript

       See  readtags(1)	 to know how to	use readtags. This section is for dis-
       cussing some notable topics for client tools.

   Build Filter/Sorter Expressions
       Certain escape sequences	in expressions are recognized by readtags. For
       example,	 when searching	for a tag that matches a\?b, if	using a	filter
       expression like '(eq? $name "a\?b")', since \?  is  translated  into  a
       single ?	by readtags, it	actually searches for a?b.

       Another	problem	 is  if	 a  single  quote appear in filter expressions
       (which is also wrapped by single	quotes), it terminates the expression,
       producing  broken  expressions, and may even cause unintended shell in-
       jection.	Single quotes can be escaped using '"'"'.

       So, client tools	need to:

       o Replace \ by \\

       o Replace ' by '"'"'

       inside the expressions. If the expression also contains strings,	 "  in
       the strings needs to be replaced	by \".

       Client  tools  written  in Lisp could build the expression using	lists.
       prin1 (in Common	Lisp style Lisps) and write (in	 Scheme	 style	Lisps)
       can translate the list into a string that can be	directly used. For ex-
       ample, in EmacsLisp:

	  (let ((name "hi"))
	    (prin1 `(eq? $name ,name)))
	  => "(eq\\? $name "hi")"

       The "?" is escaped, and readtags	can  handle  it.  Scheme  style	 Lisps
       should  do  proper escaping so the expression readtags gets is just the
       expression passed into write. Common Lisp style Lisps may  produce  un-
       recognized  escape  sequences  by  readtags, like \#. Readtags provides
       some aliases for	these Lisps:

       o Use true for #t.

       o Use false for #f.

       o Use nil or () for ().

       o Use (string->regexp "PATTERN")	for  #/PATTERN/.  Use  (string->regexp
	 "PATTERN"  :case-fold true) for #/PATTERN/i. Notice that string->reg-
	 exp doesn't require escaping "/" in the pattern.

       Notice that even	when the client	tool uses this method, '  still	 needs
       to  be replaced by '"'"'	to prevent broken expressions and shell	injec-

       Another thing to	notice is that missing fields are represented  by  #f,
       and applying string operators to	them will produce an error. You	should
       always check if a field is missing before  applying  string  operators.
       See  the	"Filtering" section in readtags(1) to know how to do this. Run
       "readtags -H filter" to see which operators take	string arguments.

   Parse Readtags Output
       In the output of	readtags, tabs can appear in all field	values	(e.g.,
       the  tag	 name itself could contain tabs), which	makes it hard to split
       the line	into fields. Client tools should  use  the  -E	option,	 which
       keeps  the  escape  sequences  in the tags file,	so the only field that
       could contain tabs is the pattern field.

       The pattern field could:

       o Use a line number. It will look like number;" (e.g. 10;").

       o Use a search pattern. It will look like /pattern/;"  or  ?pattern?;".
	 Notice	that the search	pattern	could contain tabs.

       o Combine these two, like number;/pattern/;" or number;?pattern?;".

       These  are  true	for tags files using extended format, which is the de-
       fault one.  The legacy format (i.e.  --format=1)	 doesn't  include  the
       semicolons. It's	old and	barely used, so	we won't discuss it here.

       Client tools could split	the line using the following steps:

       o Find  the  first  2  tabs  in	the line, so we	get the	name and input

       o From the 2nd tab:

	 o If a	/ follows, then	the pattern delimiter is /.

	 o If a	? follows, then	the pattern delimiter is ?.

	 o If a	number follows,	then:

	   o If	a ;/ follows the number, then the delimiter is /.

	   o If	a ;? follows the number, then the delimiter is ?.

	   o If	a ;" follows the number, then the field	uses only line number,
	     and  there's  no  pattern	delimiter (since there's no regex pat-
	     tern). In this case the pattern field ends	at the 3rd tab.

       o After the opening delimiter, find the next unescaped  pattern	delim-
	 iter, and that's the closing delimiter. It will be followed by	;" and
	 then a	tab.  That's the end of	the pattern field. By "unescaped  pat-
	 tern  delimiter",  we	mean  there's  an even number (including 0) of
	 backslashes before it.

       o From here, split the rest of the line into fields by tabs.

       Then, the escape	sequences in  fields  other  than  the	pattern	 field
       should  be  translated. See "Proposal" in tags(5) to know about all the
       escape sequences.

   Make	Use of the Pattern Field
       The pattern field specifies how to find a tag in	its source  file.  The
       code  generating	 this field seems to have a long history, so there are
       some pitfalls and it's a	bit hard to handle. A client tool could	simply
       require the line: field and jump	to the line it specifies, to avoid us-
       ing the pattern field. But anyway, we'll	discuss	how to make  the  best
       use of it here.

       You  should  take  the  words here merely as suggestions, and not stan-
       dards. A	client tool could definitely develop better (or	simpler)  ways
       to use the pattern field.

       From  the  last section,	we know	the pattern field could	contain	a line
       number and a search pattern. When it only  contains  the	 line  number,
       handling	it is easy: you	simply go to that line.

       The  search  pattern  resembles	an EX command, but as we'll see	later,
       it's actually not a valid one, so some  manual  work  are  required  to
       process it.

       The  search  pattern could look like /pat/, called "forward search pat-
       tern", or ?pat?,	called "backward search	pattern". Using	a search  pat-
       tern means even if the source file is updated, as long as the part con-
       taining the tag doesn't change, we could	still locate the tag correctly
       by searching.

       When  the  pattern  field  only	contains  the search pattern, you just
       search for it. The search direction (forward/backward) doesn't  matter,
       as it's decided solely by whether the -B	option is enabled, and not the
       actual context. You could always	start the search from say  the	begin-
       ning of the file.

       When  both  the	search	pattern	and the	line number are	presented, you
       could make good use of the line number, by going	 to  the  line	first,
       then  searching	for the	nearest	occurrence of the pattern. A way to do
       this is to search both forward and backward for the pattern,  and  when
       there is	a occurrence on	both sides, go to the nearer one.

       What's  good  about  this is when there are multiple identical lines in
       the source file (e.g. the COMMON	block in Fortran), this	could help  us
       find the	correct	one, even after	the source file	is updated and the tag
       position	is shifted by a	few lines.

       Now let's discuss how to	search for the pattern.	After you trim	the  /
       or  ?  around it, the pattern resembles a regex pattern.	It should be a
       regex pattern, as required by being a valid EX command, but it's	 actu-
       ally not, as you'll see below.

       It could	begin with a ^,	which means the	pattern	starts from the	begin-
       ning of a line. It could	also end with an unescaped $ which  means  the
       pattern	ends  at  the  end of a	line. Let's keep this information, and
       trim them too.

       Now the remaining part is the actual string containing  the  tag.  Some
       characters are escaped:

       o \.

       o $, but	only at	the end	of the string.

       o /, but	only in	forward	search patterns.

       o ?, but	only in	backward search	patterns.

       You  need  to  unescape	these to get the literal string. Now you could
       convert this literal string to a	regexp that matches it	(by  escaping,
       like  re.escape	in  Python  or regexp-quote in Elisp), and assemble it
       with ^ or $ if the pattern originally has it, and  finally  search  for
       the tag using this regexp.

   Remark: About a Previous Format of the Pattern Field
       In  some	 earlier  versions  of Universal Ctags,	the line number	in the
       pattern field is	the actual line	number minus one, for  forward	search
       patterns; or plus one, for backward search patterns. The	idea is	to re-
       semble an EX command: you go to the line, then search  forward/backward
       for  the	pattern, and you can always find the correct one. But this de-
       nies the	purpose	of using a search pattern: to tolerate	file  updates.
       For  example, the tag is	at line	50, according to this scheme, the pat-
       tern field should be:


       Then let's assume that some code	above are removed, and the tag is  now
       at line 45. Now you can't find it if you	search forward from line 49.

       Due  to	this reason, Universal Ctags turns to use the actual line num-
       ber. A client tool  could  distinguish  them  by	 the  TAG_OUTPUT_EXCMD
       pseudo  tag, it's "combine" for the old scheme, and "combineV2" for the
       present scheme. But probably there's no need to treat them differently,
       since  "search for the nearest occurrence from the line"	gives good re-
       sult on both schemes.

       Universal Ctags supports	JSON (strictly	speaking  JSON	Lines)	output
       format  if  the ctags executable	is built with libjansson.  JSON	output
       goes to standard	output by default.

       Each JSON line represents a tag.

	  $ ctags --extras=+p --output-format=json --fields=-s
	  {"_type": "ptag", "name": "JSON_OUTPUT_VERSION", "path": "0.0", "pattern": "in development"}
	  {"_type": "ptag", "name": "TAG_FILE_SORTED", "path": "1", "pattern": "0=unsorted, 1=sorted, 2=foldcase"}
	  {"_type": "tag", "name": "Klass", "path": "/tmp/", "pattern":	"/^class Klass:$/", "language":	"Python", "kind": "class"}
	  {"_type": "tag", "name": "method", "path": "/tmp/", "pattern": "/^	def method(self):$/", "language": "Python", "kind": "member", "scope": "Klass",	"scopeKind": "class"}

       A key not starting with _ is mapped  to	a  field  of  ctags.   "--out-
       put-format=json --list-fields" options list the fields.

       A  key  starting	 with  _ represents meta information of	the JSON line.
       Currently only _type key	is used. If the	value for the key is tag,  the
       JSON  line represents a normal tag. If the value	is ptag, the line rep-
       resents a pseudo-tag.

       The output format can be	changed	 in  the  future.  JSON_OUTPUT_VERSION
       pseudo-tag  provides a change client-tools to handle the	changes.  Cur-
       rent version is "0.0". A	client-tool can	extract	the version with  path
       key from	the pseudo-tag.

       The JSON	output format is newly designed	and has	no limitation found in
       the default tags	file format.

       o The values for	kind key  are  represented  in	long-name  flags.   No
	 one-letter is here.

       o Scope	names  and  scope  kinds  have	distinguished  keys: scope and
	 scopeKind.  They are combined in the default tags file	format.

   Data	type used in a field
       Values for the most of all keys are represented in  JSON	 string	 type.
       However,	 some of them are represented in string, integer, and/or bool-
       ean type.

       "--output-format=json --list-fields" options show  What	kind  of  data
       type used in a field of JSON.

	  $ ctags --output-format=json --list-fields
	  F	  input		 yes	 NONE		  s--	 no    input file
	  P	  pattern	 yes	 NONE		  s-b	 no    pattern
	  f	  file		 yes	 NONE		  --b	 no    File-restricted scoping
	  e	  end		 no	 NONE		  -i-	 no    end lines of various items

       JSTYPE column shows the data types.

       's'    string

       'i'    integer

       'b'    boolean (true or false)

       For  an example,	the value for pattern field of ctags takes a string or
       a boolean value.

       ctags(1),  ctags-lang-python(7),	 ctags-incompatibilities(7),  tags(5),

5.9.0							 CTAGS-CLIENT-TOOLS(7)


Want to link to this manual page? Use this URL:

home | help