Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
Syntax::Highlight::PerUsermContributed PerSyntax::Highlight::Perl::Improved(3)

       Syntax::Highlight::Perl::Improved - Highlighting	of Perl	Syntactical

       This file documents Syntax::Highlight::Perl::Improved version 1.0.

	   # simple procedural
	   use Syntax::Highlight::Perl::Improved ':BASIC';  # or ':FULL'

	   print format_string($my_string);

	   # OO
	   use Syntax::Highlight::Perl::Improved;

	   my $formatter = new Syntax::Highlight::Perl::Improved;
	   print $formatter->format_string($my_string);

       This module provides syntax highlighting	for Perl code.	The design
       bias is roughly line-oriented and streamed (ie, processing a file line-
       by-line in a single pass).  Provisions may be made in the future	for
       tasks related to	"back-tracking"	(ie, re-doing a	single line in the
       middle of a stream) such	as speeding up state copying.

       The only	constructor provided is	"new()".  When called on an existing
       object, "new()" will create a new copy of that object.  Otherwise,
       "new()" creates a new copy of the (internal) Default Object.  Note that
       the use of the procedural syntax	modifies the Default Object and	that
       those changes will be reflected in any subsequent "new()" calls.

       Formatting is done using	the "format_string()" method.  Call
       "format_string()" with one or more strings to format, or	it will
       default to using	$_.

   Setting and Getting Formats
       You can set the text used for formatting	a syntax element using
       "set_format()" (or set the start	and end	format individually using
       "set_start_format()" and	"set_end_format()", respectively).

       You can also retrieve the text used for formatting for an element via
       "get_start_format()" or "get_end_format".  Bulk retrieval of the	names
       or values of defined formats is possible	via "get_format_names_list()"
       (names),	"get_start_format_values_list()" and

       See "FORMAT TYPES" later	in this	document for information on what
       format elements can be used.

   Checking and	Setting	the State
       You can check certain aspects of	the state of the formatter via the
       methods:	"in_heredoc()",	"in_string()", "in_pod()", "was_pod()",
       "in_data()", and	"line_count()".

       You can reset all of the	above states (and a few	other internal ones)
       using "reset()".

   Stable and Unstable Formatting Modes
       You can set or check the	stability of formatting	via "unstable()".

       In unstable (TRUE) mode,	formatting is not considered to	be persistent
       with nested formats.  Or, put another way, when unstable, the formatter
       can only	"remember" one format at a time	and must reinstate formatting
       for each	token.	An example of unstable formatting is using ANSI	color
       escape sequences	in a terminal.

       In stable (FALSE) mode (the default), formatting	is considered
       persistent within arbitrarily nested formats.  Even in stable mode,
       however,	formatting is never allowed to span multiple lines; it is
       always fully closed at the end of the line and reinstated at the
       beginning of a new line,	if necessary.  This is to ensure properly
       balanced	tags when only formatting a partial code snippet.  An example
       of stable formatting is HTML.

       Using "define_substitution()", you can have the formatter substitute
       certain strings with others, after the original string has been parsed
       (but before formatting is applied).  This is useful for escaping
       characters special to the output	mode (eg, > and	< in HTML) without
       them affecting the way the code is parsed.

       You can retrieve	the current substitutions (as a	hash-ref) via

       The Syntax::Highlight::Perl::Improved formatter recognizes and
       differentiates between many Perl	syntactical elements.  Each type of
       syntactical element has a Format	Type associated	with it.  There	is
       also a 'DEFAULT'	type that is applied to	any element who's Format Type
       does not	have a value.

       Several of the Format Types have	underscores in their name.  This
       underscore is special, and indicates that the Format Type can be
       "generalized."  This means that you can assign a	value to just the
       first part of the Format	Type name (the part before the underscore) and
       that value will be applied to all Format	Types with the same first
       part.  For example, the Format Types for	all types of variables begin
       with "Variable_".  Thus,	if you assign a	value to the Format Type
       "Variable", it will be applied to any type of variable.	Generalized
       Format Types take precedence over non-generalized Format	Types.	So the
       value assigned to "Variable" would be applied to	"Variable_Scalar",
       even if "Variable_Scalar" had a value explicitly	assigned to it.

       You can also define a "short-cut" name for each Format Type that	can be
       generalized.  The short-cut name	would be the part of the Format	Type
       name after the underscore.  For example,	the short-cut for
       "Variable_Scalar" would be "Scalar".  Short-cut names have the least
       precedence and are only assigned	if neither the generalized Type	name,
       nor the full Type name have values.

       Following is a list of all the syntactical elements that
       Syntax::Highlight::Perl::Improved currently recognizes, along with a
       short description of what each would be applied to.

	   A normal Perl comment.  Starts with '#' and goes until the end of
	   the line.

	   Inline documentation.  Starts with a	line beginning with an equal
	   sign	('=') followed by a word (eg: '=pod') and continuing until a
	   line	beginning with '=cut'.

	   Either the "she-bang" line at the beginning of the file, or a line
	   directive altering what the compiler	thinks the current line	and
	   file	is.

	   A loop or statement label (to be the	target of a goto, next,	last
	   or redo).

	   Any string or character that	begins or ends a String.  Including,
	   but not necessarily limited to: quote-like regular expression
	   operators ("m//", "s///", "tr///", etc), a Here-Document
	   terminating line, the lone period terminating a format, and,	of
	   course, normal quotes ("'", """, "`", "q{}",	"qq{}",	"qr{}",

	   Any text within quotes, "format"s, Here-Documents, Regular
	   Expressions,	and the	like.

	   The identifier used to define, identify, or call a subroutine (or
	   method).  Note that Syntax::Highlight::Perl::Improved cannot
	   recognize a subroutine if it	is called without using	parentheses or
	   an ampersand, or methods called using the indirect object syntax.
	   It formats those as barewords.

	   A scalar variable.

	   Note	that (theoretically) this format is not	applied	to non-scalar
	   variables that are being used as scalars (ie: array or hash
	   lookups, nor	references to anything other than scalars).
	   Syntax::Highlight::Perl::Improved figures out (or at	least tries
	   to) the actual type of the variable being used (by looking at how
	   you're subscripting it) and formats it accordingly.	The first
	   character of	the variable (ie, the "$", "@",	"%", or	"*") tells you
	   the type of value being used, and the color (hopefully) tells you
	   the type of variable	being used to get that value.

	   (See	"KNOWN ISSUES" for information about when this doesn't work
	   quite right.)

	   An array variable (but not usually a	slice; see above).

	   A hash variable.

	   A typeglob.	Note that typeglobs not	beginning with an asterisk (*)
	   (eg:	filehandles) are formatted as barewords.  This is because,
	   well, they are.

	   Whitespace.	Not usually formatted but it can be.

	   A special, or backslash-escaped, character.	For example: "\n"
	   (newline), or "\d" (digits).

	   Only	occurs within strings or regular expressions.

	   A Perl keyword.  Some examples include: my, local, sub, next.

	   Note	that Perl does not make	any distinction	between	keywords and
	   built-in functions (at least	not in the documentation).  Thus I had
	   to make a subjective	call as	to what	would be considered keywords
	   and what would be built-in functions.

	   The list of keywords	can be found (and overloaded) in the variable
	   $Syntax::Highlight::Perl::Improved::keyword_list_re as a pre-
	   compiled regular expression.

	   A Perl built-in function, called as a function (ie, using

	   The list of built-in	functions can be found (and overloaded)	in the
	   variable $Syntax::Highlight::Perl::Improved::builtin_list_re	as a
	   pre-compiled	regular	expression.

	   A Perl built-in function, called as a list or unary operator	(ie,
	   without using parentheses).

	   The list of built-in	functions can be found (and overloaded)	in the
	   variable $Syntax::Highlight::Perl::Improved::builtin_list_re	as a
	   pre-compiled	regular	expression.

	   A Perl operator.

	   The list of operators can be	found (and overloaded) in the variable
	   $Syntax::Highlight::Perl::Improved::operator_list_re	as a pre-
	   compiled regular expression.

	   A bareword.	This can be user-defined subroutine called without
	   parentheses,	a typeglob used	without	an asterisk (*), or just a
	   plain old bareword.

	   The name of a package or pragmatic module.

	   Note	that this does not apply to the	package	portion	of a fully
	   qualified variable name.

	   A numeric literal.

	   A symbol (ie, non-operator punctuation).

	   The special tokens that signal the end of executable	code and the
	   begining of the DATA	section.  Specifically,	'"__END__"' and

	   Anything in the DATA	section	(see "CodeTerm").

       Syntax::Highlight::Perl::Improved uses OO method-calls internally (and
       actually	defines	a Default Object that is used when the functions are
       invoked procedurally) so	you will not gain anything (efficiency-wise)
       by using	the procedural interface.  It is just a	matter of style.

       It is actually recommended that you use the OO interface, as this
       allows you to instantiate multiple, concurrent-yet-separate formatters.
       Though I	cannot think of	why you	would need multiple formatters
       instantiated. :-)

       One point to note: the "new()" method uses the Default Object to
       initialize new objects.	This means that	any changes to the state of
       the Default Object (including Format definitions) made by using the
       procedural interface will be reflected in any subsequently created
       objects.	 This can be useful in some cases (eg, call "set_format()"
       procedurally just before	creating a batch of new	objects	to define
       default Formats for them	all) but will most likely lead to trouble.

       new PACKAGE
       new OBJECT
	   Creates a new object.  If called on an existing object, creates a
	   new copy of that object (which is thenceforth totally separate from
	   the original).

	   Resets the object's internal	state.	This breaks out	of strings and
	   here-docs, ends PODs, resets	the line-count,	and otherwise gets the
	   object back into a "normal" state to	begin processing a new stream.

	   Note	that this does not reset any user options (including formats
	   and format stability).

       unstable	EXPR
	   Returns true	if the formatter is in unstable	mode.

	   If called with a non-zero number, puts the formatter	into unstable
	   formatting mode.

	   In unstable mode, it	is assumed that	formatting is not persistent
	   one token to	the next and that each token must be explicitly

	   Returns true	if the next string to be formatted will	be inside a

	   Returns true	if the next string to be formatted will	be inside a
	   multi-line string.

	   Returns true	if the formatter would consider	the next string	passed
	   to it as begin within a POD structure.  This	is false immediately
	   before any POD instigators ("=pod", "=head1", "=item", etc),	true
	   immediately after an	instigator, throughout the POD and immediately
	   before the POD terminator ("=cut"), and false immediately after the
	   POD terminator.

	   Returns true	if the last line of the	string just formatted was part
	   of a	POD structure.	This includes the "/^=\w+/" POD	instigators
	   and terminators.

	   Returns true	if the next string to be formatted will	be inside the
	   DATA	section	(ie, follows a "__DATA__" or "__END__" tag).

	   Returns the number of lines processed by the	formatter.

	   Returns a reference to the substitution table used.	The
	   substitution	table is a hash	whose keys are the strings to be
	   replaced, and whose values are what to replace them with.

       define_substitution HASH_REF
       define_substitution LIST
	   Allows user to define certain characters that will be substituted
	   before formatting is	done (but after	they have been processed for

	   If the first	parameter is a reference to a hash, the	formatter will
	   replace it's	own hash with the given	one, and subsequent changes to
	   the hash outside the	formatter will be reflected.

	   Otherwise, it will copy the arguments passed	into it's own hash,
	   and any substitutions already defined (but not in the parameter
	   list) will be preserved. (ie, the new substitutions will be added,
	   without destroying what was there already.)

       set_start_format	HASH_REF
       set_start_format	LIST
	   Given either	a list of keys/values, or a reference to a hash	of
	   keys/values,	copy them into the object's Formats list.

       set_end_format HASH_REF
       set_end_format LIST
	   Given either	a list of keys/values, or a reference to a hash	of
	   keys/values,	copy them into the object's Formats list.

       set_format LIST
	   Sets	the formatting string for one or more formats.

	   You should pass a list of keys/values where the keys	are the	format
	   names and the values	are references to arrays containing the
	   starting and	ending formatting strings (in that order) for that

       get_start_format	LIST
	   Retrieve the	string that is inserted	to begin a given format	type
	   (starting format string).

	   The names are looked	for in the following order:

	   First: Prefer the names joined by underscore, from most general to
	   least.  For example,	given ("Variable", "Scalar"): "Variable" then

	   Second: Then	try each name singly, in reverse order.	 For example,
	   "Scalar" then "Variable".

	   See "FORMAT TYPES" for more information.

       get_end_format LIST
	   Retrieve the	string that is inserted	to end a given format type
	   (ending format string).

	   Returns a list of the names of all the Formats defined.

	   Returns a list of the values	of all the start Formats defined (in
	   the same order as the names returned	by "get_format_names_list()").

	   Returns a list of the values	of all the end Formats defined (in the
	   same	order as the names returned by "get_format_names_list()").

       format_string LIST
	   Formats one or more strings of Perl code.  If no strings are
	   specified, defaults to $_.  Returns the list	of formatted strings
	   (or the first string	formatted if called in scalar context).

	   Note:  The end of the string	is considered to be the	end of a line,
	   regardless of whether or not	there is a trailing line-break (but
	   trailing line-breaks	will not cause an extra, empty line).

	   Another Note:  The function actually	uses $/	to determine line-
	   breaks, unless $/ is	set to "\n" (newline).	If $/ is "\n", then it
	   looks for the first match of	"m/\r?\n|\n?\r/" in the	string and
	   uses	that to	determine line-breaks.	This is	to make	it easy	to
	   handle non-unix text.  Whatever characters it ends up using as
	   line-breaks are preserved.

       format_token TOKEN, LIST
	   Returns TOKEN wrapped in the	start and end Formats corresponding to
	   LIST	(as would be returned by "get_start_format( LIST )" and
	   "get_end_format( LIST )", respectively).

	   No syntax checking is done on TOKEN but substitutions defined with
	   "define_substitution()" are performed.

       o   Barewords used as keys to a hash are	formatted as strings.  This is
	   Good.  They should not be, however, if they are not the only	thing
	   within the curly braces.  That can be fixed.

       o   This	version	does not handle	formats	(see perlform(1)) very well.
	   It treats them as Here-Documents and	ignores	the rules for comment
	   lines, as well as the fact that picture lines are not supposed to
	   be interpolated.  Thus, your	picture	lines will look	strange	with
	   the '@'s being formatted as array variables (albeit,	invalid	ones).
	   Ideally, it would also treat	value lines as normal Perl code	and
	   format accordingly.	I think	I'll get to the	comment	lines and non-
	   interpolating picture lines first.  If/When I do get	this fixed, I
	   will	most likely add	a format type of 'Format' or something,	so
	   that	they can be formatted differently, if so desired.

       o   This	version	does not handle	Regular	Expression significant
	   characters.	It simply treats Regular Expressions as	interpolated

       o   User-defined	subroutines, called without parentheses, are formatted
	   as barewords.  This is because there	is no way to tell them apart
	   from	barewords without parsing the code, and	would require us to go
	   as far as perl does when doing the "-c" check (ie, executing	BEGIN
	   and END blocks and the like).  That's not going to happen.

       o   If you are indexing (subscripting) an array or hash,	the formatter
	   tries to figure out the "real" variable class by looking at how you
	   index the variable.	However, if you	do something funky (but	legal
	   in Perl) and	put line-breaks	or comments between the	variable class
	   character ($) and your identifier, the formatter will get confused
	   and treat your variable as a	scalar.	 Until it finds	the index
	   character.  Then it will format the scalar class character ($) as a
	   scalar and your identifier as the "correct" class.

       o   If you put a	line-break between your	variable identifier and	it's
	   indexing character (see above), which is also legal in Perl,	the
	   formatter will never	find it	and treat your variable	as a scalar.

       o   If you put a	line-break between a bareword hash-subscript and the
	   hash	variable, or between a bareword	and its	associated "=>"
	   operator, the bareword will not be formatted	correctly (as a
	   string).  (Noticing a pattern here?)

       Bug reports are always welcome. Email me	at b<>.

       David C.Y. Liu b<>

       based on	code by	Cory Johns

       Copyright (c) 2004 David	C.Y. Liu.  This	library	is free	software; you
       can redistribute	and/or modify it under the same	conditions as Perl

       Note: This is Cory John's todo list, not	mine. Currently	none of	these
       features	are planned for	the near future.

       1.  Improve handling of regular expressions.  Add support for regexp-
	   special characters.	Recognize the /e option	to the substitution
	   operator (maybe).

       2.  Improve handling of formats.	 Don't treat format definitions	as
	   interpolating.  Handle format-comments.  Possibly format value
	   lines as normal Perl	code.

       3.  Create in-memory deep-copy routine to replace "eval(Data::Dumper)"

       4.  Generalize state transitions	("reset()" and,	in the future,
	   "copy_state()") to use non-hard-coded keys and values for state
	   variables.  Probably	will extrapolate them into an overloadable
	   hash, and use the aforementioned deep-copy to assign	them.

       5.  Create a method to save or copy states between objects
	   ("copy_state()").  Would be useful for using	this module in an

       6.  Add support for greater-than-one length special characters.
	   Specifically, octal,	hexidecimal, and control character codes.  For
	   example, "\644", "\x1a4" or "\c[".

   05-03-2004  David C.Y. Liu (Version 1.01)
       o   Added 'our' to the keywords list.

       o   Fixed bug that prevented interpolation inside qq() quotes.

       o   Renamed to Syntax::Highlight::Perl::Improved.

   04-04-2001  Cory Johns
       o   Fixed problem with special characters not formatting	inside of

       o   Fixed bug causing hash variables to format inside of	Here-

   03-30-2001  Cory Johns
       o   Fixed bug where quote-terminators were checked for inside of	Here-

   03-29-2001  Cory Johns
       o   Moved token processing tests	from _format_line() into
	   _process_token() (where they	should've been all along), generally
	   making _format_line() more logical.	Contemplating extrapolating
	   the tokenizing and token loop into its own subroutine to avoid all
	   the recursive calls.

       o   Fixed bug that caused special characters to be recognized outside
	   of strings.

       o   Added $VERSION variable.

       o   Added support for different types of	literal	numbers: floating
	   point, exponential notation (eg: 1.3e10), hexidecimal, and

       o   Added the "CodeTerm"	and "DATA" Formats.

   03-27-2001  Cory Johns
       o   Added was_pod() and updated the documentation for in_pod().

   03-20-2001  Cory Johns
       o   Added support for Perl formats (ie, `"format	= ..."').

       Hey! The	above document had some	coding errors, which are explained

       Around line 47:
	   You forgot a	'=back'	before '=head2'

       Around line 102:
	   =back without =over

perl v5.24.1			  2004-05-Syntax::Highlight::Perl::Improved(3)


Want to link to this manual page? Use this URL:

home | help