Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
INNOTOP(1)	      User Contributed Perl Documentation	    INNOTOP(1)

NAME
       innotop - MySQL and InnoDB transaction/status monitor.

SYNOPSIS
       To monitor servers normally:

	innotop

       To monitor InnoDB status	information from a file:

	innotop	/var/log/mysql/mysqld.err

       To run innotop non-interactively	in a pipe-and-filter configuration:

	innotop	--count	5 -d 1 -n

       To monitor a database on	another	system using a particular username and
       password:

	innotop	-u <username> -p <password> -h <hostname>

DESCRIPTION
       innotop monitors	MySQL servers.	Each of	its modes shows	you a
       different aspect	of what's happening in the server.  For	example,
       there's a mode for monitoring replication, one for queries, and one for
       transactions.  innotop refreshes	its data periodically, so you see an
       updating	view.

       innotop has lots	of features for	power users, but you can start and run
       it with virtually no configuration.  If you're just getting started,
       see "QUICK-START".  Press '?' at	any time while running innotop for
       context-sensitive help.

QUICK-START
       To start	innotop, open a	terminal or command prompt.  If	you have
       installed innotop on your system, you should be able to just type
       "innotop" and press Enter; otherwise, you will need to change to
       innotop's directory and type "perl innotop".

       With no options specified, innotop will attempt to connect to a MySQL
       server on localhost using mysql_read_default_group=client for other
       connection parameters.  If you need to specify a	different username and
       password, use the -u and	-p options, respectively.  To monitor a	MySQL
       database	on another host, use the -h option.

       After you've connected, innotop should show you something like the
       following:

	[RO] Query List	(? for help) localhost,	01:11:19, 449.44 QPS, 14/7/163 con/run

	CXN	   When	  Load	QPS    Slow  QCacheHit	KCacheHit  BpsIn    BpsOut
	localhost  Total  0.00	1.07k	697	 0.00%	   98.17%  476.83k  242.83k

	CXN	   Cmd	  ID	     User  Host	     DB	  Time	 Query
	localhost  Query  766446598  test  10.0.0.1  foo  00:02	 INSERT	INTO table (

       (This sample is truncated at the	right so it will fit on	a terminal
       when running 'man innotop')

       If your server is busy, you'll see more output.	Notice the first line
       on the screen, which tells you that readonly is set to true ([RO]),
       what mode you're	in and what server you're connected to.	 You can
       change to other modes with keystrokes; press 'T'	to switch to a list of
       InnoDB transactions, for	example.

       Press the '?' key to see	what keys are active in	the current mode.  You
       can press any of	these keys and innotop will either take	the requested
       action or prompt	you for	more input.  If	your system has	Term::ReadLine
       support,	you can	use TAB	and other keys to auto-complete	and edit
       input.

       To quit innotop,	press the 'q' key.

OPTIONS
       innotop is mostly configured via	its configuration file,	but some of
       the configuration options can come from the command line.  You can also
       specify a file to monitor for InnoDB status output; see "MONITORING A
       FILE" for more details.

       You can negate some options by prefixing	the option name	with --no.
       For example, --noinc (or	--no-inc) negates "--inc".

       --color
	   Enable or disable terminal coloring.	 Corresponds to	the "color"
	   config file setting.

       --config
	   Specifies a configuration file to read.  This option	is non-sticky,
	   that	is to say it does not persist to the configuration file
	   itself.

       --count
	   Refresh only	the specified number of	times (ticks) before exiting.
	   Each	refresh	is a pause for "interval" seconds, followed by
	   requesting data from	MySQL connections and printing it to the
	   terminal.

       --delay
	   Specifies the amount	of time	to pause between ticks (refreshes).
	   Corresponds to the configuration option "interval".

       --help
	   Print a summary of command-line usage and exit.

       --host
	   Host	to connect to.

       --inc
	   Specifies whether innotop should display absolute numbers or
	   relative numbers (offsets from their	previous values).  Corresponds
	   to the configuration	option "status_inc".

       --mode
	   Specifies the mode in which innotop should start.  Corresponds to
	   the configuration option "mode".

       --nonint
	   Enable non-interactive operation.  See "NON-INTERACTIVE OPERATION"
	   for more.

       --password
	   Password to use for connection.

       --port
	   Port	to use for connection.

       --skipcentral
	   Don't read the central configuration	file.

       --timestamp
	   In -n mode, write a timestamp either	before every screenful of
	   output, or if the option is given twice, at the start of every
	   line.  The format is	controlled by the timeformat config variable.

       --user
	   User	to use for connection.

       --version
	   Output version information and exit.

       --write
	   Sets	the configuration option "readonly" to 0, making innotop write
	   the running configuration to	~/.innotop/innotop.conf	on exit, if no
	   configuration file was loaded at start-up.

HOTKEYS
       innotop is interactive, and you control it with key-presses.

       o   Uppercase keys switch between modes.

       o   Lowercase keys initiate some	action within the current mode.

       o   Other keys do something special like	change configuration or	show
	   the innotop license.

       Press '?' at any	time to	see the	currently active keys and what they
       do.

MODES
       Each of innotop's modes retrieves and displays a	particular type	of
       data from the servers you're monitoring.	 You switch between modes with
       uppercase keys.	The following is a brief description of	each mode, in
       alphabetical order.  To switch to the mode, press the key listed	in
       front of	its heading in the following list:

       A: Health Dashboard
	   This	mode displays a	single table with one row per monitored
	   server. The columns show essential overview information about the
	   server's health, and	coloration rules show whether replication is
	   running or if there are any very long-running queries or excessive
	   replication delay.

       B: InnoDB Buffers
	   This	mode displays information about	the InnoDB buffer pool,	page
	   statistics, insert buffer, and adaptive hash	index.	The data comes
	   from	SHOW INNODB STATUS.

	   This	mode contains the "buffer_pool", "page_statistics",
	   "insert_buffers", and "adaptive_hash_index" tables by default.

       C: Command Summary
	   This	mode is	similar	to mytop's Command Summary mode.  It shows the
	   "cmd_summary" table,	which looks something like the following:

	    Command Summary (? for help) localhost, 25+07:16:43, 2.45 QPS, 3 thd, 5.0.40
	    _____________________ Command Summary _____________________
	    Name		    Value    Pct     Last Incr	Pct
	    Select_scan		    3244858  69.89%	     2	100.00%
	    Select_range	    1354177  29.17%	     0	  0.00%
	    Select_full_join	      39479   0.85%	     0	  0.00%
	    Select_full_range_join     4097   0.09%	     0	  0.00%
	    Select_range_check		  0   0.00%	     0	  0.00%

	   The command summary table is	built by extracting variables from
	   "STATUS_VARIABLES".	The variables must be numeric and must match
	   the prefix given by the "cmd_filter"	configuration variable.	 The
	   variables are then sorted by	value descending and compared to the
	   last	variable, as shown above.  The percentage columns are
	   percentage of the total of all variables in the table, so you can
	   see the relative weight of the variables.

	   The example shows what you see if the prefix	is "Select_".  The
	   default prefix is "Com_".  You can choose a prefix with the 's'
	   key.

	   It's	rather like running SHOW VARIABLES LIKE	"prefix%" with memory
	   and nice formatting.

	   Values are aggregated across	all servers.  The Pct columns are not
	   correctly aggregated	across multiple	servers.  This is a known
	   limitation of the grouping algorithm	that may be fixed in the
	   future.

       D: InnoDB Deadlocks
	   This	mode shows the transactions involved in	the last InnoDB
	   deadlock.  A	second table shows the locks each transaction held and
	   waited for.	A deadlock is caused by	a cycle	in the waits-for
	   graph, so there should be two locks held and	one waited for unless
	   the deadlock	information is truncated.

	   InnoDB puts deadlock	information before some	other information in
	   the SHOW INNODB STATUS output.  If there are	a lot of locks,	the
	   deadlock information	can grow very large, and there is a limit on
	   the size of the SHOW	INNODB STATUS output.  A large deadlock	can
	   fill	the entire output, or even be truncated, and prevent you from
	   seeing other	information at all.  If	you are	running	innotop	in
	   another mode, for example T mode, and suddenly you don't see
	   anything, you might want to check and see if	a deadlock has wiped
	   out the data	you need.

	   If it has, you can create a small deadlock to replace the large
	   one.	 Use the 'w' key to 'wipe' the large deadlock with a small
	   one.	 This will not work unless you have defined a deadlock table
	   for the connection (see "SERVER CONNECTIONS").

	   You can also	configure innotop to automatically detect when a large
	   deadlock needs to be	replaced with a	small one (see
	   "auto_wipe_dl").

	   This	mode displays the "deadlock_transactions" and "deadlock_locks"
	   tables by default.

       F: InnoDB Foreign Key Errors
	   This	mode shows the last InnoDB foreign key error information, such
	   as the table	where it happened, when	and who	and what query caused
	   it, and so on.

	   InnoDB has a	huge variety of	foreign	key error messages, and	many
	   of them are just hard to parse.  innotop doesn't always do the best
	   job here, but there's so much code devoted to parsing this messy,
	   unparseable output that innotop is likely never to be perfect in
	   this	regard.	 If innotop doesn't show you what you need to see,
	   just	look at	the status text	directly.

	   This	mode displays the "fk_error" table by default.

       I: InnoDB I/O Info
	   This	mode shows InnoDB's I/O	statistics, including the I/O threads,
	   pending I/O,	file I/O miscellaneous,	and log	statistics.  It
	   displays the	"io_threads", "pending_io", "file_io_misc", and
	   "log_statistics" tables by default.

       K: InnoDB Lock Waits
	   This	mode shows information from InnoDB plugin's transaction	and
	   locking tables.  You	can use	it to find when	a transaction is
	   waiting for another,	and kill the blocking transaction. It displays
	   the "innodb_blocked_blocker"	table.

       L: Locks
	   This	mode shows information about current locks.  At	the moment
	   only	InnoDB locks are supported, and	by default you'll only see
	   locks for which transactions	are waiting.  This information comes
	   from	the TRANSACTIONS section of the	InnoDB status text.  If	you
	   have	a very busy server, you	may have frequent lock waits; it helps
	   to be able to see which tables and indexes are the "hot spot" for
	   locks.  If your server is running pretty well, this mode should
	   show	nothing.

	   You can configure MySQL and innotop to monitor not only locks for
	   which a transaction is waiting, but those currently held, too.  You
	   can do this with the	InnoDB Lock Monitor
	   (<http://dev.mysql.com/doc/en/innodb-monitor.html>).	 It's not
	   documented in the MySQL manual, but creating	the lock monitor with
	   the following statement also	affects	the output of SHOW INNODB
	   STATUS, which innotop uses:

	     CREATE TABLE innodb_lock_monitor(a	int) ENGINE=INNODB;

	   This	causes InnoDB to print its output to the MySQL file every 16
	   seconds or so, as stated in the manual, but it also makes the
	   normal SHOW INNODB STATUS output include lock information, which
	   innotop can parse and display (that's the undocumented feature).

	   This	means you can do what may have seemed impossible: to a limited
	   extent (InnoDB truncates some information in	the output), you can
	   see which transaction holds the locks something else	is waiting
	   for.	 You can also enable and disable the InnoDB Lock Monitor with
	   the key mappings in this mode.

	   This	mode displays the "innodb_locks" table by default.  Here's a
	   sample of the screen	when one connection is waiting for locks
	   another connection holds:

	    _________________________________ InnoDB Locks __________________________
	    CXN	       ID  Type	   Waiting  Wait   Active  Mode	 DB    Table  Index
	    localhost  12  RECORD	 1  00:10   00:10  X	 test  t1     PRIMARY
	    localhost  12  TABLE	 0  00:10   00:10  IX	 test  t1
	    localhost  12  RECORD	 1  00:10   00:10  X	 test  t1     PRIMARY
	    localhost  11  TABLE	 0  00:00   00:25  IX	 test  t1
	    localhost  11  RECORD	 0  00:00   00:25  X	 test  t1     PRIMARY

	   You can see the first connection, ID	12, is waiting for a lock on
	   the PRIMARY key on test.t1, and has been waiting for	10 seconds.
	   The second connection isn't waiting,	because	the Waiting column is
	   0, but it holds locks on the	same index.  That tells	you connection
	   11 is blocking connection 12.

       M: Master/Slave Replication Status
	   This	mode shows the output of SHOW SLAVE STATUS and SHOW MASTER
	   STATUS in three tables.  The	first two divide the slave's status
	   into	SQL and	I/O thread status, and the last	shows master status.
	   Filters are applied to eliminate non-slave servers from the slave
	   tables, and non-master servers from the master table.

	   This	mode displays the "slave_sql_status", "slave_io_status", and
	   "master_status" tables by default.

       O: Open Tables
	   This	section	comes from MySQL's SHOW	OPEN TABLES command.  By
	   default it is filtered to show tables which are in use by one or
	   more	queries, so you	can get	a quick	look at	which tables are
	   'hot'.  You can use this to guess which tables might	be locked
	   implicitly.

	   This	mode displays the "open_tables"	mode by	default.

       U: User Statistics
	   This	mode displays data that's available in Percona's enhanced
	   version of MySQL (also known	as Percona Server with XtraDB).
	   Specifically, it makes it easy to enable and	disable	the so-called
	   "user statistics."  This feature gathers stats on clients, threads,
	   users, tables, and indexes and makes	them available as
	   INFORMATION_SCHEMA tables.  These are invaluable for	understanding
	   what	your server is doing.  They are	also available in MariaDB.

	   The statistics supported so far are only from the TABLE_STATISTICS
	   and INDEX_STATISTICS	tables added by	Percona.  There	are three
	   views: one of table stats, one of index stats (which	can be
	   aggregated with the = key), and one of both.

	   The server doesn't gather these stats by default.  You have to set
	   the variable	userstat_running to turn it on.	 You can do this
	   easily with innotop from U mode, with the 's' key.

       Q: Query	List
	   This	mode displays the output from SHOW FULL	PROCESSLIST, much like
	   mytop's query list mode.  This mode does not	show InnoDB-related
	   information.	 This is probably one of the most useful modes for
	   general usage.

	   There is an informative header that shows general status
	   information about your server.  You can toggle it on	and off	with
	   the 'h' key.	 By default, innotop hides inactive processes and its
	   own process.	 You can toggle	these on and off with the 'i' and 'a'
	   keys.

	   You can EXPLAIN a query from	this mode with the 'e' key.  This
	   displays the	query's	full text, the results of EXPLAIN, and in
	   newer MySQL versions, even the optimized query resulting from
	   EXPLAIN EXTENDED.  innotop also tries to rewrite certain queries to
	   make	them EXPLAIN-able.  For	example, INSERT/SELECT statements are
	   rewritable.

	   This	mode displays the "q_header" and "processlist" tables by
	   default.

       R: InnoDB Row Operations	and Semaphores
	   This	mode shows InnoDB row operations, row operation	miscellaneous,
	   semaphores, and information from the	wait array.  It	displays the
	   "row_operations", "row_operation_misc", "semaphores", and
	   "wait_array"	tables by default.

       S: Variables & Status
	   This	mode calculates	statistics, such as queries per	second,	and
	   prints them out in several different	styles.	 You can show absolute
	   values, or incremental values between ticks.

	   You can switch between the views by pressing	a key.	The 's'	key
	   prints a single line	each time the screen updates, in the style of
	   vmstat.  The	'g' key	changes	the view to a graph of the same
	   numbers, sort of like tload.	 The 'v' key changes the view to a
	   pivoted table of variable names on the left,	with successive
	   updates scrolling across the	screen from left to right.  You	can
	   choose how many updates to put on the screen	with the
	   "num_status_sets" configuration variable.

	   Headers may be abbreviated to fit on	the screen in interactive
	   operation.  You choose which	variables to display with the 'c' key,
	   which selects from predefined sets, or lets you create your own
	   sets.  You can edit the current set with the	'e' key.

	   This	mode doesn't really display any	tables like other modes.
	   Instead, it uses a table definition to extract and format the data,
	   but it then transforms the result in	special	ways before outputting
	   it.	It uses	the "var_status" table definition for this.

       T: InnoDB Transactions
	   This	mode shows transactions	from the InnoDB	monitor's output, in
	   top-like format.  This mode is the reason I wrote innotop.

	   You can kill	queries	or processes with the 'k' and 'x' keys,	and
	   EXPLAIN a query with	the 'e'	or 'f' keys.  InnoDB doesn't print the
	   full	query in transactions, so explaining may not work right	if the
	   query is truncated.

	   The informational header can	be toggled on and off with the 'h'
	   key.	 By default, innotop hides inactive transactions and its own
	   transaction.	 You can toggle	this on	and off	with the 'i' and 'a'
	   keys.

	   This	mode displays the "t_header" and "innodb_transactions" tables
	   by default.

INNOTOP	STATUS
       The first line innotop displays is a "status bar" of sorts.  What it
       contains	depends	on the mode you're in, and what	servers	you're
       monitoring.  The	first few words	are always [RO]	(if readonly is	set to
       1), the innotop mode, such as "InnoDB Txns" for T mode, followed	by a
       reminder	to press '?' for help at any time.

   ONE SERVER
       The simplest case is when you're	monitoring a single server.  In	this
       case, the name of the connection	is next	on the status line.  This is
       the name	you gave when you created the connection -- most likely	the
       MySQL server's hostname.	 This is followed by the server's uptime.

       If you're in an InnoDB mode, such as T or B, the	next word is "InnoDB"
       followed	by some	information about the SHOW INNODB STATUS output	used
       to render the screen.  The first	word is	the number of seconds since
       the last	SHOW INNODB STATUS, which InnoDB uses to calculate some	per-
       second statistics.  The next is a smiley	face indicating	whether	the
       InnoDB output is	truncated.  If the smiley face is a :-), all is	well;
       there is	no truncation.	A :^| means the	transaction list is so long,
       InnoDB has only printed out some	of the transactions.  Finally, a frown
       :-( means the output is incomplete, which is probably due to a deadlock
       printing	too much lock information (see "D: InnoDB Deadlocks").

       The next	two words indicate the server's	queries	per second (QPS) and
       how many	threads	(connections) exist.  Finally, the server's version
       number is the last thing	on the line.

   MULTIPLE SERVERS
       If you are monitoring multiple servers (see "SERVER CONNECTIONS"), the
       status line does	not show any details about individual servers.
       Instead,	it shows the names of the connections that are active.	Again,
       these are connection names you specified, which are likely to be	the
       server's	hostname.  A connection	that has an error is prefixed with an
       exclamation point.

       If you are monitoring a group of	servers	(see "SERVER GROUPS"), the
       status line shows the name of the group.	 If any	connection in the
       group has an error, the group's name is followed	by the fraction	of the
       connections that	don't have errors.

       See "ERROR HANDLING" for	more details about innotop's error handling.

   MONITORING A	FILE
       If you give a filename on the command line, innotop will	not connect to
       ANY servers at all.  It will watch the specified	file for InnoDB	status
       output and use that as its data source.	It will	always show a single
       connection called 'file'.  And since it can't connect to	a server, it
       can't determine how long	the server it's	monitoring has been up;	so it
       calculates the server's uptime as time since innotop started running.

SERVER ADMINISTRATION
       While innotop is	primarily a monitor that lets you watch	and analyze
       your servers, it	can also send commands to servers.  The	most
       frequently useful commands are killing queries and stopping or starting
       slaves.

       You can kill a connection, or in	newer versions of MySQL	kill a query
       but not a connection, from "Q: Query List" and "T: InnoDB Transactions"
       modes.  Press 'k' to issue a KILL command, or 'x' to issue a KILL QUERY
       command.	 innotop will prompt you for the server	and/or connection ID
       to kill (innotop	does not prompt	you if there is	only one possible
       choice for any input).  innotop pre-selects the longest-running query,
       or the oldest connection.  Confirm the command with 'y'.

       In "Slave Replication Status"" in "M: Master mode, you can start	and
       stop slaves with	the 'a'	and 'o'	keys, respectively.  You can send
       these commands to many slaves at	once.  innotop fills in	a default
       command of START	SLAVE or STOP SLAVE for	you, but you can actually edit
       the command and send anything you wish, such as SET GLOBAL
       SQL_SLAVE_SKIP_COUNTER=1	to make	the slave skip one binlog event	when
       it starts.

       You can also ask	innotop	to calculate the earliest binlog in use	by any
       slave and issue a PURGE MASTER LOGS on the master.  Use the 'b' key for
       this.  innotop will prompt you for a master to run the command on, then
       prompt you for the connection names of that master's slaves (there is
       no way for innotop to determine this reliably itself).  innotop will
       find the	minimum	binlog in use by these slave connections and suggest
       it as the argument to PURGE MASTER LOGS.

       in "U: User Statistics" mode, you can use the 's' key to	start and stop
       the collection of the statistics	data for TABLE_STATISTICS and similar.

SERVER CONNECTIONS
       When you	create a server	connection using '@', innotop asks you for a
       series of inputs, as follows:

       DSN A DSN is a Data Source Name,	which is the initial argument passed
	   to the DBI module for connecting to a server.  It is	usually	of the
	   form

	    DBI:mysql:;mysql_read_default_group=mysql;host=HOSTNAME

	   Since this DSN is passed to the DBD::mysql driver, you should read
	   the driver's	documentation at
	   "/search.cpan.org/dist/DBD-mysql/lib/DBD/mysql.pm"" in "http: for
	   the exact details on	all the	options	you can	pass the driver	in the
	   DSN.	 You can read more about DBI at	<http://dbi.perl.org/docs/>,
	   and especially at <http://search.cpan.org/~timb/DBI/DBI.pm>.

	   The mysql_read_default_group=mysql option lets the DBD driver read
	   your	MySQL options files, such as ~/.my.cnf on UNIX-ish systems.
	   You can use this to avoid specifying	a username or password for the
	   connection.

       InnoDB Deadlock Table
	   This	optional item tells innotop a table name it can	use to
	   deliberately	create a small deadlock	(see "D: InnoDB	Deadlocks").
	   If you specify this option, you just	need to	be sure	the table
	   doesn't exist, and that innotop can create and drop the table with
	   the InnoDB storage engine.  You can safely omit or just accept the
	   default if you don't	intend to use this.

       Username
	   innotop will	ask you	if you want to specify a username.  If you say
	   'y',	it will	then prompt you	for a user name.  If you have a	MySQL
	   option file that specifies your username, you don't have to specify
	   a username.

	   The username	defaults to your login name on the system you're
	   running innotop on.

       Password
	   innotop will	ask you	if you want to specify a password.  Like the
	   username, the password is optional, but there's an additional
	   prompt that asks if you want	to save	the password in	the innotop
	   configuration file.	If you don't save it in	the configuration
	   file, innotop will prompt you for a password	each time it starts.
	   Passwords in	the innotop configuration file are saved in plain
	   text, not encrypted in any way.

       Once you	finish answering these questions, you should be	connected to a
       server.	But innotop isn't limited to monitoring	a single server; you
       can define many server connections and switch between them by pressing
       the '@' key.  See "SWITCHING BETWEEN CONNECTIONS".

SERVER GROUPS
       If you have multiple MySQL instances, you can put them into named
       groups, such as 'all', 'masters', and 'slaves', which innotop can
       monitor all together.

       You can choose which group to monitor with the '#' key, and you can
       press the TAB key to switch to the next group.  If you're not currently
       monitoring a group, pressing TAB	selects	the first group.

       To create a group, press	the '#'	key and	type the name of your new
       group, then type	the names of the connections you want the group	to
       contain.

SWITCHING BETWEEN CONNECTIONS
       innotop lets you	quickly	switch which servers you're monitoring.	 The
       most basic way is by pressing the '@' key and typing the	name(s)	of the
       connection(s) you want to use.  This setting is per-mode, so you	can
       monitor different connections in	each mode, and innotop remembers which
       connections you choose.

       You can quickly switch to the 'next' connection in alphabetical order
       with the	'n' key.  If you're monitoring a server	group (see "SERVER
       GROUPS")	this will switch to the	first connection.

       You can also type many connection names,	and innotop will fetch and
       display data from them all.  Just separate the connection names with
       spaces, for example "server1 server2."  Again, if you type the name of
       a connection that doesn't exist,	innotop	will prompt you	for connection
       information and create the connection.

       Another way to monitor multiple connections at once is with server
       groups.	You can	use the	TAB key	to switch to the 'next'	group in
       alphabetical order, or if you're	not monitoring any groups, TAB will
       switch to the first group.

       innotop does not	fetch data in parallel from connections, so if you are
       monitoring a large group	or many	connections, you may notice increased
       delay between ticks.

       When you	monitor	more than one connection, innotop's status bar
       changes.	 See "INNOTOP STATUS".

ERROR HANDLING
       Error handling is not that important when monitoring a single
       connection, but is crucial when you have	many active connections.  A
       crashed server or lost connection should	not crash innotop.  As a
       result, innotop will continue to	run even when there is an error; it
       just won't display any information from the connection that had an
       error.  Because of this,	innotop's behavior might confuse you.  It's a
       feature,	not a bug!

       innotop does not	continue to query connections that have	errors,
       because they may	slow innotop and make it hard to use, especially if
       the error is a problem connecting and causes a long time-out.  Instead,
       innotop retries the connection occasionally to see if the error still
       exists.	If so, it will wait until some point in	the future.  The wait
       time increases in ticks as the Fibonacci	series,	so it tries less
       frequently as time passes.

       Since errors might only happen in certain modes because of the SQL
       commands	issued in those	modes, innotop keeps track of which mode
       caused the error.  If you switch	to a different mode, innotop will
       retry the connection instead of waiting.

       By default innotop will display the problem in red text at the bottom
       of the first table on the screen.  You can disable this behavior	with
       the "show_cxn_errors_in_tbl" configuration option, which	is enabled by
       default.	 If the	"debug"	option is enabled, innotop will	display	the
       error at	the bottom of every table, not just the	first.	And if
       "show_cxn_errors" is enabled, innotop will print	the error text to
       STDOUT as well.	Error messages might only display in the mode that
       caused the error, depending on the mode and whether innotop is avoiding
       querying	that connection.

NON-INTERACTIVE	OPERATION
       You can run innotop in non-interactive mode, in which case it is
       entirely	controlled from	the configuration file and command-line
       options.	 To start innotop in non-interactive mode, give	the
       L"<--nonint"> command-line option.  This	changes	innotop's behavior in
       the following ways:

       o   Certain Perl	modules	are not	loaded.	 Term::Readline	is not loaded,
	   since innotop doesn't prompt	interactively.	Term::ANSIColor	and
	   Win32::Console::ANSI	modules	are not	loaded.	 Term::ReadKey is
	   still used, since innotop may have to prompt	for connection
	   passwords when starting up.

       o   innotop does	not clear the screen after each	tick.

       o   innotop does	not persist any	changes	to the configuration file.

       o   If "--count"	is given and innotop is	in incremental mode (see
	   "status_inc"	and "--inc"), innotop actually refreshes one more time
	   than	specified so it	can print incremental statistics.  This
	   suppresses output during the	first tick, so innotop may appear to
	   hang.

       o   innotop only	displays the first table in each mode.	This is	so the
	   output can be easily	processed with other command-line utilities
	   such	as awk and sed.	 To change which tables	display	in each	mode,
	   see "TABLES".  Since	"Q: Query List"	mode is	so important, innotop
	   automatically disables the "q_header" table.	 This ensures you'll
	   see the "processlist" table,	even if	you have innotop configured to
	   show	the q_header table during interactive operation.  Similarly,
	   in "T: InnoDB Transactions" mode, the "t_header" table is
	   suppressed so you see only the "innodb_transactions"	table.

       o   All output is tab-separated instead of being	column-aligned with
	   whitespace, and innotop prints the full contents of each table
	   instead of only printing one	screenful at a time.

       o   innotop only	prints column headers once instead of every tick (see
	   "hide_hdr").	 innotop does not print	table captions (see
	   "display_table_captions").  innotop ensures there are no empty
	   lines in the	output.

       o   innotop does	not honor the "shorten"	transformation,	which normally
	   shortens some numbers to human-readable formats.

       o   innotop does	not print a status line	(see "INNOTOP STATUS").

CONFIGURING
       Nearly everything about innotop is configurable.	 Most things are
       possible	to change with built-in	commands, but you can also edit	the
       configuration file.

       While running innotop, press the	'$' key	to bring up the	configuration
       editing dialog.	Press another key to select the	type of	data you want
       to edit:

       S: Statement Sleep Times
	   Edits SQL statement sleep delays, which make	innotop	pause for the
	   specified amount of time after executing a statement.  See "SQL
	   STATEMENTS" for a definition	of each	statement and what it does.
	   By default innotop does not delay after any statements.

	   This	feature	is included so you can customize the side-effects
	   caused by monitoring	your server.  You may not see any effects, but
	   some	innotop	users have noticed that	certain	MySQL versions under
	   very	high load with InnoDB enabled take longer than usual to
	   execute SHOW	GLOBAL STATUS.	If innotop calls SHOW FULL PROCESSLIST
	   immediately afterward, the processlist contains more	queries	than
	   the machine actually	averages at any	given moment.  Configuring
	   innotop to pause briefly after calling SHOW GLOBAL STATUS
	   alleviates this effect.

	   Sleep times are stored in the "stmt_sleep_times" section of the
	   configuration file.	Fractional-second sleeps are supported,
	   subject to your hardware's limitations.

       c: Edit Columns
	   Starts the table editor on one of the displayed tables.  See	"TABLE
	   EDITOR".  An	alternative way	to start the table editor without
	   entering the	configuration dialog is	with the '^' key.

       g: General Configuration
	   Starts the configuration editor to edit global and mode-specific
	   configuration variables (see	"MODES").  innotop prompts you to
	   choose a variable from among	the global and mode-specific ones
	   depending on	the current mode.

       k: Row-Coloring Rules
	   Starts the row-coloring rules editor	on one of the displayed
	   table(s).  See "COLORS" for details.

       p: Manage Plugins
	   Starts the plugin configuration editor.  See	"PLUGINS" for details.

       s: Server Groups
	   Lets	you create and edit server groups.  See	"SERVER	GROUPS".

       t: Choose Displayed Tables
	   Lets	you choose which tables	to display in this mode.  See "MODES"
	   and "TABLES".

CONFIGURATION FILE
       innotop's default configuration file locations are $HOME/.innotop and
       /usr/local/etc/innotop.conf, and	they are looked	for in that order.  If
       the first configuration file exists, the	second will not	be processed.
       Those can be overridden with the	"--config" command-line	option.	 You
       can edit	it by hand safely, however innotop reads the configuration
       file when it starts, and, if readonly is	set to 0, writes it out	again
       when it exits.  Thus, if	readonly is set	to 0, any changes you make by
       hand while innotop is running will be lost.

       innotop doesn't store its entire	configuration in the configuration
       file.  It has a huge set	of default configuration values	that it	holds
       only in memory, and the configuration file only overrides these
       defaults.  When you customize a default setting,	innotop	notices, and
       then stores the customizations into the file.  This keeps the file size
       down, makes it easier to	edit, and makes	upgrades easier.

       A configuration file is read-only be default.  You can override that
       with "--write".	See "readonly".

       The configuration file is arranged into sections	like an	INI file.
       Each section begins with	[section-name] and ends	with [/section-name].
       Each section's entries have a different syntax depending	on the data
       they need to store.  You	can put	comments in the	file; any line that
       begins with a # character is a comment.	innotop	will not read the
       comments, so it won't write them	back out to the	file when it exits.
       Comments	in read-only configuration files are still useful, though.

       The first line in the file is innotop's version number.	This lets
       innotop notice when the file format is not backwards-compatible,	and
       upgrade smoothly	without	destroying your	customized configuration.

       The following list describes each section of the	configuration file and
       the data	it contains:

       general
	   The 'general' section contains global configuration variables and
	   variables that may be mode-specific,	but don't belong in any	other
	   section.  The syntax	is a simple key=value list.  innotop writes a
	   comment above each value to help you	edit the file by hand.

	   S_func
	       Controls	S mode presentation (see "S: Variables & Status").  If
	       g, values are graphed; if s, values are like vmstat; if p,
	       values are in a pivoted table.

	   S_set
	       Specifies which set of variables	to display in "S: Variables &
	       Status" mode.  See "VARIABLE SETS".

	   auto_wipe_dl
	       Instructs innotop to automatically wipe large deadlocks when it
	       notices them.  When this	happens	you may	notice a slight	delay.
	       At the next tick, you will usually see the information that was
	       being truncated by the large deadlock.

	   charset
	       Specifies what kind of characters to allow through the
	       "no_ctrl_char" transformation.  This keeps non-printable
	       characters from confusing a terminal when you monitor queries
	       that contain binary data, such as images.

	       The default is 'ascii', which considers anything	outside	normal
	       ASCII to	be a control character.	 The other allowable values
	       are 'unicode' and 'none'.  'none' considers every character a
	       control character, which	can be useful for collapsing ALL text
	       fields in queries.

	   cmd_filter
	       This is the prefix that filters variables in "C:	Command
	       Summary"	mode.

	   color
	       Whether terminal	coloring is permitted.

	   cxn_timeout
	       On MySQL	versions 4.0.3 and newer, this variable	is used	to set
	       the connection's	timeout, so MySQL doesn't close	the connection
	       if it is	not used for a while.  This might happen because a
	       connection isn't	monitored in a particular mode,	for example.

	   debug
	       This option enables more	verbose	errors and makes innotop more
	       strict in some places.  It can help in debugging	filters	and
	       other user-defined code.	 It also makes innotop write a lot of
	       information to "debugfile" when there is	a crash.

	   debugfile
	       A file to which innotop will write information when there is a
	       crash.  See "FILES".

	   display_table_captions
	       innotop displays	a table	caption	above most tables.  This
	       variable	suppresses or shows captions on	all tables globally.
	       Some tables are configured with the hide_caption	property,
	       which overrides this.

	   global
	       Whether to show GLOBAL variables	and status.  innotop only
	       tries to	do this	on servers which support the GLOBAL option to
	       SHOW VARIABLES and SHOW STATUS.	In some	MySQL versions,	you
	       need certain privileges to do this; if you don't	have them,
	       innotop will not	be able	to fetch any variable and status data.
	       This configuration variable lets	you run	innotop	and fetch what
	       data you	can even without the elevated privileges.

	       I can no	longer find or reproduce the situation where GLOBAL
	       wasn't allowed, but I know there	was one.

	   graph_char
	       Defines the character to	use when drawing graphs	in "S:
	       Variables & Status" mode.

	   header_highlight
	       Defines how to highlight	column headers.	 This only works if
	       Term::ANSIColor is available.  Valid values are 'bold' and
	       'underline'.

	   hide_hdr
	       Hides column headers globally.

	   interval
	       The interval at which innotop will refresh its data (ticks).
	       The interval is implemented as a	sleep time between ticks, so
	       the true	interval will vary depending on	how long it takes
	       innotop to fetch	and render data.

	       This variable accepts fractions of a second.

	   mode
	       The mode	in which innotop should	start.	Allowable arguments
	       are the same as the key presses that select a mode
	       interactively.  See "MODES".

	   num_digits
	       How many	digits to show in fractional numbers and percents.
	       This variable's range is	between	0 and 9	and can	be set
	       directly	from "S: Variables & Status" mode with the '+' and '-'
	       keys.  It is used in the	"set_precision", "shorten", and
	       "percent" transformations.

	   num_status_sets
	       Controls	how many sets of status	variables to display in
	       pivoted "S: Variables & Status" mode.  It also controls the
	       number of old sets of variables innotop keeps in	its memory, so
	       the larger this variable	is, the	more memory innotop uses.

	   plugin_dir
	       Specifies where plugins can be found.  By default, innotop
	       stores plugins in the 'plugins' subdirectory of your innotop
	       configuration directory.

	   readonly
	       Whether the configuration file is readonly.  This cannot	be set
	       interactively.

	   show_cxn_errors
	       Makes innotop print connection errors to	STDOUT.	 See "ERROR
	       HANDLING".

	   show_cxn_errors_in_tbl
	       Makes innotop display connection	errors as rows in the first
	       table on	screen.	 See "ERROR HANDLING".

	   show_percent
	       Adds a '%' character after the value returned by	the "percent"
	       transformation.

	   show_statusbar
	       Controls	whether	to show	the status bar in the display.	See
	       "INNOTOP	STATUS".

	   skip_innodb
	       Disables	fetching SHOW INNODB STATUS, in	case your server(s) do
	       not have	InnoDB enabled and you don't want innotop to try to
	       fetch it.  This can also	be useful when you don't have the
	       SUPER privilege,	required to run	SHOW INNODB STATUS.

	   spark
	       Specifies how wide a spark chart	is. There are two ASCII	spark
	       charts in A mode, showing QPS and User_threads_running.

	   status_inc
	       Whether to show absolute	or incremental values for status
	       variables.  Incremental values are calculated as	an offset from
	       the last	value innotop saw for that variable.  This is a	global
	       setting,	but will probably become mode-specific at some point.
	       Right now it is honored a bit inconsistently; some modes	don't
	       pay attention to	it.

	   timeformat
	       The C-style strftime()-compatible format	for the	timestamp line
	       to be printed in	-n mode	when -t	is set.

       plugins
	   This	section	holds a	list of	package	names of active	plugins.  If
	   the plugin exists, innotop will activate it.	 See "PLUGINS" for
	   more	information.

       filters
	   This	section	holds user-defined filters (see	"FILTERS").  Each line
	   is in the format filter_name=text='filter text' tbls='table list'.

	   The filter text is the text of the subroutine's code.  The table
	   list	is a list of tables to which the filter	can apply.  By
	   default, user-defined filters apply to the table for	which they
	   were	created, but you can manually override that by editing the
	   definition in the configuration file.

       active_filters
	   This	section	stores which filters are active	on each	table.	Each
	   line	is in the format table_name=filter_list.

       tbl_meta
	   This	section	stores user-defined or user-customized columns (see
	   "COLUMNS").	Each line is in	the format col_name=properties,	where
	   the properties are a	name=quoted-value list.

       connections
	   This	section	holds the server connections you have defined.	Each
	   line	is in the format name=properties, where	the properties are a
	   name=value list.  The properties are	self-explanatory, and the only
	   one that is treated specially is 'pass' which is only present if
	   'savepass' is set.  This section of the configuration file will be
	   skipped if any DSN, username, or password command-line options are
	   used.  See "SERVER CONNECTIONS".

       active_connections
	   This	section	holds a	list of	which connections are active in	each
	   mode.  Each line is in the format mode_name=connection_list.

       server_groups
	   This	section	holds server groups.  Each line	is in the format
	   name=connection_list.  See "SERVER GROUPS".

       active_server_groups
	   This	section	holds a	list of	which server group is active in	each
	   mode.  Each line is in the format mode_name=server_group.

       max_values_seen
	   This	section	holds the maximum values seen for variables.  This is
	   used	to scale the graphs in "S: Variables & Status" mode.  Each
	   line	is in the format name=value.

       active_columns
	   This	section	holds table column lists.  Each	line is	in the format
	   tbl_name=column_list.  See "COLUMNS".

       sort_cols
	   This	section	holds the sort definition.  Each line is in the	format
	   tbl_name=column_list.  If a column is prefixed with '-', that
	   column sorts	descending.  See "SORTING".

       visible_tables
	   This	section	defines	which tables are visible in each mode.	Each
	   line	is in the format mode_name=table_list.	See "TABLES".

       varsets
	   This	section	defines	variable sets for use in "S: Status &
	   Variables" mode.  Each line is in the format	name=variable_list.
	   See "VARIABLE SETS".

       colors
	   This	section	defines	colorization rules.  Each line is in the
	   format tbl_name=property_list.  See "COLORS".

       stmt_sleep_times
	   This	section	contains statement sleep times.	 Each line is in the
	   format statement_name=sleep_time.  See "S: Statement	Sleep Times".

       group_by
	   This	section	contains column	lists for table	group_by expressions.
	   Each	line is	in the format tbl_name=column_list.  See "GROUPING".

CUSTOMIZING
       You can customize innotop a great deal.	For example, you can:

       o   Choose which	tables to display, and in what order.

       o   Choose which	columns	are in those tables, and create	new columns.

       o   Filter which	rows display with built-in filters, user-defined
	   filters, and	quick-filters.

       o   Sort	the rows to put	important data first or	group together related
	   rows.

       o   Highlight rows with color.

       o   Customize the alignment, width, and formatting of columns, and
	   apply transformations to columns to extract parts of	their values
	   or format the values	as you wish (for example, shortening large
	   numbers to familiar units).

       o   Design your own expressions to extract and combine data as you
	   need.  This gives you unlimited flexibility.

       All these and more are explained	in the following sections.

   TABLES
       A table is what you'd expect: a collection of columns.  It also has
       some other properties, such as a	caption.  Filters, sorting rules, and
       colorization rules belong to tables and are covered in later sections.

       Internally, table meta-data is defined in a data	structure called
       %tbl_meta.  This	hash holds all built-in	table definitions, which
       contain a lot of	default	instructions to	innotop.  The meta-data
       includes	the caption, a list of columns the user	has customized,	a list
       of columns, a list of visible columns, a	list of	filters, color rules,
       a sort-column list, sort	direction, and some information	about the
       table's data sources.  Most of this is customizable via the table
       editor (see "TABLE EDITOR").

       You can choose which tables to show by pressing the '$' key.  See
       "MODES" and "TABLES".

       The table life-cycle is as follows:

       o   Each	table begins with a data source, which is an array of hashes.
	   See below for details on data sources.

       o   Each	element	of the data source becomes a row in the	final table.

       o   For each element in the data	source,	innotop	extracts values	from
	   the source and creates a row.  This row is another hash, which
	   later steps will refer to as	$set.  The values innotop extracts are
	   determined by the table's columns.  Each column has an extraction
	   subroutine, compiled	from an	expression (see	"EXPRESSIONS").	 The
	   resulting row is a hash whose keys are named	the same as the	column
	   name.

       o   innotop filters the rows, removing those that don't need to be
	   displayed.  See "FILTERS".

       o   innotop sorts the rows.  See	"SORTING".

       o   innotop groups the rows together, if	specified.  See	"GROUPING".

       o   innotop colorizes the rows.	See "COLORS".

       o   innotop transforms the column values	in each	row.  See
	   "TRANSFORMATIONS".

       o   innotop optionally pivots the rows (see "PIVOTING"),	then filters
	   and sorts them.

       o   innotop formats and justifies the rows as a table.  During this
	   step, innotop applies further formatting to the column values,
	   including alignment,	maximum	and minimum widths.  innotop also does
	   final error checking	to ensure there	are no crashes due to
	   undefined values.  innotop then adds	a caption if specified,	and
	   the table is	ready to print.

       The lifecycle is	slightly different if the table	is pivoted, as noted
       above.  To clarify, if the table	is pivoted, the	process	is extract,
       group, transform, pivot,	filter,	sort, create.  If it's not pivoted,
       the process is extract, filter, sort, group, color, transform, create.
       This slightly convoluted	process	doesn't	map all	that well to SQL, but
       pivoting	complicates things pretty thoroughly.  Roughly speaking,
       filtering and sorting happen as late as needed to effect	the final
       result as you might expect, but as early	as possible for	efficiency.

       Each built-in table is described	below:

       adaptive_hash_index
	   Displays data about InnoDB's	adaptive hash index.  Data source:
	   "STATUS_VARIABLES".

       buffer_pool
	   Displays data about InnoDB's	buffer pool.  Data source:
	   "STATUS_VARIABLES".

       cmd_summary
	   Displays weighted status variables.	Data source:
	   "STATUS_VARIABLES".

       deadlock_locks
	   Shows which locks were held and waited for by the last detected
	   deadlock.  Data source: "DEADLOCK_LOCKS".

       deadlock_transactions
	   Shows transactions involved in the last detected deadlock.  Data
	   source: "DEADLOCK_TRANSACTIONS".

       explain
	   Shows the output of EXPLAIN.	 Data source: "EXPLAIN".

       file_io_misc
	   Displays data about InnoDB's	file and I/O operations.  Data source:
	   "STATUS_VARIABLES".

       fk_error
	   Displays various data about InnoDB's	last foreign key error.	 Data
	   source: "STATUS_VARIABLES".

       health_dashboard
	   Displays an overall summary of servers, one server per line,	for
	   monitoring.	Data source: "STATUS_VARIABLES", "MASTER_SLAVE",
	   "PROCESSLIST_STATS".

       index_statistics
	   Displays data from the INDEX_STATISTICS table in Percona-enhanced
	   servers.

       index_table_statistics
	   Displays data from the INDEX_STATISTICS and TABLE_STATISTICS	tables
	   in Percona-enhanced servers.	 It joins the two together, grouped by
	   the database	and table name.	 It is the default view	in "U: User
	   Statistics" mode, and makes it easy to see what tables are hot, how
	   many	rows are read from indexes, how	many changes are made, and how
	   many	changes	are made to indexes.

       innodb_blocked_blocker
	   Displays InnoDB locks and lock waits. Data source:
	   "INNODB_BLOCKED_BLOCKER".

       innodb_locks
	   Displays InnoDB locks.  Data	source:	"INNODB_LOCKS".

       innodb_transactions
	   Displays data about InnoDB's	current	transactions.  Data source:
	   "INNODB_TRANSACTIONS".

       insert_buffers
	   Displays data about InnoDB's	insert buffer.	Data source:
	   "STATUS_VARIABLES".

       io_threads
	   Displays data about InnoDB's	I/O threads.  Data source:
	   "IO_THREADS".

       log_statistics
	   Displays data about InnoDB's	logging	system.	 Data source:
	   "STATUS_VARIABLES".

       master_status
	   Displays replication	master status.	Data source:
	   "STATUS_VARIABLES".

       open_tables
	   Displays open tables.  Data source: "OPEN_TABLES".

       page_statistics
	   Displays InnoDB page	statistics.  Data source: "STATUS_VARIABLES".

       pending_io
	   Displays InnoDB pending I/O operations.  Data source:
	   "STATUS_VARIABLES".

       processlist
	   Displays current MySQL processes (threads/connections).  Data
	   source: "PROCESSLIST".

       q_header
	   Displays various status values.  Data source: "STATUS_VARIABLES".

       row_operation_misc
	   Displays data about InnoDB's	row operations.	 Data source:
	   "STATUS_VARIABLES".

       row_operations
	   Displays data about InnoDB's	row operations.	 Data source:
	   "STATUS_VARIABLES".

       semaphores
	   Displays data about InnoDB's	semaphores and mutexes.	 Data source:
	   "STATUS_VARIABLES".

       slave_io_status
	   Displays data about the slave I/O thread.  Data source:
	   "STATUS_VARIABLES".

       slave_sql_status
	   Displays data about the slave SQL thread.  Data source:
	   "STATUS_VARIABLES".

       table_statistics
	   Displays data from the TABLE_STATISTICS table in Percona-enhanced
	   servers.

       t_header
	   Displays various InnoDB status values.  Data	source:
	   "STATUS_VARIABLES".

       var_status
	   Displays user-configurable data.  Data source: "STATUS_VARIABLES".

       wait_array
	   Displays data about InnoDB's	OS wait	array.	Data source:
	   "OS_WAIT_ARRAY".

   COLUMNS
       Columns belong to tables.  You can choose a table's columns by pressing
       the '^' key, which starts the "TABLE EDITOR" and	lets you choose	and
       edit columns.  Pressing 'e' from	within the table editor	lets you edit
       the column's properties:

       o   hdr:	a column header.  This appears in the first row	of the table.

       o   just: justification.	 '-' means left-justified and '' means right-
	   justified, just as with printf formatting codes (not	a
	   coincidence).

       o   dec:	whether	to further align the column on the decimal point.

       o   num:	whether	the column is numeric.	This affects how values	are
	   sorted (lexically or	numerically).

       o   label: a small note about the column, which appears in dialogs that
	   help	the user choose	columns.

       o   src:	an expression that innotop uses	to extract the column's	data
	   from	its source (see	"DATA SOURCES").  See "EXPRESSIONS" for	more
	   on expressions.

       o   minw: specifies a minimum display width.  This helps	stabilize the
	   display, which makes	it easier to read if the data is changing
	   frequently.

       o   maxw: similar to minw.

       o   trans: a list of column transformations.  See "TRANSFORMATIONS".

       o   agg:	an aggregate function.	See "GROUPING".	 The default is
	   "first".

       o   aggonly: controls whether the column	only shows when	grouping is
	   enabled on the table	(see "GROUPING").  By default, this is
	   disabled.  This means columns will always be	shown by default,
	   whether grouping is enabled or not.	If a column's aggonly is set
	   true, the column will appear	when you toggle	grouping on the	table.
	   Several columns are set this	way, such as the count column on
	   "processlist" and "innodb_transactions", so you don't see a count
	   when	the grouping isn't enabled, but	you do when it is.

       o   agghide: the	reverse	of aggonly.  The column	is hidden when
	   grouping is enabled.

   FILTERS
       Filters remove rows from	the display.  They behave much like a WHERE
       clause in SQL.  innotop has several built-in filters, which remove
       irrelevant information like inactive queries, but you can define	your
       own as well.  innotop also lets you create quick-filters, which do not
       get saved to the	configuration file, and	are just an easy way to
       quickly view only some rows.

       You can enable or disable a filter on any table.	 Press the '%' key
       (mnemonic: % looks kind of like a line being filtered between two
       circles)	and choose which table you want	to filter, if asked.  You'll
       then see	a list of possible filters and a list of filters currently
       enabled for that	table.	Type the names of filters you want to apply
       and press Enter.

       USER-DEFINED FILTERS

       If you type a name that doesn't exist, innotop will prompt you to
       create the filter.  Filters are easy to create if you know Perl,	and
       not hard	if you don't.  What you're doing is creating a subroutine that
       returns true if the row should be displayed.  The row is	a hash
       reference passed	to your	subroutine as $set.

       For example, imagine you	want to	filter the processlist table so	you
       only see	queries	that have been running more than five minutes.	Type a
       new name	for your filter, and when prompted for the subroutine body,
       press TAB to initiate your terminal's auto-completion.  You'll see the
       names of	the columns in the "processlist" table (innotop	generally
       tries to	help you with auto-completion lists).  You want	to filter on
       the 'time' column.  Type	the text "$set->{time} > 300" to return	true
       when the	query is more than five	minutes	old.  That's all you need to
       do.

       In other	words, the code	you're typing is surrounded by an implicit
       context,	which looks like this:

	sub filter {
	   my (	$set ) = @_;
	   # YOUR CODE HERE
	}

       If your filter doesn't work, or if something else suddenly behaves
       differently, you	might have made	an error in your filter, and innotop
       is silently catching the	error.	Try enabling "debug" to	make innotop
       throw an	error instead.

       QUICK-FILTERS

       innotop's quick-filters are a shortcut to create	a temporary filter
       that doesn't persist when you restart innotop.  To create a quick-
       filter, press the '/' key.  innotop will	prompt you for the column name
       and filter text.	 Again,	you can	use auto-completion on column names.
       The filter text can be just the text you	want to	"search	for."  For
       example,	to filter the "processlist" table on queries that refer	to the
       products	table, type '/'	and then 'info product'.  Internally, the
       filter is compiled into a subroutine like this:

	sub filter {
	   my (	$set ) = @_;
	   $set->{info}	=~ m/product/;
	}

       The filter text can actually be any Perl	regular	expression, but	of
       course a	literal	string like 'product' works fine as a regular
       expression.

       What if you want	the filter to discard matching rows, rather than
       showing matching	rows?  If you're familiar with Perl regular
       expressions, you	might guess how	to do this.  You have to use a zero-
       width negative lookahead	assertion.  If you don't know what that	means,
       don't worry.  Let's filter out all rows where the command is Gandalf.
       Type the	following:

	1. /
	2. cmd ^(?!Gandalf)

       Behind the scenes innotop compiles the quick-filter into	a specially
       tagged filter that is otherwise like any	other filter.  It just isn't
       saved to	the configuration file.

       To clear	quick-filters, press the '\' key and innotop will clear	them
       all at once.

   SORTING
       innotop has sensible built-in defaults to sort the most important rows
       to the top of the table.	 Like anything else in innotop,	you can
       customize how any table is sorted.

       To start	the sort dialog, start the "TABLE EDITOR" with the '^' key,
       choose a	table if necessary, and	press the 's' key.  You'll see a list
       of columns you can use in the sort expression and the current sort
       expression, if any.  Enter a list of columns by which you want to sort
       and press Enter.	 If you	want to	reverse	sort, prefix the column	name
       with a minus sign.  For example,	if you want to sort by column a
       ascending, then column b	descending, type 'a -b'.  You can also
       explicitly add a	+ in front of columns you want to sort ascending, but
       it's not	required.

       Some modes have keys mapped to open this	dialog directly, and to
       quickly reverse sort direction.	Press '?' as usual to see which	keys
       are mapped in any mode.

   GROUPING
       innotop can group, or aggregate,	rows together (the terms are used
       interchangeably).  This is quite	similar	to an SQL GROUP	BY clause.
       You can specify to group	on certain columns, or if you don't specify
       any, the	entire set of rows is treated as one group.  This is quite
       like SQL	so far,	but unlike SQL,	you can	also select un-grouped
       columns.	 innotop actually aggregates every column.  If you don't
       explicitly specify a grouping function, the default is 'first'.	This
       is basically a convenience so you don't have to specify an aggregate
       function	for every column you want in the result.

       You can quickly toggle grouping on a table with the '=' key, which
       toggles its aggregate property.	This property doesn't persist to the
       config file.

       The columns by which the	table is grouped are specified in its group_by
       property.  When you turn	grouping on, innotop places the	group_by
       columns at the far left of the table, even if they're not supposed to
       be visible.  The	rest of	the visible columns appear in order after
       them.

       Two tables have default group_by	lists and a count column built in:
       "processlist" and "innodb_transactions".	 The grouping is by connection
       and status, so you can quickly see how many queries or transactions are
       in a given status on each server	you're monitoring.  The	time columns
       are aggregated as a sum;	other columns are left at the default 'first'
       aggregation.

       By default, the table shown in "S: Variables & Status" mode also	uses
       grouping	so you can monitor variables and status	across many servers.
       The default aggregation function	in this	mode is	'avg'.

       Valid grouping functions	are defined in the %agg_funcs hash.  They
       include

       first
	   Returns the first element in	the group.

       count
	   Returns the number of elements in the group,	including undefined
	   elements, much like SQL's COUNT(*).

       avg Returns the average of defined elements in the group.

       sum Returns the sum of elements in the group.

       Here's an example of grouping at	work.  Suppose you have	a very busy
       server with hundreds of open connections, and you want to see how many
       connections are in what status.	Using the built-in grouping rules, you
       can press 'Q' to	enter "Q: Query	List" mode.  Press '=' to toggle
       grouping	(if necessary, select the "processlist"	table when prompted).

       Your display might now look like	the following:

	Query List (? for help)	localhost, 32:33, 0.11 QPS, 1 thd, 5.0.38-log

	CXN	   Cmd	      Cnt  ID	   User	  Host		 Time	Query
	localhost  Query      49    12933  webusr localhost	 19:38	SELECT * FROM
	localhost  Sending Da 23     2383  webusr localhost	 12:43	SELECT col1,
	localhost  Sleep      120     140  webusr localhost    5:18:12
	localhost  Statistics 12    19213  webusr localhost	 01:19	SELECT * FROM

       That's actually quite a worrisome picture.  You've got a	lot of idle
       connections (Sleep), and	some connections executing queries (Query and
       Sending Data).  That's okay, but	you also have a	lot in Statistics
       status, collectively spending over a minute.  That means	the query
       optimizer is having a really hard time generating execution plans for
       your statements.	 Something is wrong; it	should normally	take
       milliseconds to plan queries.  You might	not have seen this pattern if
       you didn't look at your connections in aggregate.  (This	is a made-up
       example,	but it can happen in real life).

   PIVOTING
       innotop can pivot a table for more compact display, similar to a	Pivot
       Table in	a spreadsheet (also known as a crosstab).  Pivoting a table
       makes columns into rows.	 Assume	you start with this table:

	foo bar
	=== ===
	1   3
	2   4

       After pivoting, the table will look like	this:

	name set0 set1
	==== ==== ====
	foo  1	  2
	bar  3	  4

       To get reasonable results, you might need to group as well as pivoting.
       innotop currently does this for "S: Variables & Status" mode.

   COLORS
       By default, innotop highlights rows with	color so you can see at	a
       glance which rows are more important.  You can customize	the
       colorization rules and add your own to any table.  Open the table
       editor with the '^' key,	choose a table if needed, and press 'o'	to
       open the	color editor dialog.

       The color editor	dialog displays	the rules applied to the table,	in the
       order they are evaluated.  Each row is evaluated	against	each rule to
       see if the rule matches the row;	if it does, the	row gets the specified
       color, and no further rules are evaluated.  The rules look like the
       following:

	state  eq  Locked	black on_red
	cmd    eq  Sleep	white
	user   eq  system user	white
	cmd    eq  Connect	white
	cmd    eq  Binlog Dump	white
	time   >   600		red
	time   >   120		yellow
	time   >   60		green
	time   >   30		cyan

       This is the default rule	set for	the "processlist" table.  In order of
       priority, these rules make locked queries black on a red	background,
       "gray out" connections from replication and sleeping queries, and make
       queries turn from cyan to red as	they run longer.

       (For some reason, the ANSI color	code "white" is	actually a light gray.
       Your terminal's display may vary; experiment to find colors you like).

       You can use keystrokes to move the rules	up and down, which re-orders
       their priority.	You can	also delete rules and add new ones.  If	you
       add a new rule, innotop prompts you for the column, an operator for the
       comparison, a value against which to compare the	column,	and a color to
       assign if the rule matches.  There is auto-completion and prompting at
       each step.

       The value in the	third step needs to be correctly quoted.  innotop does
       not try to quote	the value because it doesn't know whether it should
       treat the value as a string or a	number.	 If you	want to	compare	the
       column against a	string,	as for example in the first rule above,	you
       should enter 'Locked' surrounded	by quotes.  If you get an error
       message about a bareword, you probably should have quoted something.

   EXPRESSIONS
       Expressions are at the core of how innotop works, and are what enables
       you to extend innotop as	you wish.  Recall the table lifecycle
       explained in "TABLES".  Expressions are used in the earliest step,
       where it	extracts values	from a data source to form rows.

       It does this by calling a subroutine for	each column, passing it	the
       source data set,	a set of current values, and a set of previous values.
       These are all needed so the subroutine can calculate things like	the
       difference between this tick and	the previous tick.

       The subroutines that extract the	data from the set are compiled from
       expressions.  This gives	significantly more power than just naming the
       values to fill the columns, because it allows the column's value	to be
       calculated from whatever	data is	necessary, but avoids the need to
       write complicated and lengthy Perl code.

       innotop begins with a string of text that can look as simple as a
       value's name or as complicated as a full-fledged	Perl expression.  It
       looks at	each 'bareword'	token in the string and	decides	whether	it's
       supposed	to be a	key into the $set hash.	 A bareword is an unquoted
       value that isn't	already	surrounded by code-ish things like dollar
       signs or	curly brackets.	 If innotop decides that the bareword isn't a
       function	or other valid Perl code, it converts it into a	hash access.
       After the whole string is processed, innotop compiles a subroutine,
       like this:

	sub compute_column_value {
	   my (	$set, $cur, $pre ) = @_;
	   my $val = # EXPANDED	STRING GOES HERE
	   return $val;
	}

       Here's a	concrete example, taken	from the header	table "q_header" in
       "Q: Query List" mode.  This expression calculates the qps, or Queries
       Per Second, column's values, from the values returned by	SHOW STATUS:

	Questions/Uptime_hires

       innotop decides both words are barewords, and transforms	this
       expression into the following Perl code:

	$set->{Questions}/$set->{Uptime_hires}

       When surrounded by the rest of the subroutine's code, this is
       executable Perl that calculates a high-resolution queries-per-second
       value.

       The arguments to	the subroutine are named $set, $cur, and $pre.	In
       most cases, $set	and $cur will be the same values.  However, if
       "status_inc" is set, $cur will not be the same as $set, because $set
       will already contain values that	are the	incremental difference between
       $cur and	$pre.

       Every column in innotop is computed by subroutines compiled in the same
       fashion.	 There is no difference	between	innotop's built-in columns and
       user-defined columns.  This keeps things	consistent and predictable.

   TRANSFORMATIONS
       Transformations change how a value is rendered.	For example, they can
       take a number of	seconds	and display it in H:M:S	format.	 The following
       transformations are defined:

       commify
	   Adds	commas to large	numbers	every three decimal places.

       distill
	   Distills SQL	into verb-noun-noun format for quick comprehension.

       dulint_to_int
	   Accepts two unsigned	integers and converts them into	a single
	   longlong.  This is useful for certain operations with InnoDB, which
	   uses	two integers as	transaction identifiers, for example.

       fuzzy_time
	   Converts a number of	seconds	into a friendly, readable value	like
	   "1h35m".

       no_ctrl_char
	   Removes quoted control characters from the value.  This is affected
	   by the "charset" configuration variable.

	   This	transformation only operates within quoted strings, for
	   example, values to a	SET clause in an UPDATE	statement.  It will
	   not alter the UPDATE	statement, but will collapse the quoted	string
	   to [BINARY] or [TEXT], depending on the charset.

       percent
	   Converts a number to	a percentage by	multiplying it by two,
	   formatting it with "num_digits" digits after	the decimal point, and
	   optionally adding a percent sign (see "show_percent").

       secs_to_time
	   Formats a number of seconds as time in days+hours:minutes:seconds
	   format.

       set_precision
	   Formats numbers with	"num_digits" number of digits after the
	   decimal point.

       shorten
	   Formats a number as a unit of 1024 (k/M/G/T)	and with "num_digits"
	   number of digits after the decimal point.

   TABLE EDITOR
       The innotop table editor	lets you customize tables with keystrokes.
       You start the table editor with the '^' key.  If	there's	more than one
       table on	the screen, it will prompt you to choose one of	them.  Once
       you do, innotop will show you something like this:

	Editing	table definition for Buffer Pool.  Press ? for help, q to quit.

	name		   hdr		label		       src
	cxn		   CXN		Connection from	which  cxn
	buf_pool_size	   Size		Buffer pool size       IB_bp_buf_poo
	buf_free	   Free	Bufs	Buffers	free in	the b  IB_bp_buf_fre
	pages_total	   Pages	Pages total	       IB_bp_pages_t
	pages_modified	   Dirty Pages	Pages modified (dirty  IB_bp_pages_m
	buf_pool_hit_rate  Hit Rate	Buffer pool hit	rate   IB_bp_buf_poo
	total_mem_alloc	   Memory	Total memory allocate  IB_bp_total_m
	add_pool_alloc	   Add'l Pool	Additonal pool alloca  IB_bp_add_poo

       The first line shows which table	you're editing,	and reminds you	again
       to press	'?' for	a list of key mappings.	 The rest is a tabular
       representation of the table's columns, because that's likely what
       you're trying to	edit.  However,	you can	edit more than just the
       table's columns;	this screen can	start the filter editor, color rule
       editor, and more.

       Each row	in the display shows a single column in	the table you're
       editing,	along with a couple of its properties such as its header and
       source expression (see "EXPRESSIONS").

       The key mappings	are Vim-style, as in many other	places.	 Pressing 'j'
       and 'k' moves the highlight up or down.	You can	then (d)elete or
       (e)dit the highlighted column.  You can also (a)dd a column to the
       table.  This actually just activates one	of the columns already defined
       for the table; it prompts you to	choose from among the columns
       available but not currently displayed.  Finally,	you can	re-order the
       columns with the	'+' and	'-' keys.

       You can do more than just edit the columns with the table editor, you
       can also	edit other properties, such as the table's sort	expression and
       group-by	expression.  Press '?' to see the full list, of	course.

       If you want to really customize and create your own column, as opposed
       to just activating a built-in one that's	not currently displayed, press
       the (n)ew key, and innotop will prompt you for the information it
       needs:

       o   The column name: this needs to be a word without any	funny
	   characters, e.g. just letters, numbers and underscores.

       o   The column header: this is the label	that appears at	the top	of the
	   column, in the table	header.	 This can have spaces and funny
	   characters, but be careful not to make it too wide and waste	space
	   on-screen.

       o   The column's	data source: this is an	expression that	determines
	   what	data from the source (see "TABLES") innotop will put into the
	   column.  This can just be the name of an item in the	source,	or it
	   can be a more complex expression, as	described in "EXPRESSIONS".

       Once you've entered the required	data, your table has a new column.
       There is	no difference between this column and the built-in ones; it
       can have	all the	same properties	and behaviors.	innotop	will write the
       column's	definition to the configuration	file, so it will persist
       across sessions.

       Here's an example: suppose you want to track how	many times your	slaves
       have retried transactions.  According to	the MySQL manual, the
       Slave_retried_transactions status variable gives	you that data: "The
       total number of times since startup that	the replication	slave SQL
       thread has retried transactions.	This variable was added	in version
       5.0.4."	This is	appropriate to add to the "slave_sql_status" table.

       To add the column, switch to the	replication-monitoring mode with the
       'M' key,	and press the '^' key to start the table editor.  When
       prompted, choose	slave_sql_status as the	table, then press 'n' to
       create the column.  Type	'retries' as the column	name, 'Retries'	as the
       column header, and 'Slave_retried_transactions' as the source.  Now the
       column is created, and you see the table	editor screen again.  Press
       'q' to exit the table editor, and you'll	see your column	at the end of
       the table.

VARIABLE SETS
       Variable	sets are used in "S: Variables & Status" mode to define	more
       easily what variables you want to monitor.  Behind the scenes they are
       compiled	to a list of expressions, and then into	a column list so they
       can be treated just like	columns	in any other table, in terms of	data
       extraction and transformations.	However, you're	protected from the
       tedious details by a syntax that	ought to feel very natural to you: a
       SQL SELECT list.

       The data	source for variable sets, and indeed the entire	S mode,	is the
       combination of SHOW STATUS, SHOW	VARIABLES, and SHOW INNODB STATUS.
       Imagine that you	had a huge table with one column per variable returned
       from those statements.  That's the data source for variable sets.  You
       can now query this data source just like	you'd expect.  For example:

	Questions, Uptime, Questions/Uptime as QPS

       Behind the scenes innotop will split that variable set into three
       expressions, compile them and turn them into a table definition,	then
       extract as usual.  This becomes a "variable set," or a "list of
       variables you want to monitor."

       innotop lets you	name and save your variable sets, and writes them to
       the configuration file.	You can	choose which variable set you want to
       see with	the 'c'	key, or	activate the next and previous sets with the
       '>' and '<' keys.  There	are many built-in variable sets	as well, which
       should give you a good start for	creating your own.  Press 'e' to edit
       the current variable set, or just to see	how it's defined.  To create a
       new one,	just press 'c' and type	its name.

       You may want to use some	of the functions listed	in "TRANSFORMATIONS"
       to help format the results.  In particular, "set_precision" is often
       useful to limit the number of digits you	see.  Extending	the above
       example,	here's how:

	Questions, Uptime, set_precision(Questions/Uptime) as QPS

       Actually, this still needs a little more	work.  If your "interval" is
       less than one second, you might be dividing by zero because Uptime is
       incremental in this mode	by default.  Instead, use Uptime_hires:

	Questions, Uptime, set_precision(Questions/Uptime_hires) as QPS

       This example is simple, but it shows how	easy it	is to choose which
       variables you want to monitor.

PLUGINS
       innotop has a simple but	powerful plugin	mechanism by which you can
       extend or modify	its existing functionality, and	add new	functionality.
       innotop's plugin	functionality is event-based: plugins register
       themselves to be	called when events happen.  They then have a chance to
       influence the event.

       An innotop plugin is a Perl module (.pm)	file placed in innotop's
       "plugin_dir" directory.	On UNIX	systems, you can place a symbolic link
       to the module instead of	putting	the actual file	there.	innotop
       automatically discovers files named "*.pm".  If there is	a
       corresponding entry in the "plugins" configuration file section,
       innotop loads and activates the plugin.

       The module must conform to innotop's plugin interface.  Additionally,
       the source code of the module must be written in	such a way that
       innotop can inspect the file and	determine the package name and
       description.

   Package Source Convention
       innotop inspects	the plugin module's source to determine	the Perl
       package name.  It looks for a line of the form "package Foo;" and if
       found, considers	the plugin's package name to be	Foo.  Of course	the
       package name can	be a valid Perl	package	name such as Foo::Bar, with
       double colons (::) and so on.

       It also looks for a description in the source code, to make the plugin
       editor more human-friendly.  The	description is a comment line of the
       form "# description: Foo", where	"Foo" is the text innotop will
       consider	to be the plugin's description.

   Plugin Interface
       The innotop plugin interface is quite simple: innotop expects the
       plugin to be an object-oriented module it can call certain methods on.
       The methods are

       new(%variables)
	   This	is the plugin's	constructor.  It is passed a hash of innotop's
	   variables, which it can manipulate (see "Plugin Variables").	 It
	   must	return a reference to the newly	created	plugin object.

	   At construction time, innotop has only loaded the general
	   configuration and created the default built-in variables with their
	   default contents (which is quite a lot).  Therefore,	the state of
	   the program is exactly as in	the innotop source code, plus the
	   configuration variables from	the "general" section in the config
	   file.

	   If your plugin manipulates the variables, it	is changing global
	   data, which is shared by innotop and	all plugins.  Plugins are
	   loaded in the order they're listed in the config file.  Your	plugin
	   may load before or after another plugin, so there is	a potential
	   for conflict	or interaction between plugins if they modify data
	   other plugins use or	modify.

       register_for_events()
	   This	method must return a list of events in which the plugin	is
	   interested, if any.	See "Plugin Events" for	the defined events.
	   If the plugin returns an event that's not defined, the event	is
	   ignored.

       event handlers
	   The plugin must implement a method named the	same as	each event for
	   which it has	registered.  In	other words, if	the plugin returns
	   qw(foo bar) from register_for_events(), it must have	foo() and
	   bar() methods.  These methods are callbacks for the events.	See
	   "Plugin Events" for more details about each event.

   Plugin Variables
       The plugin's constructor	is passed a hash of innotop's variables, which
       it can manipulate.  It is probably a good idea if the plugin object
       saves a copy of it for later use.  The variables	are defined in the
       innotop variable	%pluggable_vars, and are as follows:

       action_for
	   A hashref of	key mappings.  These are innotop's global hot-keys.

       agg_funcs
	   A hashref of	functions that can be used for grouping.  See
	   "GROUPING".

       config
	   The global configuration hash.

       connections
	   A hashref of	connection specifications.  These are just
	   specifications of how to connect to a server.

       dbhs
	   A hashref of	innotop's database connections.	 These are actual DBI
	   connection objects.

       filters
	   A hashref of	filters	applied	to table rows.	See "FILTERS" for
	   more.

       modes
	   A hashref of	modes.	See "MODES" for	more.

       server_groups
	   A hashref of	server groups.	See "SERVER GROUPS".

       tbl_meta
	   A hashref of	innotop's table	meta-data, with	one entry per table
	   (see	"TABLES" for more information).

       trans_funcs
	   A hashref of	transformation functions.  See "TRANSFORMATIONS".

       var_sets
	   A hashref of	variable sets.	See "VARIABLE SETS".

   Plugin Events
       Each event is defined somewhere in the innotop source code.  When
       innotop runs that code, it executes the callback	function for each
       plugin that expressed its interest in the event.	 innotop passes	some
       data for	each event.  The events	are defined in the %event_listener_for
       variable, and are as follows:

       extract_values($set, $cur, $pre,	$tbl)
	   This	event occurs inside the	function that extracts values from a
	   data	source.	 The arguments are the set of values, the current
	   values, the previous	values,	and the	table name.

       set_to_tbl
	   Events are defined at many places in	this subroutine, which is
	   responsible for turning an arrayref of hashrefs into	an arrayref of
	   lines that can be printed to	the screen.  The events	all pass the
	   same	data: an arrayref of rows and the name of the table being
	   created.  The events	are set_to_tbl_pre_filter,
	   set_to_tbl_pre_sort,set_to_tbl_pre_group, set_to_tbl_pre_colorize,
	   set_to_tbl_pre_transform, set_to_tbl_pre_pivot,
	   set_to_tbl_pre_create, set_to_tbl_post_create.

       draw_screen($lines)
	   This	event occurs inside the	subroutine that	prints the lines to
	   the screen.	$lines is an arrayref of strings.

   Simple Plugin Example
       The easiest way to explain the plugin functionality is probably with a
       simple example.	The following module adds a column to the beginning of
       every table and sets its	value to 1.  (If you copy and paste this
       example code, be	sure to	remove the first space from each line; lines
       such as '# description' must not	start with whitespace).

	use strict;
	use warnings FATAL => 'all';

	package	Innotop::Plugin::Example;
	# description: Adds an 'example' column	to every table

	sub new	{
	   my (	$class,	%vars )	= @_;
	   # Store reference to	innotop's variables in $self
	   my $self = bless { %vars }, $class;

	   # Design the	example	column
	   my $col = {
	      hdr   => 'Example',
	      just  => '',
	      dec   => 0,
	      num   => 1,
	      label => 'Example',
	      src   => 'example', # Get	data from this column in the data source
	      tbl   => '',
	      trans => [],
	   };

	   # Add the column to every table.
	   my $tbl_meta	= $vars{tbl_meta};
	   foreach my $tbl ( values %$tbl_meta ) {
	      #	Add the	column to the list of defined columns
	      $tbl->{cols}->{example} =	$col;
	      #	Add the	column to the list of visible columns
	      unshift @{$tbl->{visible}}, 'example';
	   }

	   # Be	sure to	return a reference to the object.
	   return $self;
	}

	# I'd like to be called	when a data set	is being rendered into a table,	please.
	sub register_for_events	{
	   my (	$self )	= @_;
	   return qw(set_to_tbl_pre_filter);
	}

	# This method will be called when the event fires.
	sub set_to_tbl_pre_filter {
	   my (	$self, $rows, $tbl ) = @_;
	   # Set the example column's data source to the value 1.
	   foreach my $row ( @$rows ) {
	      $row->{example} =	1;
	   }
	}

	1;

   Plugin Editor
       The plugin editor lets you view the plugins innotop discovered and
       activate	or deactivate them.  Start the editor by pressing $ to start
       the configuration editor	from any mode.	Press the 'p' key to start the
       plugin editor.  You'll see a list of plugins innotop discovered.	 You
       can use the 'j' and 'k' keys to move the	highlight to the desired one,
       then press the *	key to toggle it active	or inactive.  Exit the editor
       and restart innotop for the changes to take effect.

SQL STATEMENTS
       innotop uses a limited set of SQL statements to retrieve	data from
       MySQL for display.  The statements are customized depending on the
       server version against which they are executed; for example, on MySQL 5
       and newer, INNODB_STATUS	executes "SHOW ENGINE INNODB STATUS", while on
       earlier versions	it executes "SHOW INNODB STATUS".  The statements are
       as follows:

	Statement	    SQL	executed
	=================== ===============================
	INDEX_STATISTICS    SELECT * FROM INFORMATION_SCHEMA.INDEX_STATISTICS
	INNODB_STATUS	    SHOW [ENGINE] INNODB STATUS
	KILL_CONNECTION	    KILL
	KILL_QUERY	    KILL QUERY
	OPEN_TABLES	    SHOW OPEN TABLES
	PROCESSLIST	    SHOW FULL PROCESSLIST
	SHOW_MASTER_LOGS    SHOW MASTER	LOGS
	SHOW_MASTER_STATUS  SHOW MASTER	STATUS
	SHOW_SLAVE_STATUS   SHOW SLAVE STATUS
	SHOW_STATUS	    SHOW [GLOBAL] STATUS
	SHOW_VARIABLES	    SHOW [GLOBAL] VARIABLES
	TABLE_STATISTICS    SELECT * FROM INFORMATION_SCHEMA.TABLE_STATISTICS

DATA SOURCES
       Each time innotop extracts values to create a table (see	"EXPRESSIONS"
       and "TABLES"), it does so from a	particular data	source.	 Largely
       because of the complex data extracted from SHOW INNODB STATUS, this is
       slightly	messy.	SHOW INNODB STATUS contains a mixture of single	values
       and repeated values that	form nested data sets.

       Whenever	innotop	fetches	data from MySQL, it adds two extra bits	to
       each set: cxn and Uptime_hires.	cxn is the name	of the connection from
       which the data came.  Uptime_hires is a high-resolution version of the
       server's	Uptime status variable,	which is important if your "interval"
       setting is sub-second.

       Here are	the kinds of data sources from which data is extracted:

       STATUS_VARIABLES
	   This	is the broadest	category, into which the most kinds of data
	   fall.  It begins with the combination of SHOW STATUS	and SHOW
	   VARIABLES, but other	sources	may be included	as needed, for
	   example, SHOW MASTER	STATUS and SHOW	SLAVE STATUS, as well as many
	   of the non-repeated values from SHOW	INNODB STATUS.

       DEADLOCK_LOCKS
	   This	data is	extracted from the transaction list in the LATEST
	   DETECTED DEADLOCK section of	SHOW INNODB STATUS.  It	is nested two
	   levels deep:	transactions, then locks.

       DEADLOCK_TRANSACTIONS
	   This	data is	from the transaction list in the LATEST	DETECTED
	   DEADLOCK section of SHOW INNODB STATUS.  It is nested one level
	   deep.

       EXPLAIN
	   This	data is	from the result	set returned by	EXPLAIN.

       INNODB_BLOCKED_BLOCKER
	   This	data is	from the INFORMATION_SCHEMA tables related to InnoDB
	   locks and the processlist.

       INNODB_TRANSACTIONS
	   This	data is	from the TRANSACTIONS section of SHOW INNODB STATUS.

       IO_THREADS
	   This	data is	from the list of threads in the	the FILE I/O section
	   of SHOW INNODB STATUS.

       INNODB_LOCKS
	   This	data is	from the TRANSACTIONS section of SHOW INNODB STATUS
	   and is nested two levels deep.

       MASTER_SLAVE
	   This	data is	from the combination of	SHOW MASTER STATUS and SHOW
	   SLAVE STATUS.

       OPEN_TABLES
	   This	data is	from SHOW OPEN TABLES.

       PROCESSLIST
	   This	data is	from SHOW FULL PROCESSLIST.

       PROCESSLIST_STATS
	   This	data is	from SHOW FULL PROCESSLIST and computes	stats such as
	   the maximum time a user query has been running, and how many	user
	   queries are running.	A "user	query" excludes	replication threads.

       OS_WAIT_ARRAY
	   This	data is	from the SEMAPHORES section of SHOW INNODB STATUS and
	   is nested one level deep.  It comes from the	lines that look	like
	   this:

	    --Thread 1568861104	has waited at btr0cur.c	line 424 ....

MYSQL PRIVILEGES
       o   You must connect to MySQL as	a user who has the SUPER privilege for
	   many	of the functions.

       o   If you don't	have the SUPER privilege, you can still	run some
	   functions, but you won't necessarily	see all	the same data.

       o   You need the	PROCESS	privilege to see the list of currently running
	   queries in Q	mode.

       o   You need special privileges to start	and stop slave servers.

       o   You need appropriate	privileges to create and drop the deadlock
	   tables if needed (see "SERVER CONNECTIONS").

SYSTEM REQUIREMENTS
       You need	Perl to	run innotop, of	course.	 You also need a few Perl
       modules:	DBI, DBD::mysql,  Term::ReadKey, and Time::HiRes.  These
       should be included with most Perl distributions,	but in case they are
       not, I recommend	using versions distributed with	your operating system
       or Perl distribution, not from CPAN.  Term::ReadKey in particular has
       been known to cause problems if installed from CPAN.

       If you have Term::ANSIColor, innotop will use it	to format headers more
       readably	and compactly.	(Under Microsoft Windows, you also need
       Win32::Console::ANSI for	terminal formatting codes to be	honored).  If
       you install Term::ReadLine, preferably Term::ReadLine::Gnu, you'll get
       nice auto-completion support.

       I run innotop on	Gentoo GNU/Linux, Debian and Ubuntu, and I've had
       feedback	from people successfully running it on Red Hat,	CentOS,
       Solaris,	and Mac	OSX.  I	don't see any reason why it won't work on
       other UNIX-ish operating	systems, but I don't know for sure.  It	also
       runs on Windows under ActivePerl	without	problem.

       innotop has been	used on	MySQL versions 3.23.58,	4.0.27,	4.1.0, 4.1.22,
       5.0.26, 5.1.15, and 5.2.3.  If it doesn't run correctly for you,	that
       is a bug	that should be reported.

FILES
       $HOMEDIR/.innotop and/or	/usr/local/etc are used	to store configuration
       information.  Files include the configuration file innotop.conf,	the
       core_dump file which contains verbose error messages if "debug" is
       enabled,	and the	plugins/ subdirectory.

GLOSSARY OF TERMS
       tick
	   A tick is a refresh event, when innotop re-fetches data from
	   connections and displays it.

ACKNOWLEDGEMENTS
       The following people and	organizations are acknowledged for various
       reasons.	 Hopefully no one has been forgotten.

       Aaron Racine, Allen K. Smith, Aurimas Mikalauskas, Bartosz Fenski,
       Brian Miezejewski, Christian Hammers, Cyril Scetbon, Dane Miller, David
       Multer, Dr. Frank Ullrich, Giuseppe Maxia, Google.com Site Reliability
       Engineers, Google Code, Jan Pieter Kunst, Jari Aalto, Jay Pipes,	Jeremy
       Zawodny,	Johan Idren, Kristian Kohntopp,	Lenz Grimmer, Maciej
       Dobrzanski, Michiel Betel, MySQL	AB, Paul McCullagh, Sebastien
       Estienne, Sourceforge.net, Steven Kreuzer, The Gentoo MySQL Team,
       Trevor Price, Yaar Schnitman, and probably more people that have	not
       been included.

       (If your	name has been misspelled, it's probably	out of fear of putting
       international characters	into this documentation; earlier versions of
       Perl might not be able to compile it then).

COPYRIGHT, LICENSE AND WARRANTY
       This program is copyright (c) 2006 Baron	Schwartz.  Feedback and
       improvements are	welcome.

       THIS PROGRAM IS PROVIDED	"AS IS"	AND WITHOUT ANY	EXPRESS	OR IMPLIED
       WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF
       MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE.

       This program is free software; you can redistribute it and/or modify it
       under the terms of the GNU General Public License as published by the
       Free Software Foundation, version 2; OR the Perl	Artistic License.  On
       UNIX and	similar	systems, you can issue `man perlgpl' or	`man
       perlartistic' to	read these licenses.

       You should have received	a copy of the GNU General Public License along
       with this program; if not, write	to the Free Software Foundation, Inc.,
       59 Temple Place,	Suite 330, Boston, MA  02111-1307  USA.

       Execute innotop and press '!' to	see this information at	any time.

AUTHOR
       Originally written by Baron Schwartz; currently maintained by Aaron
       Racine.

BUGS
       You can report bugs, ask	for improvements, and get other	help and
       support at <https://github.com/innotop/innotop>.	 There are mailing
       lists, a	source code browser, a bug tracker, etc.  Please use these
       instead of contacting the maintainer or author directly,	as it makes
       our job easier and benefits others if the discussions are permanent and
       public.	Of course, if you need to contact us in	private, please	do.

perl v5.32.0			  2020-08-29			    INNOTOP(1)

NAME | SYNOPSIS | DESCRIPTION | QUICK-START | OPTIONS | HOTKEYS | MODES | INNOTOP STATUS | SERVER ADMINISTRATION | SERVER CONNECTIONS | SERVER GROUPS | SWITCHING BETWEEN CONNECTIONS | ERROR HANDLING | NON-INTERACTIVE OPERATION | CONFIGURING | CONFIGURATION FILE | CUSTOMIZING | VARIABLE SETS | PLUGINS | SQL STATEMENTS | DATA SOURCES | MYSQL PRIVILEGES | SYSTEM REQUIREMENTS | FILES | GLOSSARY OF TERMS | ACKNOWLEDGEMENTS | COPYRIGHT, LICENSE AND WARRANTY | AUTHOR | BUGS

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=innotop&sektion=1&manpath=FreeBSD+12.2-RELEASE+and+Ports>

home | help