Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
iorate(1)		    General Commands Manual		     iorate(1)

       iorate -	tests I/O subsystems under various loads

       iorate [	-a ] [ -l ] [ -n ] [ -s	] [ -u ] [ -v ]	[ -d <debug_level> ] [
       -c skew_shift ] [ -r target_rate	] [ -f device_file ] [ -p pattern_file
       ] [ -t test_file	] [ -i iops_file ] [ -o	output_base ]

       The  iorate  utility  conducts a	series of I/O tests.  Each test	is de-
       fined in	test_file using	a collection of	patterns.  Each	pattern	is de-
       fined in	pattern_file.  The tests are run simultaneously	against	all of
       the devices in device_file.  A useful user guide	can be downloaded from
       the web page at:

       Each  pattern  defines  a  specific I/O pattern that will be run.  Each
       pattern can be either read or write, either random or sequential.   For
       sequential  patterns,  a	 maximum number	of contiguous sequential tests
       can be specified, after which the next random sequence will start at  a
       new, random location.  All I/O for a pattern is of a fixed size.

       Each  test  uses	one or more patterns, specified	as a percentage	of the
       I/Os to be performed.  The tests	specify	 how  long  they  should  run,
       where  they  start  in  the  file, and the size of the device that they
       should use for testing.	The can	also specify how much of the test  pe-
       riod  should be ignored for performance characteristic reasons to allow
       time for	the test to achieve a 'steady state,' which can	be useful when
       testing disk arrays with	large amounts of cache.

       Iorate  generates  a log	file of	the test being run.  The log file con-
       tains a copy of the configuration files being used  (device_file,  pat-
       tern_file  and  test_file).  These will not have	any comments, and will
       be printed directly from	the information	that will be used to drive the
       actual  tests.	These  copies,	if  split  out into the	separate input
       files, will reproduce the same test.  They are included in the  log  to
       show  exactly  what was run, and	to allow that test to be reproduced at
       any later time.	The log	also records the starting and ending times  of
       each  test, by device.  Any errors encountered are also included	in the

       Note that if an iops_file was used, or if the target  rate  was	scaled
       using  a	 target_rate,  then  the  iops	target	for  each  test	in the
       test_file section of the	log will have a	value reflecting these	target
       rate  adjustments.   Spliting these sections out	to run a new test will
       run the exact same test again, at the adjusted target rate, without the
       need  for  target  iops	or rate	adjustments.  If the adjustments (iops
       file and/or target rate percentage) are applied again, the tests	may be

       In  addition  to	 the log file, iorate generates	a performance file for
       the test	group.	The performance	file contains detailed	statistics  on
       each  test  run	on  each device.  Each line in the file	represents the
       data for	a single test on a single device.  The first line of the  file
       is  a  header  that describes the contents of each test line.  The file
       uses tab	separated values, that	are  easy  to  read  with  scripts  or
       spreadsheet  programs  (though  the  alignments are a challenge if read
       with a standard editor or simply	printed).  The fields are:

	  test_number -- Number	of the test being run
	  test_name -- Name of the test	being run
	  device_num --	Number of the device for this line
	  device_name -- Name of the device for	this line
	  total_sec -- Total seconds the test was run
	  measured_sec -- Time that results are	measured over
	  reads	-- Number of reads from	this device
	  KB_read -- KB	of data	reads from the device
	  read_resp -- Avg. response time for all device reads
	  writes -- Number of writes to	this device
	  KB_written --	KB of data written to the device
	  write_resp --	Avg. response time for all device writes
	  total_I/Os --	Number of I/Os to this device
	  total_KB -- KB of data transfered to the device
	  total_resp --	Avg. response time for all device I/Os
	  I/Os_per_sec -- Avg. number of device	I/Os per second
	  KB_per_sec --	Avg. KB	of data	transferred per	second
	  dev_copy -- Which copy of the	device this is for (for	 devices  with

       Each  line is a set of performance numbers for a	single device.	To get
       the total for a given test, the numbers for each	of the	devices	 needs
       to be summarized	for each test.	The gen_sums AWK program provided will
       generate	such test summaries, which are often more useful  for  compar-

       If  the	pair  of output	files (performance and log) are	available from
       any test, the numbers can be examined, and the test can be reproduced.

       The log file is usually named iorate.log, and the performance  data  is
       in  iorate.perf.	  If  the  -o  option  is  used, log file will be out-
       put_base.log and	the performance	data  will  be	output_base.perf.   If
       large  numbers of test passes will be run, it is	suggested that sub-di-
       rectories be built for each group of test.  The	output_base  parameter
       can include sub-directory names as well,	but those sub-directories must
       exist before iorate begins (iorate will not create sub-directories).

       The -l limits option causes iorate to display file limits and exit  (no
       testing).   This	 is  used  in the makefile to verify support for large
       file sizes.

       The -n no testing option	turns off test execution.   The	 program  will
       still  do  all  setup checks, including a seek to test the size of each
       file.  The program will still fork off a	child process to do the	 test-
       ing.  However, the child	will exit quickly.

       The  -s	silent	option	stops  all but the error and warning output to
       standard	out.  The other	information normally reported,	such  as  test
       configurations  and  progress,  is  still  recorded in the log file for
       later reference.

       The -u use direct I/O option sets the O_DIRECT  (or  appropriate)  flag
       when  opening the data files for	testing. This should cause file	system
       buffers to be bypassed and provide a more accurate measure of the  true
       array performance.

       The  -a	allow causes the checks	on read	sizes to be lenient.  This op-
       tion is helpful on some platforms where the reads from  files  on  file
       systems may not return as many bytes as the read	I/O is requesting.

       The  -v verbose option turns on verbose mode.  Since this will generate
       log information for every I/O of	every device, it  is  not  recommended
       for  use	 in a lengthy test.  It	will have a dramatic effect on the I/O
       rates of	many tests, and	should not be turned on	while real  test  num-
       bers are	to be generated.  Note that multiple processes (one per	active
       test process) will be writing to	the same log file, so the output lines
       may  get	 a  bit	 messy with more than one active device.  The log file
       will be very busy, and may grow to be quite large.

       The -d <debug_level> debug option tells iorate to turn on a  number  of
       debugging  outputs.   This will provide a large number of messages that
       may not be understandable to someone not	 reading  the  program	source
       code.  The  optional debug_level	allows the user	to shift from some in-
       formation (10) to  an  excrutiatingly  detailed	level  of  information

       The  -c	skew_shift option tells	iorate to alter	the sub-LUN skew data.
       This shifts skew_shift percent of the data between the  target  levels,
       with  hot  regions  becoming cold, cold regions becoming	warm, and warm
       regions ecoming hot. This is limited to be 0 and	99. Note that the  99%
       shift  hsd 100% of the data on a	new performance	level, and thus	is not
       close to	the 0% level.

       The -r target_rate option tells iorate to run at	a reduced target rate.
       This will cause each test to run	at this	percentage of the maximum iops
       given for that test (this has no	effect if an iops has not been	speci-
       fied  for that test).  The target (base)	iops rate for each test	may be
       specified in the	test_file, or in the iops_file.	  Valid	 target	 rates
       are  from  0  (no  target  rate used - leave iops values	alone) to 100%
       (full test).

       Iorate requires that all	three configuration files be properly  format-
       ted.   Any  errors in the files will be reported	as the program starts,
       and may cause testing to	be aborted.  In	general, anything following  a
       hash  mark  (#) is considered a comment to the end of the physical line
       (marked by a carriage return).  Each logical line  describes  a	single
       item  (device,  pattern,	 test, or iops rating) and ends	in a semicolon
       (;).  Logical lines (single items) can span many	input lines.

       In general, each	item will have an item type identifier	(device,  pat-
       tern,  test,  or	target)	and then information about that	item.  The de-
       tails of	the various options for	each item type are noted in  the  file
       descriptions below.

       When  providing sizes, a	number can be followed by a sizing factor (KB,
       MB, GB, TB).  Each step will increase the size by a multiplier of  1024
       (not 1000).  When providing time	values,	a number must be followed by a
       time type (sec, min, hour) which	will  scale  the  number  to  seconds.
       When  providing	numbers, general math functions	also work, so 4	* 4 GB
       is the same as 16 GB.

       Iorate will assume a device_file	name of	devices.ior if none is	speci-
       fied.  Each device entry	describes a single device, such	as:

	  Device 1 = "/mnt/datafile.tst"  capacity 3GB;

       Any valid file name can be given, whether a raw device file or the name
       of a file system	file that exists and has the stated size.  Device num-
       bers  must  be  unique,	but are	also optional (if not provided,	device
       numbers will be assigned	in order automatically - a record of which  is
       provided	in the log file).  The capacity	is the usable space in the de-
       vice for	testing	- the actual size can be much larger.

       There is	a minumum offset of 8 KB, which	is also	the default,  to  make
       sure that any device header such	as a label is not overwritten.	Alter-
       nate values can be set with offset _size_.  The capacity	of the	device
       is  the	usable capacity	starting at the	offset point.  I/O can be tar-
       getted at the last 2 GB of a 36 GB device using offset 34 GB capacity 2

       The  minumum  block  size, which	is also	the default, is	512 B for most
       operating systems (1024 B on HP-UX).  Alternate values can be set  with
       block size _size_.  This	block size is the minimum I/O size for the de-
       vice, and all I/O will be aligned to this size, as well as being	a mul-
       tiple  of  it.  The patterns in use from	any test must be a multiple of
       this size.

       Rather than placing the same file information in	the device file	repet-
       itively,	 a  single definition can have multiple	tasks set to access it
       using count _count_.  Each  process  will  open	the  file  separately,
       choosing	 a  random  start  position and	performing seeks and I/O sepa-
       rately from the others.

       The file	can be flagged as being	read-only by  specifiying  read	 only.
       This  will cause	iorate to open the file	for reads but not writes - and
       any tests attempting to do writes will cause iorate  to	fail.	Unless
       the  file is set	to read	only, iorate will attempt to open the file for
       reads and writes	- even if none of the active  patterns	in  the	 tests
       will do writes.

       There  is  also	an  option to lock the file by specifiying lock.  This
       will use	the fcntl() system call	to lock	the area of the	device	speci-
       fied  by	 the  offset and capacity.  Note that these are	exclusive (for
       writes),	non-blocking locks - if	they cannot be	obtained,  the	entire
       test  will  be aborted.	Note that multiple read	locks on the same area
       of a file are permitted,	so setting the devices to read only will allow
       them  to	 be shared even	if locking is requested.  If the device	is not
       set to read only, then iorate will attempt to aquire a write lock  (ex-
       clusive)	on the defined region of the file - even if none of the	active
       patterns	in the tests will do writes.

       Iorate will assume a pattern_file name of patterns.ior if none is spec-
       ified.  Each pattern entry describes a single pattern, such as:

	  Pattern 1 = "2K Seq Read"  io	size 2KB  sequential  read;

       The  pattern  number  must be given, since that is how the patterns are
       chosen from the tests.  They need not be	sequential.

       The io size _size_ parameter specifies the size of all I/Os  that  will
       be driven from this pattern.  To	mix I/O	sizes, tests can be configured
       that use	multiple patterns.

       The area	to run the pattern against can be specified with  from	_size_
       or  from	_count_% to set	the starting location, and size	_size_ or size
       _count_%	to set the size	to test	against	(starting at  the  from	 loca-
       tion).	Any  percentage	 sizes	are scaled from	the active size	of the
       test using this pattern.

       By default, the pattern will do all sequential  read.   Setting	random
       will  change  the pattern to random address selections.	For a mix, use
       max sequential _count_ to limit the number of sequential	I/Os that will
       be  done	 in  a row before a new, random	location is chosen.  Note that
       setting _count_ to 10 will allways to 10	sequential I/Os	before select-
       ing  a new location.  To	mix the	I/O types, select write	for 100% write
       activities, or read _count_% to have the	pattern	do a mix of reads  and

       To  more	closely	model a	real application, I/Os to the same address can
       be repeated.  Setting reuse _count_% will cause that percentage of I/Os
       to  be done to the same address as another recent I/O.  Setting history
       _count_ will determine how long of a history of I/Os is kept to reuse.

       Further modeling	is  available  by  creating  'locality	of  reference'
       zones.	By  setting zone _size_	as a fixed size, or zone _count_% as a
       percentage of the pattern test area, iorate will	create	count  _count_
       localities  to  do  I/Os	against.  To model 'moving hot spots,' specify
       limit _count_ to	determine how many I/Os	will be	done to	a given	local-
       ity  before a new one is	chosen.	 Defining any of zone, count, or limit
       will cause default values to be set for the others if none  are	speci-

       Iorate  will assume a test_file name of tests.ior if none is specified.
       WARNING:	the test-fx.ior	tests are  destructive	to  all	 files/devices
       used for	testing. Files/devices that have data contained	in them	should
       be included in the device file only if extreme care has been  taken  to
       ensure  that  all test patterns are read-only.  Because of this danger,
       the default test	file uses read-only patterns.	Even  so,  great  care
       should be taken whenever	any entry in the devices file has data that is
       valued.	Failure	to do so has resulted not only in lost	data,  but  in
       one  case  caused such damage to	the root file system (random writes of
       garbage data) that the server had to be reinstalled from	 tape.	 There
       is  no  attempt made to check for other users of	any device listed, the
       tests are simply	run.

       Each test entry describes a single test,	such as:

	  Test 1 = "Mixed 2K Seq"  for 120 sec ignore 20 sec 50%  pat  1,  50%
	  pat 2;

       Test  numbers  must  be unique, but are also optional (if not provided,
       device numbers will be assigned in order	automatically -	 a  record  of
       which  is  provided  in the log file).  The duration of the test	is set
       with for	_time_.	 To make sure that the performance during any start-up
       period  is  not	measured, but rather the steady-state numbers, setting
       ignore _time_ causes the	performance statistics to be  reset  that  far
       into  the  test time.  In the event that	one test may leave work	in for
       the test	storage, such as writes	sitting	in the cache of	a disk	array,
       pause _time_ will introduce that	length of delay	before the test	is ac-

       The area	to run the test	against	can be specified with from  _size_  or
       from  _count_%  to  set	the starting location, and size	_size_ or size
       _count_%	to set the size	to test	against	(starting at  the  from	 loca-
       tion).	Sizing that are	based on percentages are based on the capacity
       of each device being tested - so	if there are  devices  with  different
       capacities, the tests will resize themselves for	each one.

       To  limit  the  level  of I/O generated by this test, set _count_ iops.
       This sets an I/Os per second target for the entire  test	 -  and	 since
       there  are  separate processes run for each active device defined, they
       will each target	their share independently.  This means that  if	 there
       are 10 active devices, and 100 iops is specified, then the program will
       attempt to drive	10 iops	to each	device.	 In the	event that one	device
       cannot  do  its	share, the other devices will not be tasked will addi-
       tional work, as the test	process	working	on each	devie are not aware of
       each other.

       Finally,	a list of patterns to run is included.	Each pattern is	speci-
       fied as _count_%	pat _pattern_, where each percentage is	a whole	number
       (decimals  ignored),  and  the  total  percentages of all patterns must
       equal exactly 100%.

       Iorate assumes that no iops_file	is available.  Unless an iops_file  is
       given on	the command line or through the	environment variable, the only
       iops limits will	be set in the test file	itself.	 If there is  no  iops
       limit  specified	for a test, then I/O will be generated as fast as pos-
       sible on	every device.  When a file is given, each  entry  sets	a  new
       iops limit for a	given test, such as:

	  Target test 5	at 2000	iops;

       This is most useful for times when a given set of tests is to be	run on
       differing configurations, and it	is not known ahead of  time  what  the
       upper  limit  of	each test will be (maximum test	rate).	From there, it
       may be interesting to run the same tests	at some	percentage of the max-
       imum  rate.   If	a full speed pass of a test has	already	been run, then
       the gen_iops AWK	program	provided will generate such an iops_file  that
       matches	the  performance  seen	it that	run.  Together with a new tar-
       get_rate, a pattern can be tested for response times  at	 various  per-
       centages	of the maximum attained	throughput for each test.

       The  command  line  file	 names can be provided for in the environment,

	      The name of the device_file to use.

	      The name of the pattern_file to use.

	      The name of the test_file	to use.

	      The name of the iops_file	to use.

	      The base name of the output files.

       Written by Vince	Westin (, with a lot  of  assist-
       ance  from  the	EMC  Engineering  Team.	 Copyright by EMC Corporation,
       1997-2011.  All rights reserved.	 The latest version is available  from
       the  web	 page  at	Use  of	 the latest version is
       strongly	encouraged.  Check the README and README.versions  files  from
       the  latest  version  for  new features and bugs	from earlier revisions
       that have been fixed.

       This code is the	property of EMC	Corporation.  However, it may be used,
       reproduced,  and	passed on to others as long as the contents, including
       all copyright notices, remain intact.  Modifications  to,  or  modified
       versions	 of  these  files, may also be distributed, provided that they
       are clearly labeled as having been modified from	the original.  In  the
       event that modified files are created, the original files are to	be in-
       cluded with every distribution of those modified	files.	 Inclusion  of
       this  code  into	 a commercial product by any company other than	EMC is
       prohibited without prior	written	consent.

       Having said the legal stuff, this code is designed to provide  a	 good,
       generic	tool  for testing I/O subsystems under various kinds of	loads.
       If you have suggestions for improvements	in this	tool, please send them
       along to	the above address.

       read(2),	write(2), seek(2), open(2), close(2), fcntl(2)

				 October 2011			     iorate(1)


Want to link to this manual page? Use this URL:

home | help