Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
Statistics::Basic(3)  User Contributed Perl Documentation Statistics::Basic(3)

       Statistics::Basic - A collection	of very	basic statistics modules

	   use Statistics::Basic qw(:all);

       These actually return objects, not numbers.  The	objects	will
       interpolate as nicely formated numbers (using Number::Format).  Or the
       actual number will be returned when the object is used as a number.

	   my $median =	median(	1,2,3 );
	   my $mean   =	mean(  [1,2,3]); # array refs are ok too

	   my $variance	= variance( 1,2,3 );
	   my $stddev	= stddev(   1,2,3 );

       Although	passing	unblessed numbers and array refs to these functions
       works, it's sometimes better to pass vector objects so the objects can
       reuse calculated	values.

	   my $v1	= $mean->query_vector;
	   my $variance	= variance( $v1	);
	   my $stddev	= stddev(   $v1	);

       Here, the mean used by the variance and the variance used by the
       standard	deviation will not need	to be recalculated.  Now consider
       these two calculations.

	   my $covariance  = covariance(  [1 ..	3], [1 .. 3] );
	   my $correlation = correlation( [1 ..	3], [1 .. 3] );

       The covariance above would need to be recalculated by the correlation
       when these functions are	called this way.  But, if we instead built
       vectors first, that wouldn't happen:

	   # $v1 is defined above
	   my $v2  = vector(1,2,3);
	   my $cov = covariance(  $v1, $v2 );
	   my $cor = correlation( $v1, $v2 );

       Now $cor	can reuse the variance calculated in $cov.

       All of the functions above return objects that interpolate or evaluate
       as a single string or as	a number.  Statistics::Basic::LeastSquareFit
       and Statistics::Basic::Mode are different:

	   my $unimodal	  = mode(1,2,3,3);
	   my $multimodal = mode(1,2,3);

	   print "The modes are: $unimodal and $multimodal.\n";
	   print "The first is multimodal... " if $unimodal->is_multimodal;
	   print "The second is	multimodal.\n" if $multimodal->is_multimodal;

       In the first case, $unimodal will interpolate as	a string and function
       correctly as a number.  However,	in the second case, trying to use
       $multimodal as a	number will "croak" an error --	it still interpolates
       fine though.

	   my $lsf = leastsquarefit($v1, $v2);

       This $lsf will interpolate fine,	showing	"LSF( alpha: $alpha, beta:
       $beta )", but it	will "croak" if	you try	to use the object as a number.

	   my $v3	      =	$multimodal->query;
	   my ($alpha, $beta) =	$lsf->query;
	   my $average	      =	$mean->query;

       All of the objects allow	you to explicitly query, if you're not in the
       mood to use overload.

	   my @answers = (

       The following shortcut functions	can be used in place of	calling	the
       module's	"new()"	method directly.

       They all	take either array refs or lists	as arguments, with the
       exception of the	shortcuts that need two	vectors	to process (e.g.

	   Returns a Statistics::Basic::Vector object.	Arguments to
	   "vector()" can be any of: an	array ref, a list of numbers, or a
	   blessed vector object.  If passed a blessed vector object, vector
	   will	just return the	vector passed in.

       mean() average()	avg()
	   Returns a Statistics::Basic::Mean object.  You can choose to	call
	   "mean()" as "average()" or "avg()".	Arguments can be any of: an
	   array ref, a	list of	numbers, or a blessed vector object.

	   Returns a Statistics::Basic::Median object.	Arguments can be any
	   of: an array	ref, a list of numbers,	or a blessed vector object.

	   Returns a Statistics::Basic::Mode object.  Arguments	can be any of:
	   an array ref, a list	of numbers, or a blessed vector	object.

       variance() var()
	   Returns a Statistics::Basic::Variance object.  You can choose to
	   call	"variance()" as	"var()".  Arguments can	be any of: an array
	   ref,	a list of numbers, or a	blessed	vector object.	If you will
	   also	be calculating the mean	of the same list of numbers it's
	   recommended to do this:

	       my $vec	= vector(1,2,3);
	       my $mean	= mean($vec);
	       my $var	= variance($vec);

	   This	would also work:

	       my $mean	= mean(1,2,3);
	       my $var	= variance($mean->query_vector);

	   This	will calculate the same	mean twice:

	       my $mean	= mean(1,2,3);
	       my $var	= variance(1,2,3);

	   If you really only need the variance, ignore	the above and this is

	       my $variance = variance(1,2,3,4,5);

	   Returns a Statistics::Basic::StdDev object.	Arguments can be any
	   of: an array	ref, a list of numbers,	or a blessed vector object.
	   Pass	a vector object	to "stddev()" to avoid recalculating the
	   variance and	mean if	applicable (see	"variance()").

       covariance() cov()
	   Returns a Statistics::Basic::Covariance object.  Arguments to
	   "covariance()" or "cov()" must be array ref or vector objects.
	   There must be precisely two arguments (or none, setting the vectors
	   to two empty	ones), and they	must be	the same length.

       correlation() cor() corr()
	   Returns a Statistics::Basic::Correlation object.  Arguments to
	   "correlation()" or "cor()"/"corr()" must be array ref or vector
	   objects.  There must	be precisely two arguments (or none, setting
	   the vectors to two empty ones), and they must be the	same length.

       leastsquarefit()	LSF() lsf()
	   Returns a Statistics::Basic::LeastSquareFit object.	Arguments to
	   "leastsquarefit()" or "lsf()"/"LSF()" must be array ref or vector
	   objects.  There must	be precisely two arguments (or none, setting
	   the vectors to two empty ones), and they must be the	same length.

	   Returns a Statistics::Basic::ComputedVector object.	Argument must
	   be a	blessed	vector object.	See the	section	on "COMPUTED VECTORS"
	   for more information	on this.

       handle_missing_values() handle_missing()
	   Returns two Statistics::Basic::ComputedVector objects.  Arguments
	   to this function should be two vector arguments.  See the section
	   on "MISSING VALUES" for further information on this function.

       Sometimes it will be handy to have a vector computed from another (or
       at least	that updates based on the first).  Consider the	case of

	   my @a = ( (1,2,3) x 7, 15 );
	   my @b = ( (1,2,3) x 7 );

	   my $v1 = vector(@a);
	   my $v2 = vector(@b);
	   my $v3 = computed($v1);
	      $v3->set_filter(sub {
		  my $m	= mean($v1);
		  my $s	= stddev($v1);

		  grep { abs($_-$m) <= $s } @_;

       This filter sets	$v3 to always be equal to $v1 such that	all the
       elements	that differ from the mean by more than a standard deviation
       are removed.  As	such, "$v2" eq "$v3" since 15 is clearly an outlier by

	   print "$v1\n";
	   print "$v3\n";

       ... prints:

	   [1, 2, 3, 1,	2, 3, 1, 2, 3, 1, 2, 3,	1, 2, 3, 1, 2, 3, 1, 2,	3, 15]
	   [1, 2, 3, 1,	2, 3, 1, 2, 3, 1, 2, 3,	1, 2, 3, 1, 2, 3, 1, 2,	3]

       Something I get asked about quite a lot is, "can	S::B handle missing
       values?"	 The answer used to be,	"that really depends on	your data set,
       use grep," but I	recently decided (5/29/09) that	it was time to just go
       ahead and add this feature.

       Strictly	speaking, the feature was already there.  You simply need to
       add a couple filters to your data.  See "t/75_filtered_missings.t" for
       the test	example.

       This is what people usually mean	when they ask if S::B can "handle"
       missing data:

	   my $v1 = vector(1,2,3,undef,4);
	   my $v2 = vector(1,2,3,4, undef);
	   my $v3 = computed($v1);
	   my $v4 = computed($v2);

	   $v3->set_filter(sub {
	       my @v = $v2->query;
	       map {$_[$_]} grep { defined $v[$_] and defined $_[$_] } 0 .. $#_;

	   $v4->set_filter(sub {
	       my @v = $v1->query;
	       map {$_[$_]} grep { defined $v[$_] and defined $_[$_] } 0 .. $#_;

	   print "$v1 $v2\n"; #	prints:	[1, 2, 3, _, 4]	[1, 2, 3, 4, _]
	   print "$v3 $v4\n"; #	prints:	[1, 2, 3] [1, 2, 3]

       But I've	made it	even simpler.  Since this is such a common request, I
       have provided a helper function to build	the filters automatically:

	   my $v1 = vector(1,2,3,undef,4);
	   my $v2 = vector(1,2,3,4, undef);

	   my ($f1, $f2) = handle_missing_values($v1, $v2);

	   print "$f1 $f2\n"; #	prints:	[1, 2, 3] [1, 2, 3]

       Note that in practice, you would	still manipulate (insert, and shift)
       $v1 and $v2, not	the computed vectors.  But for correlations and	the
       like, you would use $f1 and $f2.


	   my $correlation = correlation($f1, $f2);

       You can still insert on $f1 and $f2, but	it updates the input vector
       rather than the computed	one (which is just a filter handler).

       Most of the objects have	a variety of query functions that allow	you to
       extract the objects used	within.	 Although, the objects are smart
       enough to prevent needless duplication.	That is, the following would
       test would pass:

	   use Statistics::Basic qw(:all);

	   my $v1 = vector(1,2,3,4,5);
	   my $v2 = vector($v1);
	   my $sd = stddev( $v1	);
	   my $v3 = $sd->query_vector;
	   my $m1 = mean( $v1 );
	   my $m2 = $sd->query_mean;
	   my $m3 = Statistics::Basic::Mean->new( $v1 );
	   my $v4 = $m3->query_vector;

	   use Scalar::Util qw(refaddr);
	   use Test; plan tests	=> 5;

	   ok( refaddr($v1), refaddr($v2) );
	   ok( refaddr($v2), refaddr($v3) );
	   ok( refaddr($m1), refaddr($m2) );
	   ok( refaddr($m2), refaddr($m3) );
	   ok( refaddr($v3), refaddr($v4) );

	   # this is t/54_* in the distribution

       Also, note that the mean	is only	calculated once	even though we've
       calculated a variance and a standard deviation above.

       Suppose you'd like a copy of the	Statistics::Basic::Variance object
       that the	Statistics::Basic::StdDev object is using.  All	of the objects
       within should be	accessible with	query functions	as follows.

	   This	method exists in all of	the objects.
	   Statistics::Basic::LeastSquareFit is	the only one that returns two
	   values (alpha and beta) as a	list.  Statistics::Basic::Vector
	   returns either the list of elements in the vector, or reference to
	   that	array (depending on the	context).  All of the other "query()"
	   methods return a single number, the number the module purports to

	   Returns the Statistics::Basic::Mean object used by
	   Statistics::Basic::Variance and Statistics::Basic::StdDev.

	   Returns the first Statistics::Basic::Mean object used by
	   Statistics::Basic::Covariance, Statistics::Basic::Correlation and

	   Returns the second Statistics::Basic::Mean object used by
	   Statistics::Basic::Covariance, and Statistics::Basic::Correlation.

	   Returns the Statistics::Basic::Covariance object used by
	   Statistics::Basic::Correlation and

	   Returns the Statistics::Basic::Variance object used by

	   Returns the first Statistics::Basic::Variance object	used by

	   Returns the Statistics::Basic::Vector object	used by	any of the
	   single vector modules.

	   Returns the first Statistics::Basic::Vector object used by any of
	   the two vector modules.

	   Returns the second Statistics::Basic::Vector	object used by any of
	   the two vector modules.

	   Statistics::Basic::Mode objects sometimes return
	   Statistics::Basic::Vector objects instead of	numbers.  When
	   "is_multimodal()" is	true, the mode is a vector, not	a scalar.

	   Statistics::Basic::LeastSquareFit is	meant for finding a line of
	   best	fit.  This function can	be used	to find	the "y"	for a given
	   "x" based on	the calculated $beta (slope) and $alpha	(y-offset).

	   Statistics::Basic::LeastSquareFit is	meant for finding a line of
	   best	fit.  This function can	be used	to find	the "x"	for a given
	   "y" based on	the calculated $beta (slope) and $alpha	(y-offset).

	   This	function can produce divide-by-zero errors since it must
	   divide by the slope to find the "x" value.  (The slope should
	   rarely be zero though, that's a vertical line and would represent
	   very	odd data points.)

       These objects are all intended to be useful while processing long
       columns of data,	like data you'd	find in	a database.

	   Vectors try to stay the same	size when they accept new elements,
	   FIFO	style.

	       my $v1 =	vector(1,2,3); # a 3 touple
		  $v1->insert(4); # still a 3 touple

	       print "$v1\n"; #	prints:	[2, 3, 4]

	       $v1->insert(7); # still a 3 touple
	       print "$v1\n"; #	prints:	[3, 4, 7]

	   All of the other Statistics::Basic modules have this	function too.
	   The modules that track two vectors will need	two arguments to
	   insert though.

	       my $mean	= mean([1,2,3]);

	       print "mean: $mean\n"; #	prints 3 ... (2+3+4)/3

	       my $correlation = correlation($mean->query_vector,

	       print "correlation: $correlation\n"; # 1

	       print "correlation: $correlation\n"; # 0.5

	   Also, note that the underlying vectors keep track of	recalculating

	       my $v = vector(1,2,3);
	       my $m = mean($v);
	       my $s = stddev($v);

	   The mean has	not been calculated yet.

	       print "$s; $m\n"; # 0.82; 2

	   The mean has	been calculated	once (even though the
	   Statistics::Basic::StdDev uses it).

	       $v->insert(4); print "$s; $m\n";	0.82; 3
	       $m->insert(5); print "$s; $m\n";	0.82; 4
	       $s->insert(6); print "$s; $m\n";	0.82; 5

	   The mean has	been calculated	thrice more and	only thrice more.

       append()	ginsert()
	   You can grow	the vectors instead of sliding them (FIFO). For	this,
	   use "append()" (or "ginsert()", same	thing).

	       my $v = vector(1,2,3);
	       my $m = mean($v);
	       my $s = stddev($v);

	       $v->append(4); print "$s; $m\n";	1.12; 2.5
	       $m->append(5); print "$s; $m\n";	1.41; 3
	       $s->append(6); print "$s; $m\n";	1.71; 1.71

	       print "$v\n"; # [1, 2, 3, 4, 5, 6]
	       print "$s\n"; # 1.71

	   Of course, with a correlation, or a covariance, it'd	look more like

	       my $c = correlation([1,2,3], [3,4,5]);

	       print "c=$c\n"; # c=0.98

	   This	allows you to set the vector to	a known	state.	It takes
	   either array	ref or vector objects.

	       my $v1 =	vector(1,2,3);
	       my $v2 =	$v1->copy;

	       my $m = mean();


	       my $c = correlation();

	       $c->set_vector([1,2,3], [4,5,6]);

	   This	sets the size of the vector.  When the vector is made bigger,
	   the vector is filled	to the new length with leading zeros (i.e.,
	   they	are the	first to be kicked out after new "insert()"s.

	       my $v = vector(1,2,3);

	       print "$v\n"; # [0, 0, 0, 0, 1, 2, 3]

	       my $m = mean();

	       print "", $m->query_vector, "\n";
		# [0, 0, 0, 0, 0, 0, 0]

	       my $c = correlation([3],[3]);

	       print "", $c->query_vector1, "\n";
	       print "", $c->query_vector2, "\n";
		# [0, 0, 0, 0, 0, 0, 3]
		# [0, 0, 0, 0, 0, 0, 3]

       Each of the following options can be specified on package import	like

	   use Statistics::Basic qw(unbias=0); # start with unbias disabled
	   use Statistics::Basic qw(unbias=1); # start with unbias enabled

       When specified on import, each option has certain defaults.

	   use Statistics::Basic qw(unbias); # start with unbias enabled
	   use Statistics::Basic qw(nofill); # start with nofill enabled
	   use Statistics::Basic qw(toler);  # start with toler	disabled
	   use Statistics::Basic qw(ipres);  # start with ipres=2

       Additionally, with the exception	of "ignore_env", they can all be
       accessed	via package variables of the same name in all upper case.

	   # code code code

	   $Statistics::Basic::UNBIAS =	0; # turn UNBIAS off

	   # code code code

	   $Statistics::Basic::UNBIAS =	1; # turn it back on

	   # code code code

	       local $Statistics::Basic::DEBUG_STATS_B = 1; # debug, this block	only

       Special caveat: "toler" can in fact be changed via the package var
       (e.g., "$Statistics::Basic::TOLER=0.0001").  But, for speed reasons, it
       must be defined before any other	packages are imported or it will not
       actually	do anything when changed.

	   This	module uses the	sum(X -	mean(X))/N definition of variance.

	   If you wish to use the unbiased, sum(X-mean(X)/(N-1)	definition,
	   then	set the	$Statistics::Basic::UNBIAS true	(possibly with "use
	   Statistics::Basic qw(unbias)").

	   This	can be changed at any time with	the package variable or	at
	   compile time.

	   This	feature	was requested by "Robert McGehee

	   [NOTE 2008-11-06:
	   <>,	this can also
	   be called "population (n)" vs "sample (n-1)"	and is indeed fully
	   addressed right here!]

	   "ipres" defaults to 2.  It is passed	to Number::Format as the
	   second argument to format_number() during string interpolation
	   (see: overload).

	   When	set, $Statistics::Basic::TOLER (which is not enabled by
	   default), instructs the stats objects to test true when within some
	   tolerable range, pretty much	like this:

	       sub is_equal {
		   return abs($_[0]-$_[1])<$Statistics::Basic::TOLER
		       if defined($Statistics::Basic::TOLER)

		   return $_[0]	== $_[1]

	   For performance reasons, this must be defined before	the import of
	   any other Statistics::Basic modules or the modules will fail	to
	   overload the	"==" operator.

	   $Statistics::Basic::TOLER totally disabled:

	       use Statistics::Basic qw(:all toler);

	   $Statistics::Basic::TOLER disabled, but changeable:

	       use Statistics::Basic qw(:all toler=0);

	       $Statistics::Basic::TOLER = 0.000_001;

	   You can change the tolerance	at runtime, but	it must	be set (or
	   unset) at compile time before the packages load.

	   Normally when you set the size of a vector it automatically fills
	   with	zeros on the first-out side of the vector.  You	can disable
	   the autofilling with	this option.  It can be	changed	at any time.

	   Enable debugging with "use Statistics::Basic	qw(debug)" or disable
	   a specific level (including 0 to disable) with "use
	   Statistics::Basic qw(debug=2)".

	   This	is also	accessible at runtime using
	   $Statistics::Basic::DEBUG_STATS_B and can be	switched on and	off at
	   any time.

	   Normally the	defaults for these options can be changed in the
	   environment of the program.	Example:

	       UNBIAS=1	perl ./

	   This	does the same thing as "$Statistics::Basic::UNBIAS=1" or "use
	   Statistics::Basic qw(unbias)" unless	you disable the	%ENV checking
	   with	this option.

	       use Statistics::Basic qw(ignore_env);

       You can change the defaults (assuming ignore_env	is not used) from your
       bash prompt.  Example:

	   DEBUG_STATS_B=1 perl	./

	   Sets	the default value of "debug".

	   Sets	the default value of "unbias".

	   Sets	the default value of "nofill".

	   Sets	the default value of "ipres".

	   Sets	the default value of "toler".

       All of the objects are true in numeric context.	All of the objects
       print useful strings when evaluated as a	string.	 Most of the objects
       evaluate	usefully as numbers, although Statistics::Basic::Vector
       objects,	Statistics::Basic::ComputedVector objects, and
       Statistics::Basic::LeastSquareFit objects do not	-- they	instead	raise
       an error.

Author's note on Statistics::Descriptive
       I've been asked a couple	times now why I	don't link to
       Statistics::Descriptive in my see also section.	As a rule, I only link
       to packages there that I	think are related or that I actually used in
       the package construction.  I've never personally	used Descriptive, but
       it surely seems to do quite a lot more.	In a sense, this package
       really doesn't do statistics, not like a	scientist would	think about it
       anyway.	So I always figured people could find their own	way to
       Descriptive anyway.

       The one thing this package does do, that	I don't	think Descriptive does
       (correct	me if I'm wrong) is time difference computations.  If there
       are say,	200 things in the mean object, then after inserting (using
       this package) there'll still be 200 things, allowing the	computation of
       a moving	average, moving	stddev,	moving correlation, etc.  You might
       argue that this is rarely needed, but it	is really the only time	I need
       to compute these	things.

	 while(	$data =	$fetch_sth->fetchrow_arrayref )	{
	     $moving_avg_sth->execute(0	+ $mean);

       Since I opened the topic	I'd also like to mention that I	find this
       package easier to use.  That is a matter	of taste and since I wrote
       this, you might say I'm a little	biased.	 Your mileage may vary.

       Paul Miller "<>"

       I am using this software	in my own projects...  If you find bugs,
       please please please let	me know. :) Actually, let me know if you find
       it handy	at all.	 Half the fun of releasing this	stuff is knowing that
       people use it.

       Copyright 2012 Paul Miller -- Licensed under the	LGPL version 2.

       perl(1),	Number::Format,	overload, Statistics::Basic::Vector,
       Statistics::Basic::ComputedVector, Statistics::Basic::_OneVectorBase,
       Statistics::Basic::Mean,	Statistics::Basic::Median,
       Statistics::Basic::Mode,	Statistics::Basic::Variance,
       Statistics::Basic::StdDev, Statistics::Basic::_TwoVectorBase,
       Statistics::Basic::Correlation, Statistics::Basic::Covariance,

perl v5.32.0			  2012-01-23		  Statistics::Basic(3)


Want to link to this manual page? Use this URL:

home | help