Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
AnyEvent::Intro(3)    User Contributed Perl Documentation   AnyEvent::Intro(3)

NAME
       AnyEvent::Intro - an introductory tutorial to AnyEvent

Introduction to	AnyEvent
       This is a tutorial that will introduce you to the features of AnyEvent.

       The first part introduces the core AnyEvent module (after swamping you
       a bit in	evangelism), which might already provide all you ever need: If
       you are only interested in AnyEvent's event handling capabilities, read
       no further.

       The second part focuses on network programming using sockets, for which
       AnyEvent	offers a lot of	support	you can	use, and a lot of workarounds
       around portability quirks.

What is	AnyEvent?
       If you don't care for the whys and want to see code, skip this section!

       AnyEvent	is first of all	just a framework to do event-based
       programming. Typically such frameworks are an all-or-nothing thing: If
       you use one such	framework, you can't (easily, or even at all) use
       another in the same program.

       AnyEvent	is different - it is a thin abstraction	layer on top of	other
       event loops, just like DBI is an	abstraction of many different database
       APIs. Its main purpose is to move the choice of the underlying
       framework (the event loop) from the module author to the	program	author
       using the module.

       That means you can write	code that uses events to control what it does,
       without forcing other code in the same program to use the same
       underlying framework as you do -	i.e. you can create a Perl module that
       is event-based using AnyEvent, and users	of that	module can still
       choose between using Gtk2, Tk, Event (or	run inside Irssi or rxvt-
       unicode)	or any other supported event loop. AnyEvent even comes with
       its own pure-perl event loop implementation, so your code works
       regardless of other modules that	might or might not be installed. The
       latter is important, as AnyEvent	does not have any hard dependencies to
       other modules, which makes it easy to install, for example, when	you
       lack a C	compiler. No matter what environment, AnyEvent will just cope
       with it.

       A typical limitation of existing	Perl modules such as Net::IRC is that
       they come with their own	event loop: In Net::IRC, a program which uses
       it needs	to start the event loop	of Net::IRC. That means	that one
       cannot integrate	this module into a Gtk2	GUI for	instance, as that
       module, too, enforces the use of	its own	event loop (namely Glib).

       Another example is LWP: it provides no event interface at all. It's a
       pure blocking HTTP (and FTP etc.) client	library, which usually means
       that you	either have to start another process or	have to	fork for a
       HTTP request, or	use threads (e.g. Coro::LWP), if you want to do
       something else while waiting for	the request to finish.

       The motivation behind these designs is often that a module doesn't want
       to depend on some complicated XS-module (Net::IRC), or that it doesn't
       want to force the user to use some specific event loop at all (LWP),
       out of fear of severly limiting the usefulness of the module: If	your
       module requires Glib, it	will not run in	a Tk program.

       AnyEvent	solves this dilemma, by	not forcing module authors to either:

       - write their own event loop (because it	guarantees the availability of
       an event	loop everywhere	- even on windows with no extra	modules
       installed).
       - choose	one specific event loop	(because AnyEvent works	with most
       event loops available for Perl).

       If the module author uses AnyEvent for all his (or her) event needs (IO
       events, timers, signals,	...) then all other modules can	just use his
       module and don't	have to	choose an event	loop or	adapt to his event
       loop. The choice	of the event loop is ultimately	made by	the program
       author who uses all the modules and writes the main program. And	even
       there he	doesn't	have to	choose,	he can just let	AnyEvent choose	the
       most efficient event loop available on the system.

       Read more about this in the main	documentation of the AnyEvent module.

Introduction to	Event-Based Programming
       So what exactly is programming using events? It quite simply means that
       instead of your code actively waiting for something, such as the	user
       entering	something on STDIN:

	  $| = 1; print	"enter your name> ";

	  my $name = <STDIN>;

       You instead tell	your event framework to	notify you in the event	of
       some data being available on STDIN, by using a callback mechanism:

	  use AnyEvent;

	  $| = 1; print	"enter your name> ";

	  my $name;

	  my $wait_for_input = AnyEvent->io (
	     fh	  => \*STDIN, #	which file handle to check
	     poll => "r",     #	which event to wait for	("r"ead	data)
	     cb	  => sub {    #	what callback to execute
		$name =	<STDIN>; # read	it
	     }
	  );

	  # do something else here

       Looks more complicated, and surely is, but the advantage	of using
       events is that your program can do something else instead of waiting
       for input (side note: combining AnyEvent	with a thread package such as
       Coro can	recoup much of the simplicity, effectively getting the best of
       two worlds).

       Waiting as done in the first example is also called "blocking" the
       process because you "block"/keep	your process from executing anything
       else while you do so.

       The second example avoids blocking by only registering interest in a
       read event, which is fast and doesn't block your	process. The callback
       will be called only when	data is	available and can be read without
       blocking.

       The "interest" is represented by	an object returned by "AnyEvent->io"
       called a	"watcher" object - thus	named because it "watches" your	file
       handle (or other	event sources) for the event you are interested	in.

       In the example above, we	create an I/O watcher by calling the
       "AnyEvent->io" method. A	lack of	further	interest in some event is
       expressed by simply forgetting about its	watcher, for example by
       "undef"-ing the only variable it	is stored in. AnyEvent will
       automatically clean up the watcher if it	is no longer used, much	like
       Perl closes your	file handles if	you no longer use them anywhere.

       A short note on callbacks

       A common	issue that hits	people is the problem of passing parameters to
       callbacks. Programmers used to languages	such as	C or C++ are often
       used to a style where one passes	the address of a function (a function
       reference) and some data	value, e.g.:

	  sub callback {
	     my	($arg) = @_;

	     $arg->method;
	  }

	  my $arg = ...;

	  call_me_back_later \&callback, $arg;

       This is clumsy, as the place where behaviour is specified (when the
       callback	is registered) is often	far away from the place	where
       behaviour is implemented. It also doesn't use Perl syntax to invoke the
       code. There is also an abstraction penalty to pay as one	has to name
       the callback, which often is unnecessary	and leads to nonsensical or
       duplicated names.

       In Perl,	one can	specify	behaviour much more directly by	using
       closures. Closures are code blocks that take a reference	to the
       enclosing scope(s) when they are	created. This means lexical variables
       in scope	when a closure is created can be used inside the closure:

	  my $arg = ...;

	  call_me_back_later sub { $arg->method	};

       Under most circumstances, closures are faster, use fewer	resources and
       result in much clearer code than	the traditional	approach. Faster,
       because parameter passing and storing them in local variables in	Perl
       is relatively slow. Fewer resources, because closures take references
       to existing variables without having to create new ones,	and clearer
       code because it is immediately obvious that the second example calls
       the "method" method when	the callback is	invoked.

       Apart from these, the strongest argument	for using closures with
       AnyEvent	is that	AnyEvent does not allow	passing	parameters to the
       callback, so closures are the only way to achieve that in most cases
       :->

       A little	hint to	catch mistakes

       AnyEvent	does not check the parameters you pass in, at least not	by
       default.	to enable checking, simply start your program with
       "AE_STRICT=1" in	the environment, or put	"use AnyEvent::Strict" near
       the top of your program:

	  AE_STRICT=1 perl myprogram

       You can find more info on this and additional debugging aids later in
       this introduction.

   Condition Variables
       Back to the I/O watcher example:	The code is not	yet a fully working
       program,	and will not work as-is. The reason is that your callback will
       not be invoked out of the blue; you have	to run the event loop first.
       Also, event-based programs need to block	sometimes too, such as when
       there is	nothing	to do, and everything is waiting for new events	to
       arrive.

       In AnyEvent, this is done using condition variables. Condition
       variables are named "condition variables" because they represent	a
       condition that is initially false and needs to be fulfilled.

       You can also call them "merge points", "sync points", "rendezvous
       ports" or even callbacks	and many other things (and they	are often
       called these names in other frameworks).	The important point is that
       you can create them freely and later wait for them to become true.

       Condition variables have	two sides - one	side is	the "producer" of the
       condition (whatever code	detects	and flags the condition), the other
       side is the "consumer" (the code	that waits for that condition).

       In our example in the previous section, the producer is the event
       callback	and there is no	consumer yet - let's change that right now:

	  use AnyEvent;

	  $| = 1; print	"enter your name> ";

	  my $name;

	  my $name_ready = AnyEvent->condvar;

	  my $wait_for_input = AnyEvent->io (
	     fh	  => \*STDIN,
	     poll => "r",
	     cb	  => sub {
		$name =	<STDIN>;
		$name_ready->send;
	     }
	  );

	  # do something else here

	  # now	wait until the name is available:
	  $name_ready->recv;

	  undef	$wait_for_input; # watcher no longer needed

	  print	"your name is $name\n";

       This program creates an AnyEvent	condvar	by calling the
       "AnyEvent->condvar" method. It then creates a watcher as	usual, but
       inside the callback it "send"s the $name_ready condition	variable,
       which causes whoever is waiting on it to	continue.

       The "whoever" in	this case is the code that follows, which calls
       "$name_ready->recv": The	producer calls "send", the consumer calls
       "recv".

       If there	is no $name available yet, then	the call to
       "$name_ready->recv" will	halt your program until	the condition becomes
       true.

       As the names "send" and "recv" imply, you can actually send and receive
       data using this,	for example, the above code could also be written like
       this, without an	extra variable to store	the name in:

	  use AnyEvent;

	  $| = 1; print	"enter your name> ";

	  my $name_ready = AnyEvent->condvar;

	  my $wait_for_input = AnyEvent->io (
	     fh	=> \*STDIN, poll => "r",
	     cb	=> sub { $name_ready->send (scalar <STDIN>) }
	  );

	  # do something else here

	  # now	wait and fetch the name
	  my $name = $name_ready->recv;

	  undef	$wait_for_input; # watcher no longer needed

	  print	"your name is $name\n";

       You can pass any	number of arguments to "send", and every subsequent
       call to "recv" will return them.

   The "main loop"
       Most event-based	frameworks have	something called a "main loop" or
       "event loop run function" or something similar.

       Just like in "recv" AnyEvent, these functions need to be	called
       eventually so that your event loop has a	chance of actually looking for
       the events you are interested in.

       For example, in a Gtk2 program, the above example could also be written
       like this:

	  use Gtk2 -init;
	  use AnyEvent;

	  ############################################
	  # create a window and	some label

	  my $window = new Gtk2::Window	"toplevel";
	  $window->add (my $label = new	Gtk2::Label "soon replaced by name");

	  $window->show_all;

	  ############################################
	  # do our AnyEvent stuff

	  $| = 1; print	"enter your name> ";

	  my $wait_for_input = AnyEvent->io (
	     fh	=> \*STDIN, poll => "r",
	     cb	=> sub {
		# set the label
		$label->set_text (scalar <STDIN>);
		print "enter another name> ";
	     }
	  );

	  ############################################
	  # Now	enter Gtk2's event loop

	  main Gtk2;

       No condition variable anywhere in sight - instead, we just read a line
       from STDIN and replace the text in the label. In	fact, since nobody
       "undef"s	$wait_for_input	you can	enter multiple lines.

       Instead of waiting for a	condition variable, the	program	enters the
       Gtk2 main loop by calling "Gtk2->main", which will block	the program
       and wait	for events to arrive.

       This also shows that AnyEvent is	quite flexible - you didn't have to do
       anything	to make	the AnyEvent watcher use Gtk2 (actually	Glib) -	it
       just worked.

       Admittedly, the example is a bit	silly -	who would want to read names
       from standard input in a	Gtk+ application? But imagine that instead of
       doing that, you make an HTTP request in the background and display its
       results.	In fact, with event-based programming you can make many	HTTP
       requests	in parallel in your program and	still provide feedback to the
       user and	stay interactive.

       And in the next part you	will see how to	do just	that - by implementing
       an HTTP request,	on our own, with the utility modules AnyEvent comes
       with.

       Before that, however, let's briefly look	at how you would write your
       program using only AnyEvent, without ever calling some other event
       loop's run function.

       In the example using condition variables, we used those to start
       waiting for events, and in fact,	condition variables are	the solution:

	  my $quit_program = AnyEvent->condvar;

	  # create AnyEvent watchers (or not) here

	  $quit_program->recv;

       If any of your watcher callbacks	decide to quit (this is	often called
       an "unloop" in other frameworks), they can just call
       "$quit_program->send". Of course, they could also decide	not to and
       call "exit" instead, or they could decide never to quit (e.g. in	a
       long-running daemon program).

       If you don't need some clean quit functionality and just	want to	run
       the event loop, you can do this:

	  AnyEvent->condvar->recv;

       And this	is, in fact, the closest to the	idea of	a main loop run
       function	that AnyEvent offers.

   Timers and other event sources
       So far, we have used only I/O watchers. These are useful	mainly to find
       out whether a socket has	data to	read, or space to write	more data. On
       sane operating systems this also	works for console windows/terminals
       (typically on standard input), serial lines, all	sorts of other
       devices,	basically almost everything that has a file descriptor but
       isn't a file itself. (As	usual, "sane" excludes windows - on that
       platform	you would need different functions for all of these,
       complicating code immensely - think "socket only" on windows).

       However,	I/O is not everything -	the second most	important event	source
       is the clock. For example when doing an HTTP request you	might want to
       time out	when the server	doesn't	answer within some predefined amount
       of time.

       In AnyEvent, timer event	watchers are created by	calling	the
       "AnyEvent->timer" method:

	  use AnyEvent;

	  my $cv = AnyEvent->condvar;

	  my $wait_one_and_a_half_seconds = AnyEvent->timer (
	     after => 1.5,  # after how	many seconds to	invoke the cb?
	     cb	   => sub { # the callback to invoke
		$cv->send;
	     },
	  );

	  # can	do something else here

	  # now	wait till our time has come
	  $cv->recv;

       Unlike I/O watchers, timers are only interested in the amount of
       seconds they have to wait. When (at least) that amount of time has
       passed, AnyEvent	will invoke your callback.

       Unlike I/O watchers, which will call your callback as many times	as
       there is	data available,	timers are normally one-shot: after they have
       "fired" once and	invoked	your callback, they are	dead and no longer do
       anything.

       To get a	repeating timer, such as a timer firing	roughly	once per
       second, you can specify an "interval" parameter:

	  my $once_per_second =	AnyEvent->timer	(
	     after => 0,    # first invoke ASAP
	     interval => 1, # then invoke every	second
	     cb	   => sub { # the callback to invoke
		$cv->send;
	     },
	  );

       More esoteric sources

       AnyEvent	also has some other, more esoteric event sources you can tap
       into: signal, child and idle watchers.

       Signal watchers can be used to wait for "signal events",	which means
       your process was	sent a signal (such as "SIGTERM" or "SIGUSR1").

       Child-process watchers wait for a child process to exit.	They are
       useful when you fork a separate process and need	to know	when it	exits,
       but you do not want to wait for that by blocking.

       Idle watchers invoke their callback when	the event loop has handled all
       outstanding events, polled for new events and didn't find any, i.e.,
       when your process is otherwise idle. They are useful if you want	to do
       some non-trivial	data processing	that can be done when your program
       doesn't have anything better to do.

       All these watcher types are described in	detail in the main AnyEvent
       manual page.

       Sometimes you also need to know what the	current	time is:
       "AnyEvent->now" returns the time	the event toolkit uses to schedule
       relative	timers,	and is usually what you	want. It is often cached
       (which means it can be a	bit outdated). In that case, you can use the
       more costly "AnyEvent->time" method which will ask your operating
       system for the current time, which is slower, but also more up to date.

Network	programming and	AnyEvent
       So far you have seen how	to register event watchers and handle events.

       This is a great foundation to write network clients and servers,	and
       might be	all that your module (or program) ever requires, but writing
       your own	I/O buffering again and	again becomes tedious, not to mention
       that it attracts	errors.

       While the core AnyEvent module is still small and self-contained, the
       distribution comes with some very useful	utility	modules	such as
       AnyEvent::Handle, AnyEvent::DNS and AnyEvent::Socket. These can make
       your life as a non-blocking network programmer a	lot easier.

       Here is a quick overview	of these three modules:

   AnyEvent::DNS
       This module allows fully	asynchronous DNS resolution. It	is used	mainly
       by AnyEvent::Socket to resolve hostnames	and service ports for you, but
       is a great way to do other DNS resolution tasks,	such as	reverse
       lookups of IP addresses for log files.

   AnyEvent::Handle
       This module handles non-blocking	IO on (socket-,	pipe- etc.) file
       handles in an event based manner. It provides a wrapper object around
       your file handle	that provides queueing and buffering of	incoming and
       outgoing	data for you.

       It also implements the most common data formats,	such as	text lines, or
       fixed and variable-width	data blocks.

   AnyEvent::Socket
       This module provides you	with functions that handle socket creation and
       IP address magic. The two main functions	are "tcp_connect" and
       "tcp_server". The former	will connect a (streaming) socket to an
       internet	host for you and the later will	make a server socket for you,
       to accept connections.

       This module also	comes with transparent IPv6 support, this means: If
       you write your programs with this module, you will be IPv6 ready
       without doing anything special.

       It also works around a lot of portability quirks	(especially on the
       windows platform), which	makes it even easier to	write your programs in
       a portable way (did you know that windows uses different	error codes
       for all socket functions	and that Perl does not know about these? That
       "Unknown	error 10022" (which is "WSAEINVAL") can	mean that our
       "connect" call was successful? That unsuccessful	TCP connects might
       never be	reported back to your program? That "WSAEINPROGRESS" means
       your "connect" call was ignored instead of being	in progress?
       AnyEvent::Socket	works around all of these Windows/Perl bugs for	you).

   Implementing	a parallel finger client with non-blocking connects and
       AnyEvent::Socket
       The finger protocol is one of the simplest protocols in use on the
       internet. Or in use in the past,	as almost nobody uses it anymore.

       It works	by connecting to the finger port on another host, writing a
       single line with	a user name and	then reading the finger	response, as
       specified by that user. OK, RFC 1288 specifies a	vastly more complex
       protocol, but it	basically boils	down to	this:

	  # telnet freebsd.org finger
	  Trying 8.8.178.135...
	  Connected to freebsd.org (8.8.178.135).
	  Escape character is '^]'.
	  larry
	  Login: lile				  Name:	Larry Lile
	  Directory: /home/lile			  Shell: /usr/local/bin/bash
	  No Mail.
	  Mail forwarded to: lile@stdio.com
	  No Plan.

       So let's	write a	little AnyEvent	function that makes a finger request:

	  use AnyEvent;
	  use AnyEvent::Socket;

	  sub finger($$) {
	     my	($user,	$host) = @_;

	     # use a condvar to	return results
	     my	$cv = AnyEvent->condvar;

	     # first, connect to the host
	     tcp_connect $host,	"finger", sub {
		# the callback receives	the socket handle - or nothing
		my ($fh) = @_
		   or return $cv->send;

		# now write the	username
		syswrite $fh, "$user\015\012";

		my $response;

		# register a read watcher
		my $read_watcher; $read_watcher	= AnyEvent->io (
		   fh	=> $fh,
		   poll	=> "r",
		   cb	=> sub {
		      my $len =	sysread	$fh, $response,	1024, length $response;

		      if ($len <= 0) {
			 # we are done,	or an error occured, lets ignore the latter
			 undef $read_watcher; #	no longer interested
			 $cv->send ($response);	# send results
		      }
		   },
		);
	     };

	     # pass $cv	to the caller
	     $cv
	  }

       That's a	mouthful! Let's	dissect	this function a	bit, first the overall
       function	and execution flow:

	  sub finger($$) {
	     my	($user,	$host) = @_;

	     # use a condvar to	return results
	     my	$cv = AnyEvent->condvar;

	     # first, connect to the host
	     tcp_connect $host,	"finger", sub {
		...
	     };

	     $cv
	  }

       This isn't too complicated, just	a function with	two parameters that
       creates a condition variable $cv, initiates a TCP connect to $host, and
       returns $cv. The	caller can use the returned $cv	to receive the finger
       response, but one could equally well pass a third argument, a callback,
       to the function.

       Since we	are programming	event'ish, we do not wait for the connect to
       finish -	it could block the program for a minute	or longer!

       Instead,	we pass	"tcp_connect" a	callback to invoke when	the connect is
       done. The callback is called with the socket handle as its first
       argument	if the connect succeeds, and no	arguments otherwise. The
       important point is that it will always be called	as soon	as the outcome
       of the TCP connect is known.

       This style of programming is also called	"continuation style": the
       "continuation" is simply	the way	the program continues -	normally at
       the next	line after some	statement (the exception is loops or things
       like "return"). When we are interested in events, however, we instead
       specify the "continuation" of our program by passing a closure, which
       makes that closure the "continuation" of	the program.

       The "tcp_connect" call is like saying "return now, and when the
       connection is established or the	attempt	failed,	continue there".

       Now let's look at the callback/closure in more detail:

		# the callback receives	the socket handle - or nothing
		my ($fh) = @_
		   or return $cv->send;

       The first thing the callback does is to save the	socket handle in $fh.
       When there was an error (no arguments), then our	instinct as expert
       Perl programmers	would tell us to "die":

		my ($fh) = @_
		   or die "$host: $!";

       While this would	give good feedback to the user (if he happens to watch
       standard	error),	our program would probably stop	working	here, as we
       never report the	results	to anybody, certainly not the caller of	our
       "finger"	function, and most event loops continue	even after a "die"!

       This is why we instead "return",	but also call "$cv->send" without any
       arguments to signal to the condvar consumer that	something bad has
       happened. The return value of "$cv->send" is irrelevant,	as is the
       return value of our callback. The "return" statement is used for	the
       side effect of, well, returning immediately from	the callback.
       Checking	for errors and handling	them this way is very common, which is
       why this	compact	idiom is so handy.

       As the next step	in the finger protocol,	we send	the username to	the
       finger daemon on	the other side of our connection (the kernel.org
       finger service doesn't actually wait for	a username, but	the net	is
       running out of finger servers fast):

		syswrite $fh, "$user\015\012";

       Note that this isn't 100% clean socket programming - the	socket could,
       for whatever reasons, not accept	our data. When writing a small amount
       of data like in this example it doesn't matter, as a socket buffer is
       almost always big enough	for a mere "username", but for real-world
       cases you might need to implement some kind of write buffering -	or use
       AnyEvent::Handle, which handles these matters for you, as shown in the
       next section.

       What we do have to do is	implement our own read buffer -	the response
       data could arrive late or in multiple chunks, and we cannot just	wait
       for it (event-based programming,	you know?).

       To do that, we register a read watcher on the socket which waits	for
       data:

		my $read_watcher; $read_watcher	= AnyEvent->io (
		   fh	=> $fh,
		   poll	=> "r",

       There is	a trick	here, however: the read	watcher	isn't stored in	a
       global variable,	but in a local one - if	the callback returns, it would
       normally	destroy	the variable and its contents, which would in turn
       unregister our watcher.

       To avoid	that, we refer to the watcher variable in the watcher
       callback.  This means that, when	the "tcp_connect" callback returns,
       perl thinks (quite correctly) that the read watcher is still in use -
       namely inside the inner callback	- and thus keeps it alive even if
       nothing else in the program refers to it	anymore	(it is much like Baron
       MA1/4nchhausen keeping himself from dying by pulling himself out	of a
       swamp).

       The trick, however, is that instead of:

	  my $read_watcher = AnyEvent->io (...

       The program does:

	  my $read_watcher; $read_watcher = AnyEvent->io (...

       The reason for this is a	quirk in the way Perl works: variable names
       declared	with "my" are only visible in the next statement. If the whole
       "AnyEvent->io" call, including the callback, would be done in a single
       statement, the callback could not refer to the $read_watcher variable
       to "undef"ine it, so it is done in two statements.

       Whether you'd want to format it like this is of course a	matter of
       style.  This way	emphasizes that	the declaration	and assignment really
       are one logical statement.

       The callback itself calls "sysread" for as many times as	necessary,
       until "sysread" returns either an error or end-of-file:

		   cb	=> sub {
		      my $len =	sysread	$fh, $response,	1024, length $response;

		      if ($len <= 0) {

       Note that "sysread" has the ability to append data it reads to a	scalar
       if we specify an	offset,	a feature which	we make	use of in this
       example.

       When "sysread" indicates	we are done, the callback "undef"ines the
       watcher and then	"send"s	the response data to the condition variable.
       All this	has the	following effects:

       Undefining the watcher destroys it, as our callback was the only	one
       still having a reference	to it. When the	watcher	gets destroyed,	it
       destroys	the callback, which in turn means the $fh handle is no longer
       used, so	that gets destroyed as well. The result	is that	all resources
       will be nicely cleaned up by perl for us.

       Using the finger	client

       Now, we could probably write the	same finger client in a	simpler	way if
       we used "IO::Socket::INET", ignored the problem of multiple hosts and
       ignored IPv6 and	a few other things that	"tcp_connect" handles for us.

       But the main advantage is that we can not only run this finger function
       in the background, we even can run multiple sessions in parallel, like
       this:

	  my $f1 = finger "kuriyama", "freebsd.org";
	  my $f2 = finger "icculus?listarchives=1", "icculus.org";
	  my $f3 = finger "mikachu", "icculus.org";

	  print	"kuriyama's gpg	key\n"	  , $f1->recv, "\n";
	  print	"icculus' plan archive\n" , $f2->recv, "\n";
	  print	"mikachu's plan	zomgn\n"  , $f3->recv, "\n";

       It doesn't look like it,	but in fact all	three requests run in
       parallel. The code waits	for the	first finger request to	finish first,
       but that	doesn't	keep it	from executing them parallel: when the first
       "recv" call sees	that the data isn't ready yet, it serves events	for
       all three requests automatically, until the first request has finished.

       The second "recv" call might either find	the data is already there, or
       it will continue	handling events	until that is the case,	and so on.

       By taking advantage of network latencies, which allows us to serve
       other requests and events while we wait for an event on one socket, the
       overall time to do these	three requests will be greatly reduced,
       typically all three are done in the same	time as	the slowest of the
       three requests.

       By the way, you do not actually have to wait in the "recv" method on an
       AnyEvent	condition variable - after all,	waiting	is evil	- you can also
       register	a callback:

	  $f1->cb (sub {
	     my	$response = shift->recv;
	     # ...
	  });

       The callback will be invoked only when "send" is	called.	In fact,
       instead of returning a condition	variable you could also	pass a third
       parameter to your finger	function, the callback to invoke with the
       response:

	  sub finger($$$) {
	     my	($user,	$host, $cb) = @_;

       How you implement it is a matter	of taste - if you expect your function
       to be used mainly in an event-based program you would normally prefer
       to pass a callback directly. If you write a module and expect your
       users to	use it "synchronously" often (for example, a simple http-get
       script would not	really care much for events), then you would use a
       condition variable and tell them	"simply	"->recv" the data".

       Problems	with the implementation	and how	to fix them

       To make this example more real-world-ready, we would not	only implement
       some write buffering (for the paranoid, or maybe	denial-of-service
       aware security expert), but we would also have to handle	timeouts and
       maybe protocol errors.

       Doing this quickly gets unwieldy, which is why we introduce
       AnyEvent::Handle	in the next section, which takes care of all these
       details for you and lets	you concentrate	on the actual protocol.

   Implementing	simple HTTP and	HTTPS GET requests with	AnyEvent::Handle
       The AnyEvent::Handle module has been hyped quite	a bit in this document
       so far, so let's	see what it really offers.

       As finger is such a simple protocol, let's try something	slightly more
       complicated: HTTP/1.0.

       An HTTP GET request works by sending a single request line that
       indicates what you want the server to do	and the	URI you	want to	act it
       on, followed by as many "header"	lines ("Header:	data", same as e-mail
       headers)	as required for	the request, followed by an empty line.

       The response is formatted very similarly, first a line with the
       response	status,	then again as many header lines	as required, then an
       empty line, followed by any data	that the server	might send.

       Again, let's try	it out with "telnet" (I	condensed the output a bit -
       if you want to see the full response, do	it yourself).

	  # telnet www.google.com 80
	  Trying 209.85.135.99...
	  Connected to www.google.com (209.85.135.99).
	  Escape character is '^]'.
	  GET /test HTTP/1.0

	  HTTP/1.0 404 Not Found
	  Date:	Mon, 02	Jun 2008 07:05:54 GMT
	  Content-Type:	text/html; charset=UTF-8

	  <html><head>
	  [...]
	  Connection closed by foreign host.

       The "GET	..." and the empty line	were entered manually, the rest	of the
       telnet output is	google's response, in this case	a "404 not found" one.

       So, here	is how you would do it with "AnyEvent::Handle":

	  sub http_get {
	     my	($host,	$uri, $cb) = @_;

	     # store results here
	     my	($response, $header, $body);

	     my	$handle; $handle = new AnyEvent::Handle
		connect	 => [$host => 'http'],
		on_error => sub	{
		   $cb->("HTTP/1.0 500 $!");
		   $handle->destroy; # explicitly destroy handle
		},
		on_eof	 => sub	{
		   $cb->($response, $header, $body);
		   $handle->destroy; # explicitly destroy handle
		};

	     $handle->push_write ("GET $uri HTTP/1.0\015\012\015\012");

	     # now fetch response status line
	     $handle->push_read	(line => sub {
		my ($handle, $line) = @_;
		$response = $line;
	     });

	     # then the	headers
	     $handle->push_read	(line => "\015\012\015\012", sub {
		my ($handle, $line) = @_;
		$header	= $line;
	     });

	     # and finally handle any remaining	data as	body
	     $handle->on_read (sub {
		$body .= $_[0]->rbuf;
		$_[0]->rbuf = "";
	     });
	  }

       And now let's go	through	it step	by step. First,	as usual, the overall
       "http_get" function structure:

	  sub http_get {
	     my	($host,	$uri, $cb) = @_;

	     # store results here
	     my	($response, $header, $body);

	     my	$handle; $handle = new AnyEvent::Handle
		... create handle object

	     ... push data to write

	     ... push what to expect to	read queue
	  }

       Unlike in the finger example, this time the caller has to pass a
       callback	to "http_get". Also, instead of	passing	a URL as one would
       expect, the caller has to provide the hostname and URI -	normally you
       would use the "URI" module to parse a URL and separate it into those
       parts, but that is left to the inspired reader :)

       Since everything	else is	left to	the caller, all	"http_get" does	is
       initiate	the connection by creating the AnyEvent::Handle	object (which
       calls "tcp_connect" for us) and leave everything	else to	its callback.

       The handle object is created, unsurprisingly, by	calling	the "new"
       method of AnyEvent::Handle:

	     my	$handle; $handle = new AnyEvent::Handle
		connect	 => [$host => 'http'],
		on_error => sub	{
		   $cb->("HTTP/1.0 500 $!");
		   $handle->destroy; # explicitly destroy handle
		},
		on_eof	 => sub	{
		   $cb->($response, $header, $body);
		   $handle->destroy; # explicitly destroy handle
		};

       The "connect" argument tells AnyEvent::Handle to	call "tcp_connect" for
       the specified host and service/port.

       The "on_error" callback will be called on any unexpected	error, such as
       a refused connection, or	unexpected end-of-file while reading headers.

       Instead of having an extra mechanism to signal errors, connection
       errors are signalled by crafting	a special "response status line", like
       this:

	  HTTP/1.0 500 Connection refused

       This means the caller cannot distinguish	(easily) between locally-
       generated errors	and server errors, but it simplifies error handling
       for the caller a	lot.

       The error callback also destroys	the handle explicitly, because we are
       not interested in continuing after any errors. In AnyEvent::Handle
       callbacks you have to call "destroy" explicitly to destroy a handle.
       Outside of those	callbacks you can just forget the object reference and
       it will be automatically	cleaned	up.

       Last but	not least, we set an "on_eof" callback that is called when the
       other side indicates it has stopped writing data, which we will use to
       gracefully shut down the	handle and report the results. This callback
       is only called when the read queue is empty - if	the read queue expects
       some data and the handle	gets an	EOF from the other side	this will be
       an error	- after	all, you did expect more to come.

       If you wanted to	write a	server using AnyEvent::Handle, you would use
       "tcp_accept" and	then create the	AnyEvent::Handle with the "fh"
       argument.

       The write queue

       The next	line sends the actual request:

	  $handle->push_write ("GET $uri HTTP/1.0\015\012\015\012");

       No headers will be sent (this is	fine for simple	requests), so the
       whole request is	just a single line followed by an empty	line to	signal
       the end of the headers to the server.

       The more	interesting question is	why the	method is called "push_write"
       and not just write. The reason is that you can always add some write
       data without blocking, and to do	this, AnyEvent::Handle needs some
       write queue internally -	and "push_write" pushes	some data onto the end
       of that queue, just like	Perl's "push" pushes data onto the end of an
       array.

       The deeper reason is that at some point in the future, there might be
       "unshift_write" as well,	and in any case, we will shortly meet
       "push_read" and "unshift_read", and it's	usually	easiest	to remember if
       all those functions have	some symmetry in their name. So	"push" is used
       as the opposite of "unshift" in AnyEvent::Handle, not as	the opposite
       of "pull" - just	like in	Perl.

       Note that we call "push_write" right after creating the
       AnyEvent::Handle	object,	before it has had time to actually connect to
       the server. This	is fine, pushing the read and write requests will
       queue them in the handle	object until the connection has	been
       established. Alternatively, we could do this "on	demand"	in the
       "on_connect" callback.

       If "push_write" is called with more than	one argument, then you can do
       formatted I/O. For example, this	would JSON-encode your data before
       pushing it to the write queue:

	  $handle->push_write (json => [1, 2, 3]);

       This pretty much	summarises the write queue, there is little else to
       it.

       Reading the response is far more	interesting, because it	involves the
       more powerful and complex read queue:

       The read	queue

       The response consists of	three parts: a single line with	the response
       status, a single	paragraph of headers ended by an empty line, and the
       request body, which is the remaining data on the	connection.

       For the first two, we push two read requests onto the read queue:

	  # now	fetch response status line
	  $handle->push_read (line => sub {
	     my	($handle, $line) = @_;
	     $response = $line;
	  });

	  # then the headers
	  $handle->push_read (line => "\015\012\015\012", sub {
	     my	($handle, $line) = @_;
	     $header = $line;
	  });

       While one can just push a single	callback to parse all the data on the
       queue, formatted	I/O really comes to our	aid here, since	there is a
       ready-made "read	line" read type. The first read	expects	a single line,
       ended by	"\015\012" (the	standard end-of-line marker in internet
       protocols).

       The second "line" is actually a single paragraph	- instead of reading
       it line by line we tell "push_read" that	the end-of-line	marker is
       really "\015\012\015\012", which	is an empty line. The result is	that
       the whole header	paragraph will be treated as a single line and read.
       The word	"line" is interpreted very freely, much	like Perl itself does
       it.

       Note that push read requests are	pushed immediately after creating the
       handle object - since AnyEvent::Handle provides a queue we can push as
       many requests as	we want, and AnyEvent::Handle will handle them in
       order.

       There is, however, no read type for "the	remaining data". For that, we
       install our own "on_read" callback:

	  # and	finally	handle any remaining data as body
	  $handle->on_read (sub	{
	     $body .= $_[0]->rbuf;
	     $_[0]->rbuf = "";
	  });

       This callback is	invoked	every time data	arrives	and the	read queue is
       empty - which in	this example will only be the case when	both response
       and header have been read. The "on_read"	callback could actually	have
       been specified when constructing	the object, but	doing it this way
       preserves logical ordering.

       The read	callback adds the current read buffer to its $body variable
       and, most importantly, empties the buffer by assigning the empty	string
       to it.

       Given these instructions, AnyEvent::Handle will handle incoming data -
       if all goes well, the callback will be invoked with the response	data;
       if not, it will get an error.

       In general, you can implement pipelining	(a semi-advanced feature of
       many protocols) very easily with	AnyEvent::Handle: If you have a
       protocol	with a request/response	structure, your	request
       methods/functions will all look like this (simplified):

	  sub request {

	     # send the	request	to the server
	     $handle->push_write (...);

	     # push some response handlers
	     $handle->push_read	(...);
	  }

       This means you can queue	as many	requests as you	want, and while
       AnyEvent::Handle	goes through its read queue to handle the response
       data, the other side can	work on	the next request - queueing the
       request just appends some data to the write queue and installs a
       handler to be called later.

       You might ask yourself how to handle decisions you can only make	after
       you have	received some data (such as handling a short error response or
       a long and differently-formatted	response). The answer to this problem
       is "unshift_read", which	we will	introduce together with	an example in
       the coming sections.

       Using "http_get"

       Finally,	here is	how you	would use "http_get":

	  http_get "www.google.com", "/", sub {
	     my	($response, $header, $body) = @_;

	     print
		$response, "\n",
		$body;
	  };

       And of course, you can run as many of these requests in parallel	as you
       want (and your memory supports).

       HTTPS

       Now, as promised, let's implement the same thing	for HTTPS, or more
       correctly, let's	change our "http_get" function into a function that
       speaks HTTPS instead.

       HTTPS is	a standard TLS connection (Transport Layer Security is the
       official	name for what most people refer	to as "SSL") that contains
       standard	HTTP protocol exchanges. The only other	difference to HTTP is
       that by default it uses port 443	instead	of port	80.

       To implement these two differences we need two tiny changes, first, in
       the "connect" parameter,	we replace "http" by "https" to	connect	to the
       https port:

		connect	 => [$host => 'https'],

       The other change	deals with TLS,	which is something AnyEvent::Handle
       does for	us if the Net::SSLeay module is	available. To enable TLS with
       AnyEvent::Handle, we pass an additional "tls" parameter to the call to
       "AnyEvent::Handle::new":

		tls	 => "connect",

       Specifying "tls"	enables	TLS, and the argument specifies	whether
       AnyEvent::Handle	is the server side ("accept") or the client side
       ("connect") for the TLS connection, as unlike TCP, there	is a clear
       server/client relationship in TLS.

       That's all.

       Of course, all this should be handled transparently by "http_get" after
       parsing the URL.	If you need this, see the part about exercising	your
       inspiration earlier in this document. You could also use	the
       AnyEvent::HTTP module from CPAN,	which implements all this and works
       around a	lot of quirks for you too.

       The read	queue -	revisited

       HTTP always uses	the same structure in its responses, but many
       protocols require parsing responses differently depending on the
       response	itself.

       For example, in SMTP, you normally get a	single response	line:

	  220 mail.example.net Neverusesendmail	8.8.8 <mailme@example.net>

       But SMTP	also supports multi-line responses:

	  220-mail.example.net Neverusesendmail	8.8.8 <mailme@example.net>
	  220-hey guys
	  220 my response is longer than yours

       To handle this, we need "unshift_read". As the name (we hope) implies,
       "unshift_read" will not append your read	request	to the end of the read
       queue, but will prepend it to the queue instead.

       This is useful in the situation above: Just push	your response-line
       read request when sending the SMTP command, and when handling it, you
       look at the line	to see if more is to come, and "unshift_read" another
       reader callback if required, like this:

	  my $response;	# response lines end up	in here

	  my $read_response; $read_response = sub {
	     my	($handle, $line) = @_;

	     $response .= "$line\n";

	     # check for continuation lines ("-" as 4th	character")
	     if	($line =~ /^...-/) {
		# if yes, then unshift another line read
		$handle->unshift_read (line => $read_response);

	     } else {
		# otherwise we are done

		# free callback
		undef $read_response;

		print "we are don reading: $response\n";
	     }
	  };

	  $handle->push_read (line => $read_response);

       This recipe can be used for all similar parsing problems, for example
       in NNTP,	the response code to some commands indicates that more data
       will be sent:

	  $handle->push_write ("article	42");

	  # read response line
	  $handle->push_read (line => sub {
	     my	($handle, $status) = @_;

	     # article data following?
	     if	($status =~ /^2/) {
		# yes, read article body

		$handle->unshift_read (line => "\012.\015\012",	sub {
		   my ($handle,	$body) = @_;

		   $finish->($status, $body);
		});

	     } else {
		# some error occured, no article data

		$finish->($status);
	     }
	  }

       Your own	read queue handler

       Sometimes your protocol doesn't play nice, and uses lines or chunks of
       data not	formatted in a way handled out of the box by AnyEvent::Handle.
       In this case you	have to	implement your own read	parser.

       To make up a contorted example, imagine you are looking for an even
       number of characters followed by	a colon	(":"). Also imagine that
       AnyEvent::Handle	has no "regex" read type which could be	used, so you'd
       have to do it manually.

       To implement a read handler for this, you would "push_read" (or
       "unshift_read") a single	code reference.

       This code reference will	then be	called each time there is (new)	data
       available in the	read buffer, and is expected to	either successfully
       eat/consume some	of that	data (and return true) or to return false to
       indicate	that it	wants to be called again.

       If the code reference returns true, then	it will	be removed from	the
       read queue (because it has parsed/consumed whatever it was supposed to
       consume), otherwise it stays in the front of it.

       The example above could be coded	like this:

	  $handle->push_read (sub {
	     my	($handle) = @_;

	     # check for even number of	characters + ":"
	     # and remove the data if a	match is found.
	     # if not, return false (actually nothing)

	     $handle->{rbuf} =~	s/^( (?:..)* ) ://x
		or return;

	     # we got some data	in $1, pass it to whoever wants	it
	     $finish->($1);

	     # and return true to indicate we are done
	     1
	  });

Debugging aids
       Now that	you have seen how to use AnyEvent, here's what to use when you
       don't use it correctly, or simply hit a bug somewhere and want to debug
       it:

       Enable strict argument checking during development
	   AnyEvent does not, by default, do any argument checking. This can
	   lead	to strange and unexpected results especially if	you are	just
	   trying to find your way with	AnyEvent.

	   AnyEvent supports a special "strict"	mode - off by default -	which
	   does	very strict argument checking, at the expense of slowing down
	   your	program. During	development, however, this mode	is very	useful
	   because it quickly catches the msot common errors.

	   You can enable this strict mode either by having an environment
	   variable "AE_STRICT"	with a true value in your environment:

	      AE_STRICT=1 perl myprog

	   Or you can write "use AnyEvent::Strict" in your program, which has
	   the same effect (do not do this in production, however).

       Increase	verbosity, configure logging
	   AnyEvent, by	default, only logs critical messages. If something
	   doesn't work, maybe there was a warning about it that you didn't
	   see because it was suppressed.

	   So during development it is recommended to push up the logging
	   level to at least warn level	(5):

	      AE_VERBOSE=5 perl	myprog

	   Other levels	that might be helpful are debug	(8) or even trace (9).

	   AnyEvent's logging is quite versatile - the AnyEvent::Log manpage
	   has all the details.

       Watcher wrapping, tracing, the shell
	   For even more debugging, you	can enable watcher wrapping:

	     AE_DEBUG_WRAP=2 perl myprog

	   This	will have the effect of	wrapping every watcher into a special
	   object that stores a	backtrace of when it was created, stores a
	   backtrace when an exception occurs during watcher execution,	and
	   stores a lot	of other information. If that slows down your program
	   too much, then "AE_DEBUG_WRAP=1" avoids the costly backtraces.

	   Here	is an example of what of information is	stored:

	      59148536 DC::DB:472(Server::run)>io>DC::DB::Server::fh_read
	      type:    io watcher
	      args:    poll r fh GLOB(0x35283f0)
	      created: 2011-09-01 23:13:46.597336 +0200	(1314911626.59734)
	      file:    ./blib/lib/Deliantra/Client/private/DC/DB.pm
	      line:    472
	      subname: DC::DB::Server::run
	      context:
	      tracing: enabled
	      cb:      CODE(0x2d1fb98) (DC::DB::Server::fh_read)
	      invoked: 0 times
	      created
	      (eval 25)	line 6	      AnyEvent::Debug::Wrap::__ANON__('AnyEvent','fh',GLOB(0x35283f0),'poll','r','cb',CODE(0x2d1fb98)=DC::DB::Server::fh_read)
	      DC::DB line 472	      AE::io(GLOB(0x35283f0),'0',CODE(0x2d1fb98)=DC::DB::Server::fh_read)
	      bin/deliantra line 2776 DC::DB::Server::run()
	      bin/deliantra line 2941 main::main()

	   There are many ways to get at this data - see the AnyEvent::Debug
	   and AnyEvent::Log manpages for more details.

	   The most interesting	and interactive	way is to create a debug
	   shell, for example by setting "AE_DEBUG_SHELL":

	     AE_DEBUG_WRAP=2 AE_DEBUG_SHELL=$HOME/myshell ./myprog

	     # while myprog is running:
	     socat readline $HOME/myshell

	   Note	that anybody who can access $HOME/myshell can make your
	   program do anything he or she wants,	so if you are not the only
	   user	on your	machine, better	put it into a secure location ($HOME
	   might not be	secure enough).

	   If you don't	have "socat" (a	shame!)	and care even less about
	   security, you can also use TCP and "telnet":

	     AE_DEBUG_WRAP=2 AE_DEBUG_SHELL=127.0.0.1:1234 ./myprog

	     telnet 127.0.0.1 1234

	   The debug shell can enable and disable tracing of watcher
	   invocations,	can display the	trace output, give you a list of
	   watchers and	lets you investigate watchers in detail.

       This concludes our little tutorial.

Where to go from here?
       This introduction should	have explained the key concepts	of AnyEvent -
       event watchers and condition variables, AnyEvent::Socket	- basic
       networking utilities, and AnyEvent::Handle - a nice wrapper around
       sockets.

       You could either	start coding stuff right away, look at those manual
       pages for the gory details, or roam CPAN	for other AnyEvent modules
       (such as	AnyEvent::IRC or AnyEvent::HTTP) to see	more code examples (or
       simply to use them).

       If you need a protocol that doesn't have	an implementation using
       AnyEvent, remember that you can mix AnyEvent with one other event
       framework, such as POE, so you can always use AnyEvent for your own
       tasks plus modules of one other event framework to fill any gaps.

       And last	not least, you could also look at Coro,	especially
       Coro::AnyEvent, to see how you can turn event-based programming from
       callback	style back to the usual	imperative style (also called
       "inversion of control" -	AnyEvent calls you, but	Coro lets you call
       AnyEvent).

Authors
       Robin Redeker "<elmex at	ta-sa.org>", Marc Lehmann
       <schmorp@schmorp.de>.

perl v5.32.1			  2019-08-08		    AnyEvent::Intro(3)

NAME | Introduction to AnyEvent | What is AnyEvent? | Introduction to Event-Based Programming | Network programming and AnyEvent | Debugging aids | Where to go from here? | Authors

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=AnyEvent::Intro&sektion=3&manpath=FreeBSD+13.0-RELEASE+and+Ports>

home | help