Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
VM::EC2::Staging::ServUser)Contributed Perl DocumenVM::EC2::Staging::Server(3)

       VM::EC2::Staging::Server	- High level interface to EC2-based servers

	use VM::EC2::Staging::Manager;

	# get a	new staging manager
	my $ec2	    = VM::EC2->new;
	my $staging = $ec2->staging_manager();					       );

	# Fetch	a server named 'my_server'. Create it if it does not already exist.
	my $server1 = $staging->get_server(-name	      => 'my_server',
					  -availability_zone  => 'us-east-1a',
					  -architecture	      => 'i386',
					  -instance_type      => 't1.micro');

	# As above, but	force a	new server to be provisioned.
	my $server2 = $staging->provision_server(-name		    => 'my_server',
						 -availability_zone => 'us-east-1a',
						 -architecture	    => 'i386',
						 -instance_type	    => 't1.micro');

	# open up a terminal emulator in a separate window

	# Run a	command	over ssh on the	server.	Standard in and	out will be connected to

	# run a	command	over ssh on the	server,	returning standard output as an	array of lines or a
	# scalar string, similar to backticks (``)
	my @password_lines = $server1->scmd('cat /etc/passwd');

	# run a	command	on the server and read from it using a filehandle
	my $fh	= $server1->scmd_read('ls -R /usr/lib');
	while (<$fh>) {	# do something }

	# run a	command	on the server and write	to it using a filehandle
	my $fh	= $server1->scmd_write('sudo -s	"cat >>/etc/fstab"');
	print $fh "/dev/sdf3 /mnt/demo ext3 0 2\n";
	close $fh;

	# provision and	mount a	5 gig ext3 volume mounted on /opt, returning
	# VM::EC2::Staging::Volume object
	my $opt	= $server1->provision_volume(-mtpt   =>	'/opt',
					     -fstype =>	'ext3',
					     -size   =>	5);

	# copy some data from the local	filesystem onto	the opt	volume
	$server1->rsync("$ENV{HOME}/local_staging_volume/" => $opt);

	# same thing, but using	server path name
	$server1->put("$ENV{HOME}/local_staging_volume/" => '/opt');

	# provision a volume attached to another server, and let the
	# system choose	the filesystem and mount point for us
	my $backups = $server2->provision_volume(-name => 'Backup',
						 -size => 10);

	# copy some data from opt to the new volume using rsync
	$server1->rsync($opt =>	"$backups/opt");

	# Do a block-level copy	between	disks -	warning, the filesystem	must be	unmounted
	# before you attempt this.
	$server1->dd($opt => $backups);

       VM::EC2::Staging::Server	objects	are an extension of VM::EC2::Instance
       to allow	for higher-level access, including easy	management of ssh
       keys, remote copying of data from one server to another,	and executing
       of remote commands on the server	from within Perl. See
       VM::EC2::Staging::Manager for an	overview of staging servers and

       Note that proper	functioning of this module is heavily dependent	on
       running on a host system	that has access	to ssh,	rsync and terminal
       emulator	command-line tools. It will most likely	fail when run on a
       Windows host.

Staging	Server Creation
       Staging servers are usually created via a staging manager's
       get_server() or provision_server() methods. See

       There is	also a new() class method that is intended to be used
       internally in most cases. It is called like this:

   $server = VM::EC2::Staging::Server->new(%args)
       With the	arguments:

	-keyfile    path to the	ssh public/private keyfile for this instance
	-username   username for remote	login on this instance
	-instance   VM::EC2::Instance to attach	this server to
	-manager    VM::EC2::Staging::Manager in same zone as the instance

       Note that you will have to launch a staging manager, start an instance,
       and appropriate provision the SSH credentials for that instance before
       invoking	new() directly.

Information about the Server
       VM::EC2::Staging::Server	objects	have all the methods of
       VM::EC2::Instance, such as dnsName(), but add several new methods. The
       new methods involving getting basic information about the server	are
       listed in this section.

   $name = $server->name
       This method returns the server's	symbolic name, if any.

       Servers can optionally be assigned a symbolic name at the time they are
       created by the manager's	get_server() or	provision_server() methods.
       The name	persists as long as the	underlying instance exists (including
       in stopped state	for EBS-backed instances). Calling
       $manager->get_server() with this	name returns the server	object.

   $ec2	= $server->ec2
       Return the VM::EC2 object associated with the server.

   $ec2	= $server->endpoint
       Return the endpoint URL associated with this server.

   $instance = $server->instance
       Return the VM::EC2::Instance associated with this server.

   $file = $server->keyfile
       Return the full path to the SSH PEM keyfile used	to log into this

   $user = $server->username
       Return the name of the user (e.g. 'ubuntu') used	to ssh into this

   $manager = $server->manager
       Returns the VM::EC2::Staging::Manager that manages this server.

Lifecycle Methods
       The methods in this section manage the lifecycle	of a server.

   $flag = $server->ping
       The ping() method returns true if the server is running and is
       reachable via ssh. It is	different from checking	that the underlying
       instance	is "running" via a call	to current_status, because it also
       checks the usability of the ssh daemon, the provided ssh	key and
       username, firewall rules, and the network connectivity.

       The result of ping is cached so that subsequent invocations return

   $result = $server->start
       Attempt to start	a stopped server. The method will wait until a ping()
       is successful, or until a timeout of 120	seconds. The result code will
       be true if the server was successfully started and is reachable.

       If you wish to start a set of servers without waiting for each one
       individually, then you may call the underling instance's	start()


       You may then wish to call the staging manager's wait_for_instances()
       method to wait on all of	the servers to start:


       Also check out $manager->start_all_servers().

   $result = $server->stop
       Attempt to stop a running server. The method will wait until the	server
       has entered the "stopped" state before returning. It will return	a true
       result if the underlying	instance stopped successfully.

       If you wish to stop a set of servers without waiting for	each one
       individually, then you may call the underling instance's	start()


       You may then wish to call the staging manager's wait_for_instances()
       method to wait on all of	the servers to start:

	$status	= $manager->wait_for_servers(@servers);

       Also check out $manager->stop_all_servers().

   $result = $server->terminate
       Terminate a server and unregister it from the manager. This method will
       stop and	wait until the server is terminated.

       If you wish to stop a set of servers without waiting for	each one
       individually, then you may call the underling instance's	start()


Remote Shell Methods
       The methods in this section allow you to	run remote commands on the
       staging server and interrogate the results. Since the staging manager
       handles the creation of SSH keys	internally, you	do not need to worry
       about finding the right public/private keypair.

   $result = $server->ssh(@command)
       The ssh() method	invokes	a command on the remote	server.	You may
       provide the command line	as a single string, or broken up by argument:

	 $server->ssh('ls -lR /var/log');

       The output of the command will appear on	STDOUT and STDERR of the perl
       process.	Input, if needed, will be read from STDIN. If no command is
       provided, then an interactive ssh session will be started on the	remote
       server and the script will wait until you have logged out.

       If the remote command was successful, the method	result will be true.

   $output = $server->scmd(@command)
       This is similar to ssh(), except	that the standard output of the	remote
       command will be captured	and returned as	the function result, similar
       to the way backticks work in perl:

	my $output = $server->scmd('date');
	print "The localtime for the server is $output";

   $fh = $server->scmd_write(@command)
       This method executes @command on	the remote server, and returns a
       filehandle that is attached to the standard input of the	command. Here
       is a slightly dangerous example that appends a line to /etc/passwd:

	my $fh = $server->scmd_write('sudo -s "cat >>/etc/passwd"');
	print $fh "whoopsie:x:119:130::/nonexistent:/bin/false\n";
	close $fh;

   $fh = $server->scmd_read(@command)
       This method executes @command on	the remote server, and returns a
       filehandle that is attached to the standard output of the command. Here
       is an example of	reading	syslog:

	my $fh = $server->scmd_read('sudo cat /var/log/syslog');
	while (<$fh>) {
	   next	unless /kernel/;
	   print $_;
	close $fh;

       This method works in an X Windowing environment by launching a new
       terminal	window and running an interactive ssh session on the server
       host. The terminal window is executed in	a fork()ed session, so that
       the rest	of the script continues	running.  If X Windows is not running,
       then the	method behaves the same	as calling ssh() with no arguments.

       The terminal emulator to	run is determined by calling the method

Volume Management Methods
       The methods in this section allow you to	create and manage volumes
       attached	to the server. These supplement	the EC2	facilities for
       creating	and attaching EBS volumes with the ability to format the
       volumes with a variety of filesystems, and mount	them at	a desired

   $volume = $server->provision_volume(%args)
       Provision and mount a new volume. If successful,	the volume is returned
       as a VM::EC2::Staging::Volume object.

       Arguments (default):

	-name	      Symbolic name for	the desired volume (autogenerated)
	-fstype	      Filesystem type for desired volume (ext4)
	-size	      Size for the desired volume in GB	(1)
	-mtpt	      Mountpoint for this volume (/mnt/Staging/$name)
	-mount	      Alias for	-mtpt
	-volume_id    ID of existing volume to attach &	mount (none)
	-snapshot_id  ID of existing snapshot to use to	create this volume (none)
	-reuse	      Reuse an existing	managed	volume of same name (false)
	-label	      Disk label to assign during formatting ($name)
	-uuid	      UUID to assign during formatting (none)

       None of the arguments are required, and reasonable defaults will	be
       chosen if they are missing.

       The -name argument specifies the	symbolic name to be assigned to	the
       newly-created staging volume. The name allows the staging manager to
       retrieve	this volume at a later date if it is detached from the server
       and returned to the available pool. If no name is provided, then	an
       arbitrary one will be autogenerated.

       The -fstype argument specifies the filesystem to	be generated on	the
       volume, ext4 by default.	The following filesystems are currently
       supported: ext2,	ext3, ext4, xfs, reiserfs, jfs,	ntfs, nfs, vfat,
       msdos. In addition, you can specify a filesystem	of "raw", which	means
       to provision and	attach the volume to the server, but not to format it.
       This can	be used	to set up LVM and RAID devices.	Note that if the
       server does not currently have the package needed to manage the desired
       filesystem, it will use "apt-get" to install it.

       The -mtpt and -mount arguments (they are	equivalent) specify the	mount
       point for the volume on the server filesystem. The default is
       "/mnt/Staging/$name", where $name is the	symbolic name provided by
       -name or	autogenerated. No checking is done on the sensibility of the
       mount point, so try to avoid mounting disks over	essential parts	of the

       -volume_id and -snapshot_id instruct the	method to construct the
       staging volume from an existing EBS volume or snapshot. -volume_id is
       an EBS volume ID. If provided, the volume must be located in the
       server's	availability zone and be in the	"available" state.
       -snapshot_id is an EBS snapshot ID in the server's region. In no	case
       will provision_volume() attempt to reformat the resulting volume, even
       if the -fstype argument is provided. However, in	the case of a volume
       created from a snapshot,	you may	specify	a -size	argument larger	than
       the snapshot and	the filesystem will be dynamically resized to fill the
       requested space.	This currently only works with ext2, ext3 and ext4
       volumes,	and cannot be used to make filesystems smaller.

       If the -reuse argument is true, and a symbolic name is provided in
       -name, then the method will look	for an available staging volume	of the
       same name and mount this	at the specified location. If no suitable
       staging volume is found,	then the method	will look for a	snapshot
       created earlier from a staging volume of	the same name. If neither a
       suitable	volume nor a snapshot is available, then a new volume is
       provisioned. This is intended to	support	the following use case of
       synchronizing a filesystem somewhere to an EBS snapshot:

	my $server = $staging_manager->get_server('my_server');
	my $volume = $server->provision_volume(-name=>'backup_1',
					       -reuse  => 1,
					       -fstype => 'ext3',
					       -size   => 10);

       The -label and -uuid arguments are used to set the volume label and
       UUID during formatting of new filesystems. The default behavior is to
       create no label and to allow the	server to choose an arbitrary UUID.

   $volume = $server->add_volume(%args)
       This is the same	as provision_volume().

   @volumes = $server->volumes()
       Return a	list of	all the	staging	volumes	attached to this server.
       Unmanaged volumes, such as the root volume, are not included in the

       Unmount the volume $volume. The volume will remain attached to the
       server. This method will	die with a fatal error if the operation	fails.

       See VM::EC2::Staging::Volume->detach() for the recommended way to
       unmount and detach the volume.

       Unmount and detach the volume from the server, waiting until EC2
       reports that the	detachment completed. A	fatal error will occur if the
       operation fails.

   $server->mount_volume($volume [,$mountpt])
       Mount the volume	$volume	using the mount	information recorded inside
       the VM::EC2::Staging::Volume object (returned by	its mtpt() and mtdev()
       methods). If the	volume has not previously been mounted on this server,
       then it will be attached	to the server and a new	mountpoint will	be
       allocated automatically.	You can	change the mount point by specifying
       it explicitly in	the second argument.

       Here is the recommended way to detach a staging volume from one server
       and attach it to	another:


       This method will	die in case of error.

       This is similar to mount_volume(), except that it will fail with	a
       fatal error if the volume was not previously mounted on this server.
       This is to be used when temporarily unmounting and remounting a volume
       on the same server:

	# do some work on the volume

       Unmount,	detach,	and then delete	the indicated volume entirely.

   $snap = $server->create_snapshot($volume,$description)
       Unmount the volume, snapshot it using the provided description, and
       then remount the	volume.	If successful, returns the snapshot.

       The snapshot is tagged with the identifying information needed to
       associate the snapshot with the staging volume. This information	then
       used when creating new volumes from the snapshot	with

Data Copying Methods
       The methods in this section are used to copy data from one staging
       server to another, and to copy data from	a local	file system to a
       staging server.

   $result = $server->rsync($src1,$src2,$src3...,$dest)
       This method is a	passthrough to VM::EC2::Staging::Manager->rsync(), and
       provides	efficient file-level synchronization (rsync) file-level
       copying between one or more source locations and	a destination location
       via an ssh tunnel. Copying among	arbitrary combinations of local	and
       remote filesystems is supported,	with the caveat	that the remote
       filesystems must	be contained on	volumes	and servers managed by this
       module (see below for a workaround).

       You may provide two or more directory paths. The	last path will be
       treated as the copy destination,	and the	source paths will be treated
       as copy sources.	All copying is performed using the -avz	options, which
       activates recursive directory copying in	which ownership, modification
       times and permissions are preserved, and	compresses the data to reduce
       network usage.

       Source paths can	be formatted in	one of several ways:

	     Copy the contents of the directory	/absolute/path located on the
	     local machine to the destination. This will create	a
	     subdirectory named	"path" on the destination disk.	Add a slash
	     to	the end	of the path (i.e. "/absolute/path/") in	order to
	     avoid creating this subdirectory on the destination disk.

	     Relative paths work the way you expect, and depend	on the current
	     working directory.	The terminating	slash rule applies.

	    Pass a staging server object and absolute path to copy the contents
	    of this path to the	destination disk. Because of string interpolation
	    you	can include server objects in quotes: "$my_server:/opt"

	    This form will copy	data from paths	relative to the	remote user's home
	    directory on the staging server. Typically not very	useful,	but supported.

	     Pass a VM::EC2::Staging::Volume to	copy the contents of the
	     volume to the destination disk starting at	the root of the
	     volume. Note that you do *not* need to have any knowledge of the
	     mount point for this volume in order to copy its contents.

	     Copy a subdirectory of a staging volume to	the destination	disk.
	     The root of the volume is its top level, regardless of where it
	     is	mounted	on the staging server.	Because	of string
	     interpolation magic, you can enclose staging volume object	names
	     in	quotes in order	to construct the path, as in
	     "$picture_volume:/family/vacations/". As in local paths, a
	     terminating slash indicates that the contents of the last
	     directory in the path are to be copied without creating the
	     enclosing directory on the	desetination. Note that	you do *not*
	     need to have any knowledge	of the mount point for this volume in
	     order to copy its contents.

	    These are alternatives to the previous syntax, and all have	the
	    same effect	as $staging_volume:relative/path. There	is no

       The same	syntax is supported for	destination paths, except that it
       makes no	difference whether a path has a	trailing slash or not.

       Note that neither the source nor	destination paths need to reside on
       this server.

       See VM::EC2::Staging::Manager->rsync() for examples and more details.

       This method is a	passthrough to VM::EC2::Staging::Manager->dd(),	and
       performs	block-level copying of the contents of $source_vol to
       $dest_vol by using dd over an SSH tunnel, where both source and
       destination volumes are VM::EC2::Staging::Volume	objects. The volumes
       must be attached	to a server but	not mounted. Everything	in the volume,
       including its partition table, is copied, allowing you to make an exact
       image of	a disk.

       The volumes do not actually need	to reside on this server, but can be
       attached	to any staging server in the zone.

       Use rsync to copy the indicated source directories into the destination
       path indicated by $dest.	The destination	is either a path on the	server
       machine,	or a staging volume object mounted on the server (string
       interpolation is	accepted). The sources can be local paths on the
       machine the perl	script is running on, or any of	the formats described
       for rsync().


	$server1->put("$ENV{HOME}/my_pictures"	   => '/var/media');
	$server1->put("$ENV{HOME}/my_pictures","$ENV{HOME}/my_audio" =>	'/var/media');
	$server1->put("$ENV{HOME}/my_pictures"	   => "$backup_volume/home_media");
	$server1->put("" => "$backup_volume/home_media");

       Use rsync to copy the indicated source directories into the destination
       path indicated by $dest.	The source directories are either paths	on the
       server, or staging volume(s) mounted on the server (string
       interpolation to	indicate subdirectories	on the staging volume also
       works). The destination can be any of the path formats described	for
       rsync(),	including unmanaged hosts that accept ssh login.


	$server1->get('/var/media' =>"$ENV{HOME}/my_pictures");
	$server1->get('/var/media','/usr/bin' => "$ENV{HOME}/test");
	$server1->get("$backup_volume/home_media" => "$ENV{HOME}/my_pictures");
	$server1->get("$backup_volume/home_media" => "");

Internal Methods
       This section documents internal methods.	They are not intended for use
       by end-user scripts but may be useful to	know about during subclassing.
       There are also additional undocumented methods that begin with a	"_"
       character which you can explore in the source code.

   $description	= $server->volume_description($vol)
       This method is called to	get the	value of the Name tag assigned to new
       staging volume objects. The current value is "Staging volume for	$name
       created by VM::EC2::Staging::Server."

       You will	see these names	associated with	EBS volumes in the AWS

   ($ebs_device,$local_device) = $server->unused_block_device([$major_start])
       This method returns an unused block device path.	It is invoked when
       provisioning and	mounting new volumes. The behavior is to take the
       following search	path:


       You can modify the search path slightly by providing a single character
       major start. For	example, to leave all the sdf's	free and to start the
       search at /dev/sdg:

	($ebs_device,$local_device) = $server->unused_block_device('g');

       The result is a two element list	consisting of the unused device	name
       from the	perspective of EC2 and the server respectively.	The issue here
       is that on some recent Linux kernels, the EC2 device /dev/sdf1 is known
       to the server as	/dev/xvdf1. This module	understands that complication
       and uses	the EC2	block device name when managing	EBS volumes, and the
       kernel block device name	when communicating with	the server.

   $flag = $server->has_key($keyname)
       Returns true if the server has a	copy of	the private key	corresponding
       to $keyname. This is used by the	rsync()	method to enable server	to
       server data transfers.

   $flag = $server->accepts_key($keyname)
       Returns true if the server has a	copy of	the public key part of
       $keyname	in its .ssh/authorized_keys file. This is used by the rsync()
       method to enable	server to server data transfers.

   $up = $server->is_up([$new_value])
       Get/set the internal is_up() flag, which	indicates whether the server
       is up and running. This is used to cache	the results of the ping()

   $path = $server->default_mtpt($volume)
       Given a staging volume, return its default mount	point on the server
       ('/mnt/Staging/'.$volume->name).	Can also pass a	string corresponding
       to the volume's name.

       Log a message to	standard output, respecting the	staging	manager's
       verbosity() setting.

       For reasons having to do	with the order in which	objects	are created,
       VM::EC2::Staging::Server	is a wrapper around VM::EC2::Instance rather
       than a subclass of it. To access	the VM::EC2::Instance object, you call
       the server object's instance() method. In practice this means that to
       invoke the underlying instance's	method for, say, start() you will need
       to do this:


       rather than this:


       You may subclass	VM::EC2::Staging::Server in the	usual way.

       VM::EC2 VM::EC2::Instance VM::EC2::Volume VM::EC2::Snapshot

       Lincoln Stein <>.

       Copyright (c) 2011 Ontario Institute for	Cancer Research

       This package and	its accompanying libraries is free software; you can
       redistribute it and/or modify it	under the terms	of the GPL (either
       version 1, or at	your option, any later version)	or the Artistic
       License 2.0.  Refer to LICENSE for the full license text. In addition,
       please see DISCLAIMER.txt for disclaimers of warranty.

perl v5.32.0			  2020-08-23	   VM::EC2::Staging::Server(3)

NAME | SYNOPSIS | DESCRIPTION | Staging Server Creation | Information about the Server | Lifecycle Methods | Remote Shell Methods | Volume Management Methods | Data Copying Methods | Internal Methods | Subclassing | SEE ALSO | AUTHOR

Want to link to this manual page? Use this URL:

home | help