Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
Search::Elasticsearch:UsereContributedrPerllasticsearch::Client::5_0::Bulk(3)

NAME
       Search::Elasticsearch::Client::5_0::Bulk	- A helper module for the Bulk
       API

VERSION
       version 5.02

SYNOPSIS
	   use Search::Elasticsearch;

	   my $es   = Search::Elasticsearch->new;
	   my $bulk = $es->bulk_helper(
	       index   => 'my_index',
	       type    => 'my_type'
	   );

	   # Index docs:
	   $bulk->index({ id =>	1, source => { foo => 'bar' }});
	   $bulk->add_action( index => { id => 1, source => { foo=> 'bar' }});

	   # Create docs:
	   $bulk->create({ id => 1, source => {	foo => 'bar' }});
	   $bulk->add_action( create =>	{ id =>	1, source => { foo=> 'bar' }});
	   $bulk->create_docs({	foo => 'bar' })

	   # Delete docs:
	   $bulk->delete({ id => 1});
	   $bulk->add_action( delete =>	{ id =>	1 });
	   $bulk->delete_ids(1,2,3)

	   # Update docs:
	   $bulk->update({ id => 1, script => '...' });
	   $bulk->add_action( update =>	{ id =>	1, script => '...' });

	   # Manual flush
	   $bulk->flush;

DESCRIPTION
       This module provides a wrapper for the "bulk()" in
       Search::Elasticsearch::Client::5_0::Direct method which makes it	easier
       to run multiple create, index, update or	delete actions in a single
       request.

       The Search::Elasticsearch::Client::5_0::Bulk module acts	as a queue,
       buffering up actions until it reaches a maximum count of	actions, or a
       maximum size of JSON request body, at which point it issues a "bulk()"
       request.

       Once you	have finished adding actions, call "flush()" to	force the
       final "bulk()" request on the items left	in the queue.

       This class does Search::Elasticsearch::Client::5_0::Role::Bulk and
       Search::Elasticsearch::Role::Is_Sync.

CREATING A NEW INSTANCE
   "new()"
	   my $bulk = $es->bulk_helper(

	       index	   => 'default_index',	   # optional
	       type	   => 'default_type',	   # optional
	       %other_bulk_params		   # optional

	       max_count   => 1_000,		   # optional
	       max_size	   => 1_000_000,	   # optional
	       max_time	   => 5,		   # optional

	       verbose	   => 0	| 1,		   # optional

	       on_success  => sub {...},	   # optional
	       on_error	   => sub {...},	   # optional
	       on_conflict => sub {...},	   # optional

	   );

       The "new()" method returns a new	$bulk object.  You must	pass your
       Search::Elasticsearch client as the "es"	argument.

       The "index" and "type" parameters provide default values	for "index"
       and "type", which can be	overridden in each action.  You	can also pass
       any other values	which are accepted by the bulk() method.

       See "flush()" for more information about	the other parameters.

FLUSHING THE BUFFER
   "flush()"
	   $result = $bulk->flush;

       The "flush()" method sends all buffered actions to Elasticsearch	using
       a bulk()	request.

   Auto-flushing
       An automatic "flush()" is triggered whenever the	"max_count",
       "max_size", or "max_time" threshold is breached.	 This causes all
       actions in the buffer to	be sent	to Elasticsearch.

       o   "max_count"

	   The maximum number of actions to allow before triggering a
	   "flush()".  This can	be disabled by setting "max_count" to 0.
	   Defaults to "1,000".

       o   "max_size"

	   The maximum size of JSON request body to allow before triggering a
	   "flush()".  This can	be disabled by setting "max_size" to 0.
	   Defaults to "1_000,000" bytes.

       o   "max_time"

	   The maximum number of seconds to wait before	triggering a flush.
	   Defaults to 0 seconds, which	means that it is disabled.  Note: This
	   timeout is only triggered when new items are	added to the queue,
	   not in the background.

   Errors when flushing
       There are two types of error which can be thrown	when "flush()" is
       called, either manually or automatically.

       o   Temporary Elasticsearch errors

	   A "Cxn" error like a	"NoNodes" error	which indicates	that your
	   cluster is down.  These errors do not clear the buffer, as they can
	   be retried later on.

       o   Action errors

	   Individual actions may fail.	For instance, a	"create" action	will
	   fail	if a document with the same "index", "type" and	"id" already
	   exists.  These action errors	are reported via callbacks.

   Using callbacks
       By default, any Action errors (see above) cause warnings	to be written
       to "STDERR".  However, you can use the "on_error", "on_conflict"	and
       "on_success" callbacks for more fine-grained control.

       All callbacks receive the following arguments:

       $action
	   The name of the action, ie "index", "create", "update" or "delete".

       $response
	   The response	that Elasticsearch returned for	this action.

       $i  The index of	the action, ie the first action	in the flush request
	   will	have $i	set to 0, the second will have $i set to 1 etc.

       "on_success"

	   my $bulk = $es->bulk_helper(
	       on_success  => sub {
		   my ($action,$response,$i) = @_;
		   # do	something
	       },
	   );

       The "on_success"	callback is called for every action that has a
       successful response.

       "on_conflict"

	   my $bulk = $es->bulk_helper(
	       on_conflict  => sub {
		   my ($action,$response,$i,$version) =	@_;
		   # do	something
	       },
	   );

       The "on_conflict" callback is called for	actions	that have triggered a
       "Conflict" error, eg trying to "create" a document which	already
       exists.	The $version argument will contain the version number of the
       document	currently stored in Elasticsearch (if found).

       "on_error"

	   my $bulk = $es->bulk_helper(
	       on_error	 => sub	{
		   my ($action,$response,$i) = @_;
		   # do	something
	       },
	   );

       The "on_error" callback is called for any error (unless the
       "on_conflict") callback has already been	called).

   Disabling callbacks and autoflush
       If you want to be in control of flushing, and you just want to receive
       the raw response	that Elasticsearch sends instead of using callbacks,
       then you	can do so as follows:

	   my $bulk = $es->bulk_helper(
	       max_count   => 0,
	       max_size	   => 0,
	       on_error	   => undef
	   );

	   $bulk->add_actions(....);
	   $response = $bulk->flush;

CREATE,	INDEX, UPDATE, DELETE
   "add_action()"
	   $bulk->add_action(
	       create => { ...params...	},
	       index  => { ...params...	},
	       update => { ...params...	},
	       delete => { ...params...	}
	   );

       The "add_action()" method allows	you to add multiple "create", "index",
       "update"	and "delete" actions to	the queue. The first value is the
       action type, and	the second value is the	parameters that	describe that
       action.	See the	individual helper methods below	for details.

       Note: Parameters	like "index" or	"type" can be specified	as "index" or
       as "_index", so the following two lines are equivalent:

	   index => { index  =>	'index', type  => 'type', id  => 1, source => {...}},
	   index => { _index =>	'index', _type => 'type', _id => 1, source => {...}},

       Note: The "index" and "type" parameters can be specified	in the params
       for any action, but if not specified, will default to the "index" and
       "type" values specified in "new()".  These are required parameters:
       they must be specified either in	"new()"	or in every action.

   "create()"
	   $bulk->create(
	       { index => 'custom_index',	  source => { doc body }},
	       { type  => 'custom_type', id => 1, source => { doc body }},
	       ...
	   );

       The "create()" helper method allows you to add multiple "create"
       actions.	 It accepts the	same parameters	as "create()" in
       Search::Elasticsearch::Client::5_0::Direct except that the document
       body should be passed as	the "source" or	"_source" parameter, instead
       of as "body".

   "create_docs()"
	   $bulk->create_docs(
	       { doc body },
	       { doc body },
	       ...
	   );

       The "create_docs()" helper is a shorter form of "create()" which	can be
       used when you are using the default "index" and "type" as set in
       "new()" and you are not specifying a custom "id"	per document.  In this
       case, you can just pass the individual document bodies.

   "index()"
	   $bulk->index(
	       { index => 'custom_index',	  source => { doc body }},
	       { type  => 'custom_type', id => 1, source => { doc body }},
	       ...
	   );

       The "index()" helper method allows you to add multiple "index" actions.
       It accepts the same parameters as "index()" in
       Search::Elasticsearch::Client::5_0::Direct except that the document
       body should be passed as	the "source" or	"_source" parameter, instead
       of as "body".

   "delete()"
	   $bulk->delete(
	       { index => 'custom_index', id =>	1},
	       { type  => 'custom_type',  id =>	2},
	       ...
	   );

       The "delete()" helper method allows you to add multiple "delete"
       actions.	 It accepts the	same parameters	as "delete()" in
       Search::Elasticsearch::Client::5_0::Direct.

   "delete_ids()"
	   $bulk->delete_ids(1,2,3...)

       The "delete_ids()" helper method	can be used when all of	the documents
       you want	to delete have the default "index" and "type" as set in
       "new()".	 In this case, all you have to do is to	pass in	a list of IDs.

   "update()"
	   $bulk->update(
	       { id	       => 1,
		 doc	       => { partial doc	},
		 doc_as_upsert => 1
	       },
	       { id	       => 2,
		 script	       => { script }
		 upsert	       => { upsert doc }
	       },
	       ...
	   );

       The "update()" helper method allows you to add multiple "update"
       actions.	 It accepts the	same parameters	as "update()" in
       Search::Elasticsearch::Client::5_0::Direct.  An update can either use a
       partial doc which gets merged with an existing doc (example 1 above),
       or can use a "script" to	update an existing doc (example	2 above). More
       information on "script" can be found here: "update()" in
       Search::Elasticsearch::Client::5_0::Direct.

AUTHOR
       Clinton Gormley <drtech@cpan.org>

COPYRIGHT AND LICENSE
       This software is	Copyright (c) 2017 by Elasticsearch BV.

       This is free software, licensed under:

	 The Apache License, Version 2.0, January 2004

perl v5.24.1			  2Search::Elasticsearch::Client::5_0::Bulk(3)

NAME | VERSION | SYNOPSIS | DESCRIPTION | CREATING A NEW INSTANCE | FLUSHING THE BUFFER | CREATE, INDEX, UPDATE, DELETE | AUTHOR | COPYRIGHT AND LICENSE

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=Search::Elasticsearch::Client::5_0::Bulk&sektion=3&manpath=FreeBSD+12.1-RELEASE+and+Ports>

home | help