Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
Net::Google::SafeBrowsUser(Contributed Perl DocumNet::Google::SafeBrowsing2(3)

NAME
       Net::Google::SafeBrowsing2 - Perl extension for the Google Safe
       Browsing	v2 API.	(Google	Safe Browsing v1 has been deprecated by
       Google.)

SYNOPSIS
	 use Net::Google::SafeBrowsing2;
	 use Net::Google::SafeBrowsing2::Sqlite;

	 my $storage = Net::Google::SafeBrowsing2::Sqlite->new(file => 'google-v2.db');
	 my $gsb = Net::Google::SafeBrowsing2->new(
	       key     => "my key",
	       storage => $storage,
	 );

	 $gsb->update();
	 my $match = $gsb->lookup(url => 'http://www.gumblar.cn/');

	 if ($match eq MALWARE)	{
	       print "http://www.gumblar.cn/ is	flagged	as a dangerous site\n";
	 }

	 $storage->close();

DESCRIPTION
       Net::Google::SafeBrowsing2 implements the Google	Safe Browsing v2 API.

       The library passes most of the unit tests listed	in the API
       documentation. See the documentation
       (<http://code.google.com/apis/safebrowsing/developers_guide_v2.html>)
       for more	details	about the failed tests.

       The Google Safe Browsing	database must be stored	and managed locally.
       Net::Google::SafeBrowsing2::Sqlite uses Sqlite as the storage back-end,
       Net::Google::SafeBrowsing2::MySQL uses MySQL. Other storage mechanisms
       (databases, memory, etc.) can be	added and used transparently with this
       module.

       You may want to look at "Google Safe Browsing v2: Implementation	Notes"
       (<http://www.zscaler.com/research/Google%20Safe%20Browsing%20v2%20API.pdf>),
       a collection of notes and real-world numbers about the API. This	is
       intended	for people who want to learn more about	the API, whether as a
       user or to make their own implementation.

       The source code is available on github at
       <https://github.com/juliensobrier/Net-Google-SafeBrowsing2>.

       If you do not need to inspect more than 10,000 URLs a day, you can use
       Net::Google::SafeBrowsing2::Lookup with the Google Safe Browsing	v2
       Lookup API which	does not require to store and maintain a local
       database.

       IMPORTANT: If you start with an empty database, you will	need to
       perform several updates to retrieve all the Google Safe Browsing
       information. This may require up	to 24 hours. This is a limitation of
       the Google API, not of this module. See "Google Safe Browsing v2:
       Implementation Notes" at
       <http://www.zscaler.com/research/Google%20Safe%20Browsing%20v2%20API.pdf>.

CONSTANTS
       Several	constants are exported by this module:

       DATABASE_RESET
	   Google requested to reset (empty) the local database.

       MAC_ERROR
	   The replies from Google could not be	validated with the MAC keys.

       MAC_KEY_ERROR
	   The request for the MAC keys	failed.

       INTERNAL_ERROR
	   An internal error occurred.

       SERVER_ERROR
	   The server sent an error back to the	client.

       NO_UPDATE
	   No update was performed, probably because it	is too early to	make a
	   new request to Google Safe Browsing.

       NO_DATA
	   No data was sent back by Google to the client, probably because the
	   database is up to date.

       SUCCESSFUL
	   The operation was successful.

       MALWARE
	   Name	of the Malware list in Google Safe Browsing (shortcut to
	   'goog-malware-shavar')

       PHISHING
	   Name	of the Phishing	list in	Google Safe Browsing (shortcut to
	   'googpub-phish-shavar')

CONSTRUCTOR
   new()
       Create a	Net::Google::SafeBrowsing2 object

	 my $gsb = Net::Google::SafeBrowsing2->new(
	       key     => "my key",
	       storage => Net::Google::SafeBrowsing2::Sqlite->new(file => 'google-v2.db'),
	       debug   => 0,
	       mac	       => 0,
	       list    => MALWARE,
	 );

       Arguments

       key Required. Your Google Safe browsing API key

       storage
	   Required. Object which handle the storage for the Google Safe
	   Browsing database. See Net::Google::SafeBrowsing2::Storage for more
	   details.

       list
	   Optional. The Google	Safe Browsing list to handle. By default,
	   handles both	MALWARE	and PHISHING.

       mac Optional. Set to 1 to enable	Message	Authentication Code (MAC). 0
	   (disabled) by default.

       debug
	   Optional. Set to 1 to enable	debugging. 0 (disabled)	by default.

	   The debug output maybe quite	large and can slow down	significantly
	   the update and lookup functions.

       errors
	   Optional. Set to 1 to show errors to	STDOUT.	0 (disabled by
	   default).

       perf
	   Optional. Set to 1 to show performance information.

       version
	   Optional. Google Safe Browsing version. 2.2 by default

PUBLIC FUNCTIONS
   update()
       Perform a database update.

	 $gsb->update();

       Return the status of the	update (see the	list of	constants above):
       INTERNAL_ERROR, SERVER_ERROR, NO_UPDATE,	NO_DATA	or SUCCESSFUL

       This function can handle	two lists at the same time. If one of the list
       should not be updated, it will automatically skip it and	update the
       other one. It is	faster to update two lists at once rather than doing
       them one	by one.

       NOTE: If	you start with an empty	database, you will need	to perform
       several updates to retrieve all the Google Safe Browsing	information.
       This may	require	up to 24 hours.	This is	a limitation of	the Google
       API, not	of this	module.	See "Google Safe Browsing v2: Implementation
       Notes" at
       <http://www.zscaler.com/research/Google%20Safe%20Browsing%20v2%20API.pdf>.

       Arguments

       list
	   Optional. Update a specific list. Use the list(s) from new()	by
	   default.

       mac Optional. Set to 1 to enable	Message	Authentication Code (MAC). Use
	   the value from new()	by default.

       force
	   Optional. Force the update (1). Disabled by default (0).

	   Be careful if you set this option to	1 as too frequent updates
	   might result	in the blacklisting of your API	key.

   import_chunks()
       Import add and sub chunks from a	file.

	 my $result = $gsb->import_chunks(list => MALWARE, file	=> 'malware.dat');

       Return the status of the	import:	INTERNAL_ERROR or SUCCESSFUL.

       This function should be used to initialize an empty back-end storage.

       Arguments

       list
	   Required. Google list to use.

       file
	   Required. File that contains	the list of chunks. This file can be
	   created with	the "export" function inherited	from
	   "Net::Google::SafeBrowsing2::DBI".

   lookup()
       Lookup a	URL against the	Google Safe Browsing database.

	 my $match = $gsb->lookup(url => 'http://www.gumblar.cn');

       Returns the name	of the list if there is	any match, returns an empty
       string otherwise.

       Arguments

       list
	   Optional. Lookup against a specific list. Use the list(s) from
	   new() by default.

       url Required. URL to lookup.

   get_lists()
       Returns the name	of all the Google Safe Browsing	lists

	 my $@lists = $gsb->get_lists();

       NOTE: this function is useless in practice because Google includes some
       lists which cannot be used by the Google	Safe Browsing API, like	lists
       used by the Google toolbar.

   last_error()
       Get/Set the last	error message.

	 print "Last error: ", $gsb->last_error(), "\n";
	 $gsb->last_error(''); # Reset last error

       NOTE: the last error message might not come from	the last call. Returns
       an empty	string if no errors.

PRIVATE	FUNCTIONS
       These functions are not intended	to be used externally.

   lookup_suffix()
       Lookup a	host prefix.

   lookup_suffix()
       Lookup a	host prefix in the local database only.

   local_lookup()
       Lookup a	URL against the	local Google Safe Browsing database URL. This
       should be used for debugging purpose only. See the lookup for normal
       use.

	 my $match = $gsb->local_lookup(url => 'http://www.gumblar.cn');

       Returns the name	of the list if there is	any match, returns an empty
       string otherwise.

       Arguments

       list
	   Optional. Lookup against a specific list. Use the list(s) from
	   new() by default.

       url Required. URL to lookup.

   request_key()
       Request the Message Authentication Code (MAC) keys

   request_mac_keys()
       Request the Message Authentication Code (MAC) keys from Google.

   validate_data_mac()
       Validate	data against the MAC keys.

   update_error()
       Handle server errors during a database update.

   lookup_whitelist()
       Lookup a	host prefix and	suffix in the whitelist	(s chunks)

   ua()
       Create LWP::UserAgent to	make HTTP requests to Google.

   parse_s()
       Parse data from a rediration (add asnd sub chunk	information).

   parse_s()
       Parse s chunks information for a	database update.

   parse_a()
       Parse a chunks information for a	database update.

   hex_to_ascii()
       Transform hexadecimal strings to	printable ASCII	strings. Used mainly
       for debugging.

	 print $gsb->hex_to_ascii('hex value');

   ascii_to_hex()
       Transform ASCII strings to hexadecimal strings.

   debug()
       Print debug output.

   error()
       Print error message.

   error()
       Print performance message.

   canonical_domain_suffixes()
       Find all	suffixes for a domain.

   canonical_domain()
       Find all	canonical domains a domain.

   canonical_path()
       Find all	canonical paths	for a URL.

   canonical()
       Find all	canonical URLs for a URL.

   canonical_uri()
       Create a	canonical URI.

       NOTE: URI cannot	handle all the test cases provided by Google. This
       method is a hack	to pass	most of	the test. A few	tests are still
       failing.	The proper way to handle URL canonicalization according	to
       Google would be to create a new module to handle	URLs. However, I
       believe most real-life cases are	handled	correctly by this function.

   canonical()
       Return all possible full	hashes for a URL.

   prefix()
       Return a	hash prefix. The size of the prefix is set to 4	bytes.

   request_full_hash()
       Request full full hashes	for specific prefixes from Google.

   parse_full_hashes()
       Process the request for full hashes from	Google.

   get_a_range()
       Get the list of a chunks	ranges for a list update.

   get_s_range()
       Get the list of s chunks	ranges for a list update.

   create_range()
       Create a	list of	ranges (1-3, 5,	7-11) from a list of numbers.

   expand_range()
       Explode list of ranges (1-3, 5, 7-11) into a list of numbers
       (1,2,3,5,7,8,9,10,11).

CHANGELOG
       1.09
	   Use HTTPS to	access safebrowsing.clients.google.com/.

       1.07
	   Add "import_chunks()" feature to import add chunks and sub chunks
	   from	a file.

       1.05
	   No code change. Move	"local_lookup" to PRIVATE FUNCTIONS to avoid
	   confusions.

       1.04
	   Introduce Net::Google::SafeBrowsing2::Lookup. Remind	people that
	   Google Safe Browsing	v1 has been deprecated by Google.

       1.03
	   The source code is available	on github at
	   <https://github.com/juliensobrier/Net-Google-SafeBrowsing2>.

       1.02
	   Fix uninitialized $self->{errors} variable

       1.01
	   Use String::HexConvert for faster hex_to_ascii.

       1.0 Separate the	error output from the debug output.

       0.9 Fix bug with	local whitelisting (sub	chunks). Fix the parsing of
	   full	hashes.

       0.8 Reduce the number of	full hash requests.

       0.7 Add local_lookup to perform a lookup	against	the local database
	   only. This function should be used for debugging purpose only.
	   Small code optimizations.

       0.6 Handle local	database reset.

       0.5 Update documentation.

       0.4 Speed update	the database update. The first update went down	from
	   20 minutes to 20 minutes.

       0.3 Fix typos in	the documentation.

	   Remove dependency on	Switch (thanks to Curtis Jewel).

	   Fix value of	FULL_HASH_TIME.

       0.2 Add support for Message Authentication Code (MAC)

SEE ALSO
       See Net::Google::SafeBrowsing2::Storage,
       Net::Google::SafeBrowsing2::Sqlite and
       Net::Google::SafeBrowsing2::MySQL for information on storing and
       managing	the Google Safe	Browsing database.

       Google Safe Browsing v2 API:
       <http://code.google.com/apis/safebrowsing/developers_guide_v2.html>

       Net::Google::SafeBrowsing (Google Safe Browsing v1) is deprecated by
       Google since 12/01/2011.

AUTHOR
       Julien Sobrier, <jsobrier@zscaler.com> or <julien@sobrier.net>

COPYRIGHT AND LICENSE
       Copyright (C) 2012 by Julien Sobrier

       This library is free software; you can redistribute it and/or modify it
       under the same terms as Perl itself, either Perl	version	5.8.8 or, at
       your option, any	later version of Perl 5	you may	have available.

perl v5.32.1			  2013-06-06	 Net::Google::SafeBrowsing2(3)

NAME | SYNOPSIS | DESCRIPTION | CONSTANTS | CONSTRUCTOR | PUBLIC FUNCTIONS | PRIVATE FUNCTIONS | CHANGELOG | SEE ALSO | AUTHOR | COPYRIGHT AND LICENSE

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=Net::Google::SafeBrowsing2&sektion=3&manpath=FreeBSD+13.0-RELEASE+and+Ports>

home | help