Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
MD5DEEP(1)		    United States Air Force		    MD5DEEP(1)

NAME
       md5deep - Compute and compare MD5 message digests
       sha1deep	- Compute and compare SHA-1 message digests
       sha256deep - Compute and	compare	SHA-256	message	digests
       sha3deep	- Compute and compare SHA-3-256	message	digests
       tigerdeep - Compute and compare Tiger message digests
       whirlpooldeep - Compute and compare Whirlpool message digests

SYNOPSIS
       md5deep -v | -V | -h
       md5deep	[-m|-M|-x|-X  <file>]	[-a|-A <hash>] [-f <file>] [-p <size>]
       [-i <size>] [-tnwzresS0lbkqZud] [-F <bum>] [-o <fbcplsde>]  [-j	<num>]
       [[FILES]

DESCRIPTION
       Computes	 the  hashes, or message digest, for any number	of files while
       optionally recursively digging through the  directory  structure.   Can
       also  take  a  list  of known hashes and	display	the filenames of input
       files whose hashes either do or do not match any	of the	known  hashes.
       Errors are reported to standard error. If no FILES are specified, reads
       from standard input.

       -p <size>
	      Piecewise	mode. Breaks files into	chunks before hashing.	Chunks
	      may  be  specified using IEC multipliers b, k, m,	g, t, p, or e.
	      (Never let it be said that the author didn't plan	ahead!)	  This
	      mode cannot be used with the -z mode.

       -i|-I <size>
	      Size  threshold mode. Only hash files smaller than the given the
	      threshold. In -i mode, simply omits those	files larger than  the
	      threshold.  In  -I  mode,	displays all files, but	uses asterisks
	      for the hashes of	files larger than the threshold.  Sizes	may be
	      specified	using IEC multipliers b, k, m, g, t, p,	or e.

       -r     Enables recursive	mode. All subdirectories are traversed.	Please
	      note that	recursive mode cannot be used to examine all files  of
	      a	 given	file  extension. For example, calling md5deep -r *.txt
	      will examine all files in	directories that end in	.txt.

       -e     Displays a progress indicator and	estimate of time remaining for
	      each  file being processed. Time estimates for files larger than
	      4GB are not available on Windows.	This mode may not be used with
	      th -p mode.

       -m <file>
	      Enables  matching	mode. The file given should be a list of known
	      hashes.  The input files are examined one	at a  time,  and  only
	      those files that match the list of known hashes are output. This
	      flag may be used more than once to add multiple  sets  of	 known
	      hashes.  Acceptable  formats for lists of	known hashes are plain
	      (such as those  generated	 by  md5deep  or  md5sum),  Hashkeeper
	      files, iLook, and	the National Software Reference	Library	(NSRL)
	      as produced by the National Institute for	Standards in  Technol-
	      ogy.
	       If standard input is used with the -m flag, displays "stdin" if
	      the input	matches	one of the hashes in the list of known hashes.
	      If the hash does not match, the program displays no output.
	       This flag may not be used in conjunction	with the -x, -X, or -A
	      flags.  See the section "UNICODE SUPPORT"	below.

       -x <file>
	      Same as the -m flag above, but does negative matching. That  is,
	      only those files NOT in the list of known	hashes are displayed.
	       This flag may not be used in conjunction	with the -m, -M, or -a
	      flags.  See the section "UNICODE SUPPORT"	below.

       -M and -X <file>
	      Same as -m and -x	above, but displays the	 hash  for  each  file
	      that does	(or does not) match the	list of	known hashes.

       -a <hash>
	      Adds a single hash to the	list of	known hashes used for matching
	      mode, and	if not already enabled,	enables	matching mode.	Adding
	      single  hashes cannot, by	itself,	be used	to print the hashes of
	      matching files like the -M flag does. When used  in  conjunction
	      with  the	 -w flag, the filename displayed is just the hash sub-
	      mitted on	the command line.
	       This flag may not be used in conjunction	with the -x, -X, or -A
	      flags.

       -A <hash>
	      Same as -a above,	but does negative matching.  This flag may not
	      be used in conjunction with the -m, -M, or -A flags.

       -f <file>
	      Takes a list of files to be hashed from the specified file. Each
	      line  is	assumed	 to  be	a filename. This flag can only be used
	      once per invocation. If it's used	a second time, the second  in-
	      stance will clobber the first.
	      Note  that  you  can still use other flags, such as the -m or -x
	      modes, and submit	additional FILES on the	command	line.

       -w     During any of the	matching modes (-m,-M,-x,or -X), displays  the
	      filename of the known hash that matched the input	file.  See the
	      section "UNICODE SUPPORT"	below.

       -t     Display a	timestamp in GMT with each  result.  On	 Windows  this
	      timestamp	will be	the file's creation time. On all other systems
	      it should	be the file's change time.

       -n     During any of the	matching modes (-m,-M,-x,or -X), displays only
	      the  filenames  of any known hashes that were not	matched	by any
	      of the input files.

       -s     Enables silent mode. All error messages are supressed.

       -S     Like silent mode,	but still displays warnings on improperly for-
	      matted hashes in the list	of known hashes.

       -z     Enables  file size mode. Prepends	the hash with a	ten digit rep-
	      resentation of the size of each file processed. If the file size
	      is  greater than 9999999999 bytes	(about 9.3GB) the program dis-
	      plays 9999999999 for the size.

       -q     Quiet mode. File names are omitted from the output. Each hash is
	      still followed by	two spaces before the newline.

       -Z     Produces	output	in Triage format. Each line contans the	file's
	      size, a tab, a hash of the first 512 bytes, a tab, the  hash  of
	      the  complete  file, a tab, and the file name.  These values are
	      intended in increasing order of specificity. That	is, two	 files
	      with  different sizes cannot possibly match. This	is a fast com-
	      parison and should be done first.	Next, two files	with different
	      partial  hashes cannot possibly match. This is often faster than
	      hashing the whole	file. Finally, if those	two pieces align, then
	      it's worth reading and hashing the entire	file.

       -0     Uses  a  NULL character (/0) to terminate	each line instead of a
	      newline.	Useful for processing filenames	with  strange  charac-
	      ters.

       -l     Enables  relative	 file  paths. Instead of printing the absolute
	      path for each file, displays the relative	file path as indicated
	      on  the  command	line. This flag	may not	be used	in conjunction
	      with the -b flag.

       -b     Enables bare mode. Strips	any leading directory information from
	      displayed	 filenames.   This flag	may not	be used	in conjunction
	      with the -l flag.

       -k     Enables asterisk mode. An	asterisk is inserted in	lieu of	a sec-
	      ond space	between	the filename and the hash, just	like md5sum in
	      its binary (-b) mode.

       -c     Enables comma separated values output, or	CSV  mode.  This  mode
	      has  the	side  effect  of removing the 10 digit size limitation
	      from -z mode.  Also note that asterisks from  -k	mode  are  not
	      displayed	when in	CSV mode.

       -o <bcpflsd>
	      Enables  expert  mode.  Allows  the user specify which (and only
	      which) types of files are	 processed.  Directory	processing  is
	      still  controlled	 with the -r flag. The expert mode options al-
	      lowed are:
	      f	- Regular files
	      b	- Block	Devices
	      c	- Character Devices
	      p	- Named	Pipes
	      l	- Symbolic Links
	      s	- Sockets
	      d	- Solaris Doors
	      e	- Windows PE executables

       -jnn   Controls multi-threading.	By default the program will create one
	      producer	thread	to scan	the file system	and one	hashing	thread
	      per CPU core. Multi-threading causes output filenames to	be  in
	      non-deterministic	 order,	as files that take longer to hash will
	      be delayed while they are	hashed.	If a  deterministic  order  is
	      required,	specify	-j0 to disable multi-threading

       -d     Output in	Digital	Forensics XML (DFXML) format.

       -u     Quote  Unicode  output.  For  example,  the  snowman is shown as
	      U+C426.

       -F<bum>
	      Specifies	the input mode that is used to read files. The default
	      is -Fb (buffered I/O) which reads	files with fopen(). Specifying
	      -Fu will use unbuffered I/O and read the file with open(). Spec-
	      ifying  -Fm  will	 use memory-mapped I/O which will be faster on
	      some platforms, but which	(currently) will not work  with	 files
	      that produce I/O errors.

       -h     Show a help screen and exit.

       -v     Show the version number and exit.

       -V     Show copyright information and exit.

UNICODE	SUPPORT
       As  of version 3.0 the program supports Unicode characters in filenames
       on Microsoft Windows systems for	filenames  specified  on  the  command
       line  with  globbing (e.g. *), for files	specified with the -f of files
       to hash,	and for	files read from	directories using the -r option.

       By default all program input and	output should be in UTF-8.   The  pro-
       gram automatically converts this	to UTF-16 for opening files).

       On  Unix/Linux/MacOS,  you should use a terminal	emulator that supports
       UTF-8 and UTF-8 characters in filenames will be properly	displayed.

       On Windows, the programs	do not display Unicode characters on the  con-
       sole.  You must either redirect output to a file	and open the file with
       Wordpad (which can display Unicode), or you must	specify	the -u	option
       to quote	Unicode	using standard U+XXXX notation.

       Currently  the  file  name of a file containing known hashes may	not be
       specified as a unicode filename,	but you	can specify the	name using tab
       completition  or	an asterisk (e.g. md5deep -m *.txt where there is only
       one file	with a .txt extension).

RETURN VALUE
       Returns a bit-wise value	based on the success of	the operation and  the
       status of any matching operations.

       0      Success.	Note that the program considers	itself successful even
	      when it encounters read errors,  permission  denied  errors,  or
	      finds directories	when not in recursive mode.

       1      Unused  hashes.  Under  any  of the matching modes, returns this
	      value if one or more of the known	hashes was not matched by  any
	      of the input files.

       2      Unmatched	 inputs. Under any of the matching modes, returns this
	      value if one or more of the input	values did not	match  any  of
	      the known	hashes.

       64     User  error,  such  as  trying  to do both positive and negative
	      matching at the same time.

       128    Internal error, such as memory  corruption  or  uncaught	cycle.
	      All internal errors should be reported to	the developer! See the
	      section "Reporting Bugs" below.

AUTHOR
       md5deep was written by Jesse Kornblum,  research@jessekornblum.com  and
       Simson Garfinkel.

KNOWN ISSUES
       Using  the -r flag cannot be used to recursively	process	all files of a
       given extension in a directory. This is a feature, not a	bug.   If  you
       need to do this,	use the	find(1)	command.

REPORTING BUGS
       We  take	 all  bug reports very seriously. Any bug that jeopardizes the
       forensic	integrity of this program could	have serious  consequences  on
       people's	lives. When submitting a bug report, please include a descrip-
       tion of the problem, how	you found it, and your contact information.

       Send bug	reports	to the author at the address above.

COPYRIGHT
       This program is a work of the US	Government. In accordance with 17  USC
       105,  copyright protection is not available for any work	of the US Gov-
       ernment.	 This program is PUBLIC	DOMAIN.	Portions of this program  con-
       tain  code  that	 is licensed under the terms of	the General Public Li-
       cense (GPL).  Those portions retain their original  copyright  and  li-
       cense. See the file COPYING for more details.

       There  is NO warranty for this program; not even	for MERCHANTABILITY or
       FITNESS FOR A PARTICULAR	PURPOSE.

SEE ALSO
       More information	and installation instructions  can  be	found  in  the
       README  file.  Current  versions	 of both documents can be found	on the
       project homepage: http://md5deep.sourceforge.net/

       The MD5 specification, RFC 1321,	is available at
       http://www.ietf.org/rfc/rfc1321.txt

       The SHA-1 specification,	RFC 3174, is available at
       http://www.faqs.org/rfcs/rfc3174.html

       The SHA-256 specification, FIPS 180-2, is available at
       http://csrc.nist.gov/publications/fips/fips180-2/fips180-2.pdf

       The SHA-3-256 specification is available	at
       http://keccak.noekeon.org/

       The Tiger specification is available at
       http://www.cs.technion.ac.il/~biham/Reports/Tiger/

       The Whirlpool specification is available	at
       http://planeta.terra.com.br/informatica/paulobarreto/WhirlpoolPage.html

AFOSI			      v4.4 - 29	Jan 2014		    MD5DEEP(1)

NAME | SYNOPSIS | DESCRIPTION | UNICODE SUPPORT | RETURN VALUE | AUTHOR | KNOWN ISSUES | REPORTING BUGS | COPYRIGHT | SEE ALSO

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=md5deep&manpath=FreeBSD+12.0-RELEASE+and+Ports>

home | help