Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
MD5DEEP(1)		    United States Air Force		    MD5DEEP(1)

NAME
       md5deep - Compute and compare MD5 message digests
       sha1deep	- Compute and compare SHA-1 message digests
       sha256deep - Compute and	compare	SHA-256	message	digests
       sha3deep	- Compute and compare SHA-3-256	message	digests
       tigerdeep - Compute and compare Tiger message digests
       whirlpooldeep - Compute and compare Whirlpool message digests

SYNOPSIS
       md5deep -v | -V | -h
       md5deep	[-m|-M|-x|-X  <file>]	[-a|-A <hash>] [-f <file>] [-p <size>]
       [-i <size>] [-tnwzresS0lbkqZud] [-F <bum>] [-o <fbcplsde>]  [-j	<num>]
       [[FILES]

DESCRIPTION
       Computes	 the  hashes, or message digest, for any number	of files while
       optionally recursively digging through the  directory  structure.   Can
       also  take  a  list  of known hashes and	display	the filenames of input
       files whose hashes either do or do not match any	of the	known  hashes.
       Errors are reported to standard error. If no FILES are specified, reads
       from standard input.

       -p <size>
	      Piecewise	mode. Breaks files into	chunks before hashing.	Chunks
	      may  be  specified using IEC multipliers b, k, m,	g, t, p, or e.
	      (Never let it be said that the author didn't plan	ahead!)	  This
	      mode cannot be used with the -z mode.

       -i|-I <size>
	      Size  threshold mode. Only hash files smaller than the given the
	      threshold. In -i mode, simply omits those	files larger than  the
	      threshold.  In  -I  mode,	displays all files, but	uses asterisks
	      for the hashes of	files larger than the threshold.  Sizes	may be
	      specified	using IEC multipliers b, k, m, g, t, p,	or e.

       -r     Enables recursive	mode. All subdirectories are traversed.	Please
	      note that	recursive mode cannot be used to examine all files  of
	      a	 given	file  extension. For example, calling md5deep -r *.txt
	      will examine all files in	directories that end in	.txt.

       -e     Displays a progress indicator and	estimate of time remaining for
	      each  file being processed. Time estimates for files larger than
	      4GB are not available on Windows.	This mode may not be used with
	      th -p mode.

       -m <file>
	      Enables  matching	mode. The file given should be a list of known
	      hashes.  The input files are examined one	at a  time,  and  only
	      those files that match the list of known hashes are output. This
	      flag may be used more than once to add multiple  sets  of	 known
	      hashes.  Acceptable  formats for lists of	known hashes are plain
	      (such as those  generated	 by  md5deep  or  md5sum),  Hashkeeper
	      files, iLook, and	the National Software Reference	Library	(NSRL)
	      as produced by the National Institute for	Standards in  Technol-
	      ogy.
	       If standard input is used with the -m flag, displays "stdin" if
	      the input	matches	one of the hashes in the list of known hashes.
	      If the hash does not match, the program displays no output.
	       This flag may not be used in conjunction	with the -x, -X, or -A
	      flags.  See the section "UNICODE SUPPORT"	below.

       -x <file>
	      Same as the -m flag above, but does negative matching. That  is,
	      only  those files	NOT in the list	of known hashes	are displayed.
	       This flag may not be used in conjunction	with the -m, -M, or -a
	      flags.  See the section "UNICODE SUPPORT"	below.

       -M and -X <file>
	      Same  as	-m  and	 -x above, but displays	the hash for each file
	      that does	(or does not) match the	list of	known hashes.

       -a <hash>
	      Adds a single hash to the	list of	known hashes used for matching
	      mode,  and if not	already	enabled, enables matching mode.	Adding
	      single hashes cannot, by itself, be used to print	the hashes  of
	      matching	files  like the	-M flag	does. When used	in conjunction
	      with the -w flag,	the filename displayed is just the  hash  sub-
	      mitted on	the command line.
	       This flag may not be used in conjunction	with the -x, -X, or -A
	      flags.

       -A <hash>
	      Same as -a above,	but does negative matching.  This flag may not
	      be used in conjunction with the -m, -M, or -A flags.

       -f <file>
	      Takes a list of files to be hashed from the specified file. Each
	      line is assumed to be a filename.	This flag  can	only  be  used
	      once  per	 invocation.  If  it's	used a second time, the	second
	      instance will clobber the	first.
	      Note that	you can	still use other	flags, such as the  -m	or  -x
	      modes, and submit	additional FILES on the	command	line.

       -w     During  any of the matching modes	(-m,-M,-x,or -X), displays the
	      filename of the known hash that matched the input	file.  See the
	      section "UNICODE SUPPORT"	below.

       -t     Display  a  timestamp  in	 GMT with each result. On Windows this
	      timestamp	will be	the file's creation time. On all other systems
	      it should	be the file's change time.

       -n     During any of the	matching modes (-m,-M,-x,or -X), displays only
	      the filenames of any known hashes	that were not matched  by  any
	      of the input files.

       -s     Enables silent mode. All error messages are supressed.

       -S     Like silent mode,	but still displays warnings on improperly for-
	      matted hashes in the list	of known hashes.

       -z     Enables file size	mode. Prepends the hash	with a ten digit  rep-
	      resentation of the size of each file processed. If the file size
	      is greater than 9999999999 bytes (about 9.3GB) the program  dis-
	      plays 9999999999 for the size.

       -q     Quiet mode. File names are omitted from the output. Each hash is
	      still followed by	two spaces before the newline.

       -Z     Produces output in Triage	format.	Each line contans  the	file's
	      size,  a	tab, a hash of the first 512 bytes, a tab, the hash of
	      the complete file, a tab,	and the	file name.  These  values  are
	      intended	in increasing order of specificity. That is, two files
	      with different sizes cannot possibly match. This is a fast  com-
	      parison and should be done first.	Next, two files	with different
	      partial hashes cannot possibly match. This is often faster  than
	      hashing the whole	file. Finally, if those	two pieces align, then
	      it's worth reading and hashing the entire	file.

       -0     Uses a NULL character (/0) to terminate each line	instead	 of  a
	      newline.	 Useful	 for processing	filenames with strange charac-
	      ters.

       -l     Enables relative file paths. Instead of  printing	 the  absolute
	      path for each file, displays the relative	file path as indicated
	      on the command line. This	flag may not be	 used  in  conjunction
	      with the -b flag.

       -b     Enables bare mode. Strips	any leading directory information from
	      displayed	filenames.  This flag may not be used  in  conjunction
	      with the -l flag.

       -k     Enables asterisk mode. An	asterisk is inserted in	lieu of	a sec-
	      ond space	between	the filename and the hash, just	like md5sum in
	      its binary (-b) mode.

       -c     Enables  comma  separated	 values	output,	or CSV mode. This mode
	      has the side effect of removing the  10  digit  size  limitation
	      from  -z	mode.	Also  note that	asterisks from -k mode are not
	      displayed	when in	CSV mode.

       -o <bcpflsd>
	      Enables expert mode. Allows the user  specify  which  (and  only
	      which)  types  of	 files	are processed. Directory processing is
	      still controlled with the	 -r  flag.  The	 expert	 mode  options
	      allowed are:
	      f	- Regular files
	      b	- Block	Devices
	      c	- Character Devices
	      p	- Named	Pipes
	      l	- Symbolic Links
	      s	- Sockets
	      d	- Solaris Doors
	      e	- Windows PE executables

       -jnn   Controls multi-threading.	By default the program will create one
	      producer thread to scan the file system and one  hashing	thread
	      per  CPU	core. Multi-threading causes output filenames to be in
	      non-deterministic	order, as files	that take longer to hash  will
	      be  delayed  while  they are hashed. If a	deterministic order is
	      required,	specify	-j0 to disable multi-threading

       -d     Output in	Digital	Forensics XML (DFXML) format.

       -u     Quote Unicode output. For	 example,  the	snowman	 is  shown  as
	      U+C426.

       -F<bum>
	      Specifies	the input mode that is used to read files. The default
	      is -Fb (buffered I/O) which reads	files with fopen(). Specifying
	      -Fu will use unbuffered I/O and read the file with open(). Spec-
	      ifying -Fm will use memory-mapped	I/O which will	be  faster  on
	      some  platforms,	but which (currently) will not work with files
	      that produce I/O errors.

       -h     Show a help screen and exit.

       -v     Show the version number and exit.

       -V     Show copyright information and exit.

UNICODE	SUPPORT
       As of version 3.0 the program supports Unicode characters in  filenames
       on  Microsoft  Windows  systems	for filenames specified	on the command
       line with globbing (e.g.	*), for	files specified	with the -f  of	 files
       to hash,	and for	files read from	directories using the -r option.

       By  default  all	program	input and output should	be in UTF-8.  The pro-
       gram automatically converts this	to UTF-16 for opening files).

       On Unix/Linux/MacOS, you	should use a terminal emulator	that  supports
       UTF-8 and UTF-8 characters in filenames will be properly	displayed.

       On  Windows, the	programs do not	display	Unicode	characters on the con-
       sole.  You must either redirect output to a file	and open the file with
       Wordpad	(which can display Unicode), or	you must specify the -u	option
       to quote	Unicode	using standard U+XXXX notation.

       Currently the file name of a file containing known hashes  may  not  be
       specified as a unicode filename,	but you	can specify the	name using tab
       completition or an asterisk (e.g. md5deep -m *.txt where	there is  only
       one file	with a .txt extension).

RETURN VALUE
       Returns	a bit-wise value based on the success of the operation and the
       status of any matching operations.

       0      Success. Note that the program considers itself successful  even
	      when  it	encounters  read  errors, permission denied errors, or
	      finds directories	when not in recursive mode.

       1      Unused hashes. Under any of the  matching	 modes,	 returns  this
	      value  if	one or more of the known hashes	was not	matched	by any
	      of the input files.

       2      Unmatched	inputs.	Under any of the matching modes, returns  this
	      value  if	 one  or more of the input values did not match	any of
	      the known	hashes.

       64     User error, such as trying to  do	 both  positive	 and  negative
	      matching at the same time.

       128    Internal	error,	such  as  memory corruption or uncaught	cycle.
	      All internal errors should be reported to	the developer! See the
	      section "Reporting Bugs" below.

AUTHOR
       md5deep	was  written by	Jesse Kornblum,	research@jessekornblum.com and
       Simson Garfinkel.

KNOWN ISSUES
       Using the -r flag cannot	be used	to recursively process all files of  a
       given  extension	 in a directory. This is a feature, not	a bug.	If you
       need to do this,	use the	find(1)	command.

REPORTING BUGS
       We take all bug reports very seriously. Any bug	that  jeopardizes  the
       forensic	 integrity  of this program could have serious consequences on
       people's	lives. When submitting a bug report, please include a descrip-
       tion of the problem, how	you found it, and your contact information.

       Send bug	reports	to the author at the address above.

COPYRIGHT
       This  program is	a work of the US Government. In	accordance with	17 USC
       105, copyright protection is not	available for any work of the US  Gov-
       ernment.	  This program is PUBLIC DOMAIN. Portions of this program con-
       tain code that is licensed  under  the  terms  of  the  General	Public
       License	(GPL).	 Those	portions  retain  their	original copyright and
       license.	See the	file COPYING for more details.

       There is	NO warranty for	this program; not even for MERCHANTABILITY  or
       FITNESS FOR A PARTICULAR	PURPOSE.

SEE ALSO
       More  information  and  installation  instructions  can be found	in the
       README file. Current versions of	both documents can  be	found  on  the
       project homepage: http://md5deep.sourceforge.net/

       The MD5 specification, RFC 1321,	is available at
       http://www.ietf.org/rfc/rfc1321.txt

       The SHA-1 specification,	RFC 3174, is available at
       http://www.faqs.org/rfcs/rfc3174.html

       The SHA-256 specification, FIPS 180-2, is available at
       http://csrc.nist.gov/publications/fips/fips180-2/fips180-2.pdf

       The SHA-3-256 specification is available	at
       http://keccak.noekeon.org/

       The Tiger specification is available at
       http://www.cs.technion.ac.il/~biham/Reports/Tiger/

       The Whirlpool specification is available	at
       http://planeta.terra.com.br/informatica/paulobarreto/WhirlpoolPage.html

AFOSI			      v4.4 - 29	Jan 2014		    MD5DEEP(1)

NAME | SYNOPSIS | DESCRIPTION | UNICODE SUPPORT | RETURN VALUE | AUTHOR | KNOWN ISSUES | REPORTING BUGS | COPYRIGHT | SEE ALSO

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=md5deep&manpath=FreeBSD+11.1-RELEASE+and+Ports>

home | help