Imported Upstream version 1.5+git20161027.bc13175

parents
#
# Build ignores
#
#.*
*.o
*.o.*
*.a
*.so
*.so.*
*.1.gz
#
# Never ignore these
#
!.gitignore
#
# Normal output and testing dirs
#
/?dupes
/?dupes-jody
/?dupes*.exe
#
# Backups / patches
#
*~
*.orig
*.rej
/*.patch
#
# debugging and editor stuff
#
core
.gdb_history
.gdbinit
.*.swp
*.gcda
*.gcno
*.gcov
# Mac OS
.DS_Store
jdupes 1.5
- Invert -Z option: only "soft abort" if asked explicitly to do so
- Tweak internal data chunk size to reduce data cache misses
- Fix partial hash optimization
- Change PREFIX for building from /usr/local back to /usr
jdupes 1.4
- Add support for Unicode file paths on Windows platforms
- Discard floating point code of dubious value
- Remove -1/--sameline feature which is not practically useful
- Process partially complete duplicate scan if CTRL+C is pressed
- Add -Z/--hardabort option to disable the new CTRL+C behavior
- Add [n]one option to -d/--delete to discard all files in a match set
- Minor bug fixes and tweaks to improve behavior
jdupes 1.3
- Add -i/--reverse to invert the match sort order
- Add -I/--isolate to force cross-parameter matching
- Add "loud" debugging messages (-@ switch, build with 'make LOUD=1')
jdupes 1.2.1
- Fix a serious bug that caused some duplicates to be missed
jdupes 1.2
- Change I/O block size for improved performance
- Improved progress indicator behavior with large files; now the progress
indicator will update more frequently when full file reads are needed
- Windows read speed boost with _O_SEQUENTIAL file flag
- Experimental tree rebalance code tuning
jdupes 1.1.1
- Fix a bug where the -r switch was always on even if not specified
jdupes 1.1
- Work around the 1023-link limit for Windows hard linking so that linking
can continue even when the limit is reached
- Update documentation to include hard link arrow explanations
- Add "time of check to time of use" checks immediately prior to taking
actions on files so that files which changed since being checked will not
be touched, avoiding potential data loss on "live" data sets
- Add debug stats for files skipped due to Windows hard link limit
- Change default sort to filename instead of modification time
- Replaced Windows "get inode number" code with simpler, faster version
- Fixed a bug where an extra newline was at the end of printed matches
- Reduced progress delay interval; it was a bit slow on many large files
jdupes 1.0.2
- Update jody_hash code to latest version
- Change string_malloc to enable future string_free() improvements
- Add string_malloc counters for debug stat mode
- Add '+size' option to -x/--xsize switch to exclude files larger than the
specified size instead of smaller than that size
jdupes 1.0.1
- Fix bug in deletion set counter that would show e.g. "Set 1 of 0"
- Minor size reductions by merging repeated fixed strings
- Add feature flag 'fastmath' to show when compiled with -ffast-math
- Corrections to code driven by -Wconversion and -Wwrite-strings
jdupes 1.0
First release. For changes before the 'jdupes' name change, see OLD_CHANGES
Installing jdupes
--------------------------------------------------------------------
To install the program, issue the following commands:
make
su root
make install
This will install the program in /usr/local/bin. You may change this
to a different location by editing the Makefile. Please refer to the
Makefile for an explanation of compile-time options. If you're having
trouble compiling, please take a look at the Makefile.
Various build options are available and can be turned on at compile
time by setting CFLAGS_EXTRA or by passing it to 'make':
make CFLAGS_EXTRA=-DYOUR_OPTION
make CFLAGS_EXTRA='-DYOUR_OPTION_ONE -DYOUR_OPTION_TWO'
This is a list of options that can be "turned on" this way:
ENABLE_BTRFS Enable '-B/--dedupe' for btrfs deduplication
HAVE_BTRFS_IOCTL_H Same as ENABLE_BTRFS
DEBUG * Turn on algorithm statistic reporting with '-D'
OMIT_GETOPT_LONG Do not use getopt_long() C library call
ON_WINDOWS Modify code to compile with MinGW on Windows
USE_TREE_REBALANCE * Use experimental tree rebalancing code
CONSIDER_IMBALANCE * Change tree rebalance to analyze weights first
* These options may slow down the program somewhat and are off by
default. Do not enable them unless you are experimenting.
You can turn on the -@ option for "loud" debugging with:
make LOUD=1
Non-loud debugging can be enabled with
make DEBUG=1
A test directory is included so that you may familiarize yourself with
the way jdupes operates. You may test the program before installing
it by issuing a command such as "./jdupes testdir" or
"./jdupes -r testdir", just to name a couple of examples. Refer to the
documentation for information on valid options.
A comparison shell script is also included. It will run your natively
installed 'jdupes' or 'jdupes' with the directories and extra options
you specify and compare the run times and output a 'diff' of the two
program outputs. Unless the core algorithm or sort behavior is changed,
both programs should produce identical outputs and the 'diff' output
shouldn't appear at all. To use it, type:
./compare_jdupes.sh [options]
The MIT License (MIT)
Copyright (c) 2015-2016 Jody Lee Bruchon and contributors
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
# jdupes Makefile
#####################################################################
# Standand User Configuration Section #
#####################################################################
# PREFIX determines where files will be installed. Common examples
# include "/usr" or "/usr/local".
PREFIX = /usr
# Certain platforms do not support long options (command line options).
# To disable long options, uncomment the following line.
#CFLAGS_CONFIG += -DOMIT_GETOPT_LONG
# Uncomment if you have btrfs/ioctl.h. Needed for --dedupe.
#HAVE_BTRFS_IOCTL_H = -DHAVE_BTRFS_IOCTL_H
#####################################################################
# Developer Configuration Section #
#####################################################################
# PROGRAM_NAME determines the installation name and manual page name
PROGRAM_NAME=jdupes
# BIN_DIR indicates directory where program is to be installed.
# Suggested value is "$(PREFIX)/bin"
BIN_DIR = $(PREFIX)/bin
# MAN_DIR indicates directory where the jdupes man page is to be
# installed. Suggested value is "$(PREFIX)/man/man1"
MAN_BASE_DIR = $(PREFIX)/share/man
MAN_DIR = $(MAN_BASE_DIR)/man1
MAN_EXT = 1
# Required External Tools
INSTALL = install # install : UCB/GNU Install compatiable
#INSTALL = ginstall
RM = rm -f
MKDIR = mkdir -p
#MKDIR = mkdirhier
#MKDIR = mkinstalldirs
# Make Configuration
CC ?= gcc
COMPILER_OPTIONS = -Wall -Wextra -Wwrite-strings -Wcast-align -Wstrict-aliasing -pedantic -Wstrict-overflow
COMPILER_OPTIONS += -std=gnu99 -O2 -g -D_FILE_OFFSET_BITS=64 -fstrict-aliasing -pipe
# Debugging code inclusion
ifdef LOUD
DEBUG=1
COMPILER_OPTIONS += -DLOUD_DEBUG
endif
ifdef DEBUG
COMPILER_OPTIONS += -DDEBUG
endif
# MinGW needs this for printf() conversions to work
ifeq ($(OS), Windows_NT)
COMPILER_OPTIONS += -D__USE_MINGW_ANSI_STDIO=1 -municode
OBJECT_FILES += win_stat.o
endif
CFLAGS += $(COMPILER_OPTIONS) -I. $(CFLAGS_CONFIG) $(CFLAGS_EXTRA) $(HAVE_BTRFS_IOCTL_H)
INSTALL_PROGRAM = $(INSTALL) -c -m 0755
INSTALL_DATA = $(INSTALL) -c -m 0644
# ADDITIONAL_OBJECTS - some platforms will need additional object files
# to support features not supplied by their vendor. Eg: GNU getopt()
#ADDITIONAL_OBJECTS += getopt.o
OBJECT_FILES += jdupes.o jody_hash.o string_malloc.o $(ADDITIONAL_OBJECTS)
#####################################################################
# no need to modify anything beyond this point #
#####################################################################
all: jdupes
jdupes: $(OBJECT_FILES)
$(CC) $(CFLAGS) $(LDFLAGS) -o jdupes $(OBJECT_FILES)
installdirs:
test -d $(DESTDIR)$(BIN_DIR) || $(MKDIR) $(DESTDIR)$(BIN_DIR)
test -d $(DESTDIR)$(MAN_DIR) || $(MKDIR) $(DESTDIR)$(MAN_DIR)
install: jdupes installdirs
$(INSTALL_PROGRAM) jdupes $(DESTDIR)$(BIN_DIR)/$(PROGRAM_NAME)
$(INSTALL_DATA) jdupes.1 $(DESTDIR)$(MAN_DIR)/$(PROGRAM_NAME).$(MAN_EXT)
clean:
$(RM) $(OBJECT_FILES) jdupes jdupes.exe *~ *.gcno *.gcda *.gcov
=== NOTE: This is archived material from 'fdupes' development ===
=== and from the pre-'jdupes' code work. DO NOT EDIT. ===
The following list, organized by fdupes version, documents changes
to fdupes. Every item on the list includes, inside square brackets,
a list of indentifiers referring to the people who contributed
that particular item. When more than one person is listed the person
who contributed the patch or idea appears first, followed by
those who've otherwise worked on that item. For a list of
contributors names and identifiers please see the CONTRIBUTORS file.
Changes from 2.1 to 2.2 [JLB]
- Changed fdupes-jody executable names and document texts to use the
full 'fdupes-jody' name instead of 'fdupes'. Moved copyrights and
contact information to reflect 'fdupes-jody' code responsibility.
This is primarily intended to keep fdupes-jody distinctly separate
from the fdupes by Adrian Lopez upon which it is based, and to make
certain that the correct person gets harassed if it breaks ;-)
- Added '-B/--dedupe' feature (not compiled in by default) which
sends file match lists directly to the kernel btrfs driver to do
block-level data de-duplication. Patch submitted by Sebastian
Schmidt <yath@yath.de>. Thanks!
- Remove and replace some string function calls. Performance increase
shows on benchmarks but not significant in most cases.
Changes from 2.0.2 to 2.1 [JLB]
- Minor performance improvements to hashing and memory allocation code
- Added an experimental tree rebalancing function. It is compiled out
by default because benchmarks indicated either no improvement or a
slight slowdown compared to an unbalanced tree. To compile it in,
try 'make CFLAGS_EXTRA=-DUSE_TREE_REBALANCE'
- Increased size of string_malloc pages from 64K to 256K since testing
shows a minor performance improvement with large file sets
- Made variable scope and type changes for a tiny performance boost
Changes from 2.0.1 to 2.0.2 [JLB]
- Removed redundant getfilestats() calls for a tiny speed boost
- Added a -D/--debug switch to show a set of statistic counters for
various parts of the fdupes algorithms. Can be used to determine what
fdupes is doing "under the hood" and give insight into why performance
may be slower or behave strangely on your data set. To enable it, use
DEBUG=1 with your make command, i.e. 'make DEBUG=1'
- Performance note: the fdupes algorithm uses a tree data structure and
becomes progressively slower as more files are processed and the tree
depth grows larger. As of this version, a rewrite of the core
algorithm is in progress which will remove this tree structure and
significantly improve performance on most large file sets.
Changes from 2.0 to 2.0.1 [JLB]
- Hard links were treated as identical in match checking but not in
match confirmation. This version fixes the problem, increasing speed
with file sets involving lots of hard links.
- A few minor efficiency improvements were performed
Changes from 1.51-jody5 to 2.0 [JLB]
- Bumped major version to 2.0 due to the number of behavioral changes
and improvements to the code, plus it looks less messy than the
hyphenated versioning
- Increased "chunk size" for better performance and to solve the disk
thrashing problem when checking two large files for a match
- When using the -H option, hard links now automatically match each
other without performing any file reads
- Changed primary memory allocator to string_alloc by Jody Bruchon
to improve performance over generic malloc/calloc
- Progress indicator now lists the number of duplicate pairs found in
addition to the usual file progress count and completion percentage
- Progress is updated more rapidly when full file comparisons happen
so users are less likely to think fdupes is "frozen"
- Floating point code was made optional and is removed by default
- A comparison script was added to check built fdupes behavior against
whatever fdupes is currently installed on the system
- Added "undocumented" -Q / --quick option which is not fully safe but can
be used to drastically reduce run time for large data sets if some risk
of data loss is acceptable to the user
- Added -O/--paramorder option to sort files by the order their parent
directory set was specified on the command line first. This makes it
possible to choose what directories' files get preserved over others
while using -d and -N together
- The file list loading progress indicator was revamped. Rather than a
simple "spinning pipe" indicator, it now shows the number of files and
directories scanned as well as which command line specified set the
scanning is currently happening in
- fdupes was enhanced to support more than 2 million files total by
changing from 'int' internal sizes to the maximum supported by the
platform being compiled for
- Hard link code was modified to be much safer; now any file is only
permanently deleted after a hard link succeeds
- Hard links on Windows (on supporting filesystems) are now supported
- Hashing code was optimized for a benchmarked 8.4% improvement in file
processing overhead (and much more under typical real-world conditions)
- Hard linking checks for more error conditions and output is much
clearer about what action was taken on any given file
Changes from 1.51-jody4-jkl1 to 1.51-jody5 [JLB]
- Less malloc()s so less memory usage and a slight speedup
- Change --order=name to an intelligent numerically correct sort
- Fixed bug where progress text was missing until first update
- Performance boost for small files (4KB or less) by not running
redundant hashes and comparisons
- Test files added for numerically correct sort ordering
Changes from 1.51-jody4 to 1.51-jody4-jkl1 [JKL]
- added `--xsize=SIZE' option: exclude files of size < SIZE
- updated Makefile: `PREFIX = /usr/local'
- updated README: Usage to reflect curent parameters
Changes from 1.51-jody2 to 1.51-jody4 [JLB]
- Add support for hard linking duplicates with -L switch
- Updated jody_hash algorithm with much lower collision rate
- Remove freopen() call that posed a portability problem
- Improved progress indicator behavior
- Many minor bug fixes
Changes from 1.51 to 1.51-jody2 [JLB]
- Switched to C99
- Replaced MD5 with Jody Bruchon's hash function
- Added a delay to progress indications for better performance
- Removed lots of unused code
- Ported fdupes to Microsoft Windows (with MinGW)
Changes from 1.50 to 1.51
- Added support for 64-bit file offsets on 32-bit systems.
- Using tty for interactive input instead of regular stdin. This is to
allow feeding filenames via stdin in future versions of fdupes without
breaking interactive deletion feature.
- Fixed some typos in --help.
- Turned C++ style comments into C style comments.
Changes from 1.40 to 1.50-PR2
- Fixed memory leak. [JB]
- Added "--summarize" option. [AL]
- Added "--recurse:" selective recursion option. [AL]
- Added "--noprompt" option for totally automated deletion of
duplicate files.
- Now sorts duplicates (old to new) for consistent order when
listing or deleteing duplicate files.
- Now tests for early matching of files, which should help speed up
the matching process when large files are involved.
- Added warning whenever a file cannot be deleted. [CHL, AL]
- Fixed bug where some files would not be closed after failure. [AL]
- Fixed bug where confirmmatch() function wouldn't always deal
properly with zero-length files. [AL]
- Fixed bug where progress indicator would not be cleared
when no files were found. [AL]
- Removed experimental red-black tree code (it was slower on
my system than the default code). [AL]
- Modified md5/md5.c to avoid compiler warning. [CHL]
- Changes to fdupes.c for compilation under platforms where
getopt_long is unavailable. [LR, AL]
- Changes to help text for clarity. [AL]
- Various changes and improvements to Makefile. [PB, AL]
Changes from 1.31 to 1.40
- Added option to omit the first file in each group
of matches. [LM, AL]
- Added escaping of filenames containing spaces when
sameline option is specified. [AL]
- Changed version indicator format from "fdupes version X.Y"
to the simpler "fdupes X.Y". [AL]
- Changed ordering of options appearing in the help
text (--help), manpage, and README file. [AL]
Changes from 1.30 to 1.31
- Added interactive option to preserve all files during
delete procedure (something similar was already in
place, but now it's official). [AL]
- Updated delete procedure prompt format. [AL]
- Cosmetic code changes. [AL]
Changes from 1.20 to 1.30
- Added size option to display size of duplicates. [LB, AL]
- Added missing typecast for proper compilation under g++. [LB]
- Better handling of errors occurring during retrieval
of a file's signature. [KK, AL]
- No longer displays an error message when specified
directories contain no files. [AL]
- Added red-black tree structure (experimental compile-time
option, disabled by default). [AL]
Changes from 1.12 to 1.20
- Fixed bug where program would crash when files being
scanned were named pipes or sockets. [FD]
- Fix against security risk resulting from the use of a
temporary file to store md5sum output. [FD, AL]
- Using an external md5sum program is now optional. Started
using L. Peter Deutsh's MD5 library instead. [FD, AL]
- Added hardlinks option to distinguish between hard links
and actual duplicate files. [FD, AL]
- Added noempty option to exclude zero-length files
from consideration [AL]
Changes from 1.11 to 1.12
- Improved handling of extremely long input on preserve
prompt (delete option). [SSD, AL]
Changes from 1.1 to 1.11
- Started checking file sizes before signatures
for better performance. [AB, AL]
- Added fdupes manpage. [AB, AL]
Changes from 1.0 to 1.1
- Added delete option for semi-automatic deletion
of duplicate files. [AL]
=== NOTE: This is archived material from 'fdupes' development ===
=== and from the pre-'jdupes' code work. DO NOT EDIT. ===
The following people have contributed in some way to the development
of fdupes. Please see the CHANGES file for detailed information
on their contributions. Names are listed in alphabetical order.
[AB] Adrian Bridgett (adrian.bridgett@iname.com)
[AL] Adrian Lopez (adrian2@caribe.net)
[CHL] Charles Longeau (chl@tuxfamily.org)
[FD] Frank DENIS, a.k.a.
Jedi/Sector One, a.k.a.
DJ Chrysalis (j@4u.net)
[JB] Jean-Baptiste ()
[JLB] Jody Lee Bruchon (jody@jodybruchon.com)
[JKL] Jan Klabacka (jan.klabacka@gmail.com)
[KK] Kresimir Kukulj (madmax@pc-hrvoje.srce.hr)
[LB] Laurent Bonnaud (Laurent.Bonnaud@iut2.upmf-grenoble.fr)
[LM] Luca Montecchiani (m.luca@iname.com)
[LR] Lukas Ruf (lukas@lpr.ch)
[PB] Peter Bray (Sydney, Australia)
[SSD] Steven S. Dick (ssd@nevets.oau.org)
Introduction
--------------------------------------------------------------------------
jdupes is a program for identifying and taking actions upon duplicate
files. This fork known as 'jdupes' is heavily modified from and improved
over the original. See CHANGES for details.
A WORD OF WARNING: jdupes IS NOT a drop-in compatible replacement for
fdupes! Do not blindly replace fdupes with jdupes in scripts and expect
everything to work the same way. Option availability and meanings differ
between the two programs. For example, the -I switch in jdupes means
"isolate" and blocks intra-argument matching, while in fdupes it means
"immediately delete files during scanning without prompting the user."
Why use jdupes instead of the original fdupes or other forks?
--------------------------------------------------------------------------
The biggest reason is raw speed. In testing on various data sets, jdupes is
over 7 times faster than fdupes-1.51 on average.
jdupes is the only Windows port of fdupes. Most duplicate scanners built on
Linux and other UNIX-like systems do not compile for Windows out-of-the-box
and even if they do, they don't support Unicode and other Windows-specific
quirks and features.
jdupes is generally stable. All releases of jdupes are compared against a
known working reference version of fdupes or jdupes to be certain that
output does not change. You get the benefits of an aggressive development
process without putting your data at increased risk.
Code in jdupes is written with data loss avoidance as the highest priority.
If a choice must be made between being aggressive or careful, the careful
way is always chosen.
jdupes includes features that are not found in fdupes. Examples of
such features include btrfs block-level deduplication and control over
which file is kept when a match set is automatically deleted. jdupes is
not afraid of dropping features of low value; a prime example is the -1
switch which outputs all matches in a set on one line, a feature which was
found to be useless in real-world tests and therefore thrown out.
The downside is that jdupes development is never guaranteed to be bug-free!
If the program eats your dog or sets fire to your lawn, the authors cannot
be held responsible. If you notice a bug, please report it.
While jdupes maintains some degree of compatibility with fdupes from which
it was originally derived, there is no guarantee that it will continue to
maintain such compatibility in the future.
What jdupes is not: a similar (but not identical) file finding tool
--------------------------------------------------------------------------
Please note that jdupes ONLY works on 100% exact matches. It does not have
any sort of "similarity" matching, nor does it know anything about any
specific file formats such as images or sounds. Something as simple as a
change in embedded metadata such as the ID3 tags in an MP3 file or the EXIF
information in a JPEG image will not change the sound or image presented to
the user when opened, but technically it makes the file no longer identical
to the original.
Plenty of excellent tools already exist to "fuzzy match" specific file types
using knowledge of their file formats to help. There are no plans to add
this type of matching to jdupes.
Usage
--------------------------------------------------------------------------
Usage: jdupes [options] DIRECTORY...
-A --nohidden exclude hidden files from consideration
-d --delete prompt user for files to preserve and delete all
others; important: under particular circumstances,
data may be lost when using this option together
with -s or --symlinks, or when specifying a
particular directory more than once; refer to the
documentation for additional information
-f --omitfirst omit the first file in each set of matches
-h --help display this help message
-H --hardlinks treat hard-linked files as duplicate files. Normally
hard links are treated as non-duplicates for safety
-i --reverse reverse (invert) the match sort order
-I --isolate files in the same specified directory won't match
-L --linkhard hard link duplicate files to the first file in
each set of duplicates without prompting the user
-m --summarize summarize dupe information
-n --noempty exclude zero-length files from consideration
-N --noprompt together with --delete, preserve the first file in
each set of duplicates and delete the rest without
prompting the user
-o --order=BY select sort order for output, linking and deleting; by
mtime (BY=time) or filename (BY=name, the default)
-O --paramorder Parameter order is more important than selected -O sort
-p --permissions don't consider files with different owner/group or
permission bits as duplicates
-r --recurse for every directory given follow subdirectories
encountered within
-R --recurse: for each directory given after this option follow
subdirectories encountered within (note the ':' at
the end of the option, manpage for more details)
-s --symlinks follow symlinks
-S --size show size of duplicate files
-q --quiet hide progress indicator
-v --version display jdupes version and license information
-x --xsize=SIZE exclude files of size < SIZE bytes from consideration
--xsize=+SIZE '+' specified before SIZE, exclude size > SIZE
K/M/G size suffixes can be used (case-insensitive)
-Z --softabort If the user aborts (i.e. CTRL-C) act on matches so far
Duplicate files are listed together in groups with each file displayed on a
Separate line. The groups are then separated from each other by blank lines.
When using -d or --delete, care should be taken to insure against
accidental data loss. While no information will be immediately
lost, using this option together with -s or --symlink can lead
to confusing information being presented to the user when prompted
for files to preserve. Specifically, a user could accidentally
preserve a symlink while deleting the file it points to. A similar
problem arises when specifying a particular directory more than
once. All files within that directory will be listed as their own
duplicates, leading to data loss should a user preserve a file
without its "duplicate" (the file itself!).
The -I/--isolate option attempts to block matches that are contained in
the same specified directory parameter on the command line. Due to the
underlying nature of the jdupes algorithm, a lot of matches will be
blocked by this option that probably should not be. This code could use
improvement.
Hard linking status symbols and behavior
--------------------------------------------------------------------------
A set of arrows are used in hard linking to show what action was taken on
each link candidate. These arrows are as follows:
----> File was hard linked to the first file in the duplicate chain
-==-> Already a hard link to the first file in the chain
-//-> Hard linking failed due to an error during the linking process
If your data set has hard linked files and you do not use -L to always
consider them as duplicates, you may still see hard linked files appear
together in match sets. This is caused by a separate file that matches
the hard linked files and is the correct behavior.
Microsoft Windows platform-specific notes
--------------------------------------------------------------------------
The Windows port does not support Unicode, only ANSI file names. This is
because Unicode support on Windows is difficult to add to existing code
without making it very messy or breaking things. Support is eventually
planned for Unicode on Windows.
Windows has a hard limit of 1024 hard links per file. There is no way to
change this. The documentation for CreateHardLink() states: "The maximum
number of hard links that can be created with this function is 1023 per
file. If more than 1023 links are created for a file, an error results."
(The number is actually 1024, but they're ignoring the first file.)
Contact Information
--------------------------------------------------------------------------
For all jdupes inquiries, contact Jody Bruchon <jody@jodybruchon.com>
Please do NOT contact Adrian Lopez about issues with jdupes!
Legal Information
--------------------------------------------------------------------------
jdupes is Copyright (C) 2015-2016 by Jody Bruchon
Derived from the original 'fdupes' (C) 1999-2016 by Adrian Lopez
Includes jody_hash (C) 2015-2016 Jody Bruchon <jody@jodybruchon.com>
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation files
(the "Software"), to deal in the Software without restriction,
including without limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of the Software,
and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
- A bug with -S shows wrong results.
- A bug causes the following behavior:
$ jdupes --symlinks testdir
testdir/with spaces b
testdir/with spaces a
testdir/zero_b
testdir/zero_a
testdir/symlink_two
testdir/twice_one
$ cp testdir/two testdir/two_again
$ jdupes --symlinks testdir
testdir/two_again
testdir/two
testdir/twice_one
testdir/symlink_two
testdir/with spaces b
testdir/with spaces a
testdir/zero_b
testdir/zero_a
** This is not the desired behavior. Likewise: