#AllDup Manual
#AllDup Manual
#AllDup Manual
About AllDup
Product description
Requirements
Important note
Main Window
Source folder
Search Method
Search options
Archive Files
Filter
File filter
Folder filter
Environment variables
File preview
Profiles
Search Results
Options
Internet update
Search Result
Search result
Context Menu
File manager
Shortcuts
Miscellaneous
Command-line
The powerful search engine enables you to find duplicates with a combination of the following criteria: File
Name, File Extension, File Size, File Content, File Dates and File Attributes.
Additionally you can also search for similar file names, similar or almost identical pictures and similar or
almost identical music files. Furthermore, you can find video & audio files with the same or almost same
audio length or search your hard disk for Hard Links.
AllDup Features
Fast search algorithm
Search inside archives with the following files extensions: 7Z, ARJ, CAB, CB7, CBR, CBT, CBZ,
EPUB, GZ, ISO, LZH, LHA, NRG, RAR, TAR, ZIP
The built-in file viewer allows you to preview many different file formats and analyze the content of
the file before deciding what to do with it
For your safety all files that have to be deleted can be moved to the Recycle Bin or a separate
backup folder
Find duplicate files with a combination of the following criteria: file content, file name, file
extension, file dates or file attributes!
Entire folders or individual files can be excluded from the search by masks or size conditions
Find video & music files with the same or almost same audio length
Export the search result to a text file, CSV file or an Excel file
Find and remove duplicate files from your computer, notebook, network drives, flash drives,
memory drives!
AllDup helps you to find, view and remove files that have duplicate content, regardless of name!
Remove duplicate files faster and safer than with any other duplicate file finder!
It can be especially useful if you have large Music, Pictures, Downloads or Documents folders!
Workstation:
Server:
Take care and don't delete files that are a part of the windows system or any application!
Other Software Products
AllSync
Data Backup & Folder Synchronization
AllSync is the professional solution for your data synchronization, imaging, backup,
update, replication and archiving needs under Microsoft Windows.
Info Download
Source Folders
Here you can specify which drives or folders should be searched for duplicate files. You can activate a
drive or folder for the search by selecting the checkbox in front of the path. The checkbox at the column
'Subfolders' enables you to include or exclude all subfolders from the path. The column File protection
enables you to protect all files inside this source folder. No files can be deleted, moved or renamed from
inside this source folder.
Click on the button Source folders at the toolbar to open a menu to add new folders. Right-click an item at
the folder list to replace, edit or remove folders from the list. You also can drag & drop folders from the
windows explorer to the folder list or paste folders from the clipboard to the folder list.
Comparison Method
Compare files from all source folders
Compare only files within the same source and sub folder
Options
Don't search folders (NTFS junction points) that point to the contents of another folder
NTFS junction points are empty folders that point to the contents of another folder. This option enables
you to prevent these folders from being searched and processed. This option is activated by default.
When deactivating this option you should check first that no loops are created by the junction points and
no data items are processed twice.
Find video & audio files on the basis of the audio length
File Name
Finds files with the same name (without extension)
File Extension
Finds files with the same extension
File Size
Finds files with the same size
File Content
Finds files with the same or similar content
Create Date
Finds files with the same create date
File Attributes
Finds files with the same attributes
Ignore timestamp
This option enables you to ignore the timestamp when using the search method create date or last
modified date.
Ignore seconds
This option enables you to ignore the seconds from the timestamp when using the search method create
date or last modified date.
Options
Don't compare hard linked files
This option prevents the content comparison and the checksum creation of files which are hard links of
themselves.
Find video & audio files on the basis of the audio length
This search method enables you to find video and audio files with the same or almost same audio length.
The first audio track is always used to determine the audio length. Video and audio files with the following
file extensions are supported:
Audio: AAC, AC3, AIFF, APE, AU, DTS, FLAC, M4A, MP2, MP3, OGG, WAV, WMA
Video: ASF, AVI, OGV, M2TS, M4V, MKV, MOV, MP4, MPG, WEBM, WMV
File Name
This option enables you to compare only files with the same file name.
File Extension
This option enables you to compare only files with the same file extension.
Search Method 'File Name'
Compare method
Compare all characters of a file name
Compare only characters at the beginning of a file name
This option enables you to compare only a certain amount of characters at the beginning of a file
name and ignore the rest of the name.
SmartMatch
FuzzyMatch
Levenshtein
Ratcliff-Obershelp
MatchDiff
WordMatch
FuzzyPercent
Simil
All algorithms except WordMatch performing different text comparisons and calculations between the two
file names to calculate the perceptual match between them. The algorithm WordMatch splits the file
names into words and calculates the perceptual match by comparing the words. All algorithms are not
case-sensitive, meaning no distinction is made between uppercasing and lowercasing when performing
comparisons.
Match
This is the degree of match that must exist between the file names in order to be offered as duplicates.
Delimiter
Here you can specify the delimiter characters for the algorithm WordMatch to split the file names into
words. For example the file name "aa_bb_cc" will be splitted into the word collection "aa", "bb" and "cc" by
using the delimiter character space (" "). The standard delimiter characters are: space, comma,
underscore, dot, semicolon and minus. You can add or remove any number of delimiter characters.
Examples
The following file name comparison examples give you an overview which algorithm could be suitable for
your search for similar file names.
The duration at the right table column shows the necessary time to perform 10.000 file name
comparisons.
1. Example
File name A: "Wolfgang Amadeus Mozart"
File name B: "Wolfgang_Amadeus_Mozart"
COMPARE METHOD MATCH DURATION
2. Example
File name A: "Wolfgang Amadeus Mozart - 001"
File name B: "Wolfgang Amadeus Mozart - 002"
3. Example
File name A: "001 - Wolfgang Amadeus Mozart"
File name B: "002 - Wolfgang Amadeus Mozart"
4. Example
File name A: "Wolfgang Amadeus Mozart"
File name B: "Wolfgang Amadeus Mozart - BACKUP"
6. Example
File name A: "Wolfgang Amadeus Mozart"
File name B: "Mozart Wolfgang Amadeus"
8. Example
File name A: "Wolfgang Amadeus Mozart"
File name B: "Ludwig van Beethoven"
9. Example
File name A: "Beethoven"
File name B: "nevohteeb"
Comparison Method
Byte by byte
Byte by byte
This comparison method reads the content of both files chunk by chunk and compares the chunks byte by
byte. With a specified percentage match of 100% the comparison of the two files will be stopped if the
chunks are not identical. With a percentage match below 100% the complete content of both files will be
analyzed to calculate the percentage match between them.
Example: With a specified percentage match of 90% all files with a percentage match of 90% or higher
will be classified as duplicates.
Checksum
The checksum method reads the complete file content and creates a checksum of the content for further
comparisons. Using a checksum could be much faster than a byte-by-byte comparison when you compare
a lot of files. Comparing only the stored checksum will avoid to read the complete content of these files
again and again. See Speed Comparison Test.
NOTE: Files with a size of 0 bytes will be automatically excluded from the search.
ID3v1
ID3v2
Lyrics3 v2
APE v2
Info: AllDup also ignores ID3v2 tags at the beginning of the MP3 file that exists twice and ID3v1 tags at
the end of the file that exists twice.
1. The file comparison starts with a block size specified by the start value.
2. AllDup reads the block size from the two files into the memory and compares them.
3. The file comparison will be stopped if the data blocks are not identical.
4. Otherwise the block size will be increased by the increment value until it reaches the maximum
value and continues comparing the files (2).
Find Similar File Names
Comparison Method
The following comparison methods are available:
SmartMatch
FuzzyMatch
Levenshtein
Ratcliff-Obershelp
MatchDiff
WordMatch
FuzzyPercent
Simil
All comparison methods except WordMatch performing different text comparisons and calculations
between the two file names to calculate the perceptual match between them. The comparison method
WordMatch splits the file names into separate words and calculates the perceptual match by comparing
the words. All comparison methods are case-insensitive, meaning no distinction is made between
uppercasing and lowercasing when performing the comparison.
Match
This is the degree of match that must exist between the file names in order to be offered as duplicates.
Delimiter
Here you can specify the delimiter characters for the comparison method WordMatch to split the file
names into words. For example the file name "aa_bb_cc" will be splitted into the word collection "aa", "bb"
and "cc" by using the delimiter character underscore ("_"). The standard delimiter characters are space ("
"), comma (","), underscore ("_"), dot ("."), semicolon (";") and minus ("-"). You can specify any number of
delimiter characters.
Examples
The following file name comparison examples give you an overview which comparison method could be
suitable for your search for similar file names. The duration at the right table column shows the necessary
time to perform 10.000 file name comparisons.
1. Example
File name A: "Wolfgang Amadeus Mozart"
File name B: "Wolfgang_Amadeus_Mozart"
2. Example
File name A: "Wolfgang Amadeus Mozart - 001"
File name B: "Wolfgang Amadeus Mozart - 002"
3. Example
File name A: "001 - Wolfgang Amadeus Mozart"
File name B: "002 - Wolfgang Amadeus Mozart"
4. Example
File name A: "Wolfgang Amadeus Mozart"
File name B: "Wolfgang Amadeus Mozart - BACKUP"
5. Example
File name A: "Wolfgang - Amadeus , Mozart"
File name B: "Mozart_Amadeus;Wolfgang"
6. Example
File name A: "Wolfgang Amadeus Mozart"
File name B: "Mozart Wolfgang Amadeus"
7. Example
File name A: "Wolfgang Amadeus Mozart"
File name B: "Wulfgang Amadues Mazort"
8. Example
File name A: "Wolfgang Amadeus Mozart"
File name B: "Ludwig van Beethoven"
9. Example
File name A: "Beethoven"
File name B: "nevohteeb"
Files without the extension BMP, JPG, JPEG, GIF, PNG and TIF
Pictures with a width or height smaller than the specified compare size
Comparison Method
The comparison methods aHash, bhash, dHash and pHash enables you to find similar or almost identical
pictures by using a percent match lower than 100%. If you want find exactly the same pictures you have to
use a percent match of 100% or the comparison methods MD5/SHA.
aHash
The comparison method aHash (Average Hash) resizes the image to 8x8 or 16x16 pixel. The
image will be converted to grayscale and the average color of the all image pixel will be calculated.
Now the checksum will be created based on whether the pixel's value is greater than the average
color of the image. The checksum can be 64-bit (8x8 pixel) or 256-bit (16x16 pixel).
bHash
The comparison method bHash (Blockhash) resizes the image to 128x128, 256x256 or 512x512
pixel. The image will be divided into a block matrix and the median value of all blocks will be
calculated to create the checksum. The checksum can be 64-bit or 256-bit. The options " Fast" and
"Precise" enables you to influcence the accuracy of the checksum calculation.
dHash
The comparison method dHash (Difference Hash) resizes the image to 8x8 or 16x16 pixel. The
image will be converted to grayscale and the relative change in brightness intensity between two
pixels will be calculated to create the checksum. The checksum can be 64-bit (8x8 pixel) or 256-bit
(16x16 pixel).
pHash
The comparison method pHash (Perceptual Hash) resizes the image to 8x8, 16x16 or 24x24 pixel.
The image will be converted to grayscale and the 64-bit checksum will be calculated based on the
frequency spectrum data and a distance calculation doing a discrete cosine transform (DCT).
MD5, SHA
These comparison methods enables you to find exactly the same pictures. The following
comparison methods are available:
MD5 (128-Bit)
SHA-1 (160-Bit)
SHA-2 (256-Bit)
SHA-2 (384-Bit)
SHA-2 (512-Bit)
Please read Testing different image hash functions for more information about aHash, bHash, dHash and
pHash.
Percentage Match
Here you can specify the minimum percentage matching of two pictures. The calculated percentage
matching between two pictures will be shown at the column Match. The percentage always refers to the
reference picture of a group which will be shown in a different text color.
Picture Area
This option enables you to specify the picture area to be used to create the checksum. The following
options are available:
1. entire picture
When using the options 2, 3, 4 or 5, the size of the picture area can be specified in pixel. For example, a
value of 16 pixel will be using a picture area with a width and height of 16 pixel at the selected corner.
Compare Size
Here you can specify the maximum width and height of the pictures to be compared. A lower compare
size finds more similar pictures and speeds up the comparison time. A higher compare size finds more
identical pictures and less similar pictures and of course needs more time to compare them.
Checksum
The size of the checksum in bits is shown here.
File Name
This option enables you to compare only pictures with the same file name.
File Extension
This option enables you to compare only pictures with the same file extension.
Orientation
This option enables you to compare only pictures with the same orientation (portrait or landscape).
Aspect Ratio
This option enables you to compare only pictures with the same aspect ratio. The calculation of the
aspect ratio is done by the formula "width divided by height". The result of the calculation is
truncated to one decimal place. A picture with 1920x1080 pixel has an aspect ratio of "1.7".
Flipped vertically
Recognition Rate
We have done several tests with the comparison methods to determine the recognition rate with different
image changes. For the test we have created several copies of a JPEG image (2304x3072px, 1.57MB)
and changed the color, size or format of the copies. At all tests the original JPEG image and one altered
copy of the image have been compared. The following table gives you an overview how good the
comparison methods recognize the changes at the image copies. A value of 100% means that the
modified image copy will be seen as a full duplicate of the original image.
Compare
Modification of the image copy aHash dHash pHash
size
Removed 100 pixel from the left edge 8x8 97% 73% 75%
16x16 96% 80% 91%
Removed 100 pixel from the right edge 8x8 100% 89% 97%
16x16 97% 80% 91%
Compare
Modification of the image copy aHash dHash pHash
size
Removed 100 pixel from the top edge 8x8 94% 89% 84%
16x16 97% 88% 97%
Removed 100 pixel from the bottom edge 8x8 100% 95% 100%
16x16 96% 90% 97%
Removed 100 pixel from the left, right, top and bottom edge 8x8 95% 73% 84%
16x16 97% 79% 91%
Note: The test results of the 24x24 compare size will be displayed only if the percent value differs from the
test with the 16x16 compare size.
Find Similar Audio Files
This search method enables you to find similar or almost identical audio files between the file formats
MP3, FLAC, WMA, WAV, OGG, APE, AAC, MKA, M4A, M4R and MP2. The audio stream of each audio
file will be analyzed and a checksum is calculated for further comparison with other audio files.
Files without the extension MP3, FLAC, WMA, WAV, OGG, APE, AAC, MKA, M4A, M4R and MP2
Scan length
Here you can specify how many seconds at the beginning of the audio file should be used to calculate the
checksum. The standard value for this option is 30 seconds.
Percentage Match
Here you can specify the minimum percentage matching of two audio files. The calculated percentage
matching between two audio files will be shown at the column Match. The percentage always refers to the
reference audio file of a group which will be shown in a different text color.
Comparison Method
Here you can specify the comparison method to be used to generate the checksum. The standard value
for this option is 4.
1 - no information available
2 - no information available
5 - no information available
Determine duration, bit rate and sample rate of files with the following extensions
(slower)'
This option causes to determine the duration, bit rate and sample rate of audio files using the PlugIn
'MediaInfo'. With this option deactivated you also will see the duration, bit rate and sample rate at the
search result when clicking on a file or just moving the cursor over the columns 'Duration', 'Bit Rate' or
'Sample Rate'.
Log every x minutes the search statistic of the scan progress info
This option enables you to log the search statistic shown up at the progress info for analytical purposes.
Abort the search operation after a certain amount of duplicates was found
This option enables you to abort the search operation automatically after a certain amount of duplicates
was found.
Remove all groups where the files not exist at all source folders
This option removes all groups where the files not exist at all source folders.
.7z
.arj
.cab
.cb7
.cbr
.cbt
.cbz
.epub
.gz
.iso
.lzh
.lha
.nrg
.rar
.tar
.zip
Splitted archive files with the following file extensions are supported:
.zip.001
.part1.rar
.7z.001
.tar.001
Notes:
Password-protected files from inside an archive and password-protected archive files will be
automatically filtered when using the search method 'File content', 'Similar pictures' or 'Similar
music'
AllDup's file manager can only delete files from inside archives with the extension .zip or .7z.
The search method Similar Pictures is used and the option 'Detect picture modifications' is
deactivated.
The search method File Content is used and a checksum method is selected at the compare
method.
Log password-protected files
This option enables you to log password-protected files from inside an archive and password-protected
archive files.
.7z
.arj
.cab
.cb7
.cbr
.cbt
.cbz
.epub
.gz
.iso
.lzh
.lha
.nrg
.rar
.tar
.zip
Splitted archive files with the following file extensions are supported:
.zip.001
.part1.rar
.7z.001
.tar.001
Notes:
Password-protected files from inside an archive and password-protected archive files will be
automatically filtered when using the search method File Content, Similar Pictures or Similar
music.
AllDup's file manager can only delete files from inside archives with the extension .zip or .7z.
The search method Similar Pictures is used and the option 'Detect picture modifications' is
deactivated.
The search method File Content is used and a checksum method is selected at the compare
method.
Inclusive
Only files matching an activated filter text are included in the search operation. Example: If you specify the
filter text "*.ini" only files with the ".ini" extension are processed during a search operation. All other files
are filtered (= excluded) from the search operation.
Exclusive
All the files matching a filter text are excluded from the search operation. Example: If you specify the filter
text "*.ini", all files with the ".ini" extension are filtered, i.e. excluded from the search operation.
Filter List
The filter list contains all of the filter texts set up for files. Each filter text can be enabled or disabled via its
checkbox. Click on the button File Filters at the toolbar to open a menu to add new filters. Right-click an
item at the filter list to edit or remove a filter.
Filter Texts
Comparisons using the filter text are always done using the file name. When a filter text contains a
backslash (\) comparison is made with the entire path of the current file. Filters are not case-sensitive,
meaning no distinction is made between uppercasing and lowercasing when performing comparisons.
The filter text may contain various wildcards and Windows environment variables. You can specify
multiple file filters using the special character '|' as a separator.
Examples:
*.txt
temp.*
*temp.txt
temp??.*
temp.00?
%USERNAME%.txt
*\Temp\*.tmp
C:\Temp\*.ini
C:\Temp\%Username%.*
*.bmp|*.jpg|*.jpeg|*.png
Options
Ignore wildcards in filter text (treat like normal text)
This causes the characters of wildcards in a filter text to be treated as normal text.
File Properties
Ignore 0-Byte files
This option enables to ignore files with a size of 0 bytes.
Inclusive
Only folders matching an activated filter text are included during a search operation. Example: If you
specify the filter text "Test" only folders named 'Test' are searched during a search operation. All other
folders are filtered (= excluded) from the copy operation.
Exclusive
All the folders matching a filter text are excluded from the search operation. Example: If you specify the
filter text "Test", all folders named 'Test' and their subfolders are filtered, i.e. excluded from the search
operation.
Filter List
The filter list contains all of the filter texts set up for folders. Each filter text can be enabled or disabled for
a copy operation via its checkbox. Click on the button Folder Filters at the toolbar to open a menu to add
new filters. Right-click an item at the filter list to edit or remove a filter.
Filter Texts
Comparisons using a filter text are always done using the folder name. When a filter text contains a
backslash (\) comparison is made with the entire path of the current folder. Filters are not case-sensitive,
meaning no distinction is made between uppercasing and lowercasing when during comparisons. The
filter text may contain various wildcards and Windows environment variables. You can specify multiple
folder filters using the special character '|' as a separator.
Examples:
C:\Windows
C:\Prog*
temp??
C:\users\%USERNAME%
C:\Windows|C:\Users|C:\Temp
Options
Ignore wildcards in filter text (treat like normal text)
This causes the characters of wildcards in a filter text to be treated as normal text.
Syntax:
%Environment Variable%
Example:
The environment variable %USERNAME% is replaced with the user logged on. Example: The folder
"C:\%USERNAME%" becomes "C:\Admin".
File Preview
The Plugins of the file preview enables you to view many file types directly at the search result without
open them with their associated application. The file extensions at a PlugIn list can be selected or
unselected by clicking on the checkbox in front of them. Using the right mouse button at a PlugIn list will
show up a context menu with the following commands:
Add
Adds a new file extension to the PlugIn list.
Remove
Removes a file extension from the PlugIn list.
Select all
Selects all file extensions at the PlugIn list.
Deselect all
Deselects all file extensions at the PlugIn list.
Default values
Clears the list and adds the default file extensions for this PlugIn to the list.
This PlugIn requires the 32-bit version of the freeware VLC Media Player version 3 or higher. The 32-bit
version of the VLC Media Player must be either installed at your windows system, available as a portable
edition on your windows system or manually integrated into AllDup's installation folder.
2. Scroll down to the end of the FTP archive, click on the folder with the latest version number (e.g.
"3.0.6/") and then click on the folder "32bit/".
4. Extract the following files and folders from the ZIP file into the folder of AllDup:
folder: plugins
5. Run AllDup with admin rights once to register the file axvlc.dll at your Windows system
automatically or register the file axvlc.dll at your Windows system manually using the following
command: regsvr32.exe axvlc.dll
Load a profile
Double-click a profile at the list to load it or use the right-click context menu action Load profile. The name
of the loaded profile will be shown at the window title.
Save a profile
Use the right-click context menu action Save profile to save the current search settings into the selected
profile. You have to load a profile and using the file manager to save the settings of the file manager into
the profile.
Rename profile
Use the right-click context menu action Rename profile to rename a profile.
Delete profile
Use the right-click context menu action Delete profile to delete a profile.
Options
Don't import any source folders when loading a profile
This option enables you to prevent the loading of the stored source folders from a profile.
Stored Search Results
Here you will get a list of all stored search results. The list will be empty if no search results are available.
Right-click on a search result inside the list will open a menu to load or remove a search result.
Options
General
Add AllDup to the context menu of the Windows Explorer
This option enables you to start AllDup via the folder context menu of the windows explorer.
Unicode-Text Support
This option enables AllDup to display Unicode characters from languages such as Chinese, Japanese,
Korean or Russian.
User Interface
Here you can specify a color theme for the user interface.
Log
Delete entries from log file when they are more than ...
This option causes entries to be automatically removed from the log file after a specified number of days.
Search Result
External program for comparing the contents of two files:
This option enables you to specify an external file comparison utility to be used with two highlighted files
at the search result. Right-click two highlighted files at the search result and use the command ' Compare
highlighted files'. The two highlighted files from the search result have to be specified via the wildcards %1
and %2. Example: C:\Windiff.exe %1 %2
%1 = "C:\Data\db.txt"
%2 = "X:\Backup\abc.txt"
%3 = C:\Data
%4 = X:\Backup
%5 = db.txt
%6 = abc.txt
Version
Clicking on the Check button causes a check to be conducted to see whether a new version is available.
Clicking on Download causes the new version to be downloaded and installed.
Options
Check for new version when starting program
This option causes a check to automatically be conducted for a new version of AllDup on the AllDup
website every time AllDup is started.
Select files
All files at the search result can be selected or unselected by clicking the checkbox next to the file name.
You also can use the automatic selection commands at the Menu bar or Toolbar.
Holding the CTRL key and clicking with the right mouse button on a file will open a context menu
with several file commands.
Clicking with the middle mouse button on a file will show the Windows explorer context menu for
this file.
A double click on a file name opens the file with the associated application.
A double click on a file path opens the file path with the Windows explorer.
Keyboard Shortcuts
The following keyboard shortcuts are available:
With the space bar you can select or unselect a highlighted file or group.
The DEL key enables you to remove all highlighted groups and files from the search result.
Holding the CTRL key and pressing the DEL key will delete a highlighted file.
You can create your own keyboard shortcuts for all the commands at the menu bar, toolbar or context
menus by using the option Hot Keys at the menu bar.
Menu Bar / Toolbar
File
Columns
Filter
Groups
Mode
Select
Deselect
Remove
File Protection
Toolbars
Options
Hint: The toolbar context menu (right-click) enables you to hide or show buttons.
File
File Manager - Delete, copy, move or rename selected files.
Save as...
Log
Exit AllDup
Columns
Here you can show, hide or resize the columns at the search result.
Show column 'Path'
Filter
The Display Filter enables you to specify which groups should be displayed at the search result. A
combination of the following options is available:
Mode
The Process Mode enables you to specify which groups are affected from the ' Select', 'Deselect' and
'Remove' actions. You can choose one of the following options:
All groups
Expanded groups
Collapsed groups
Highlighted groups
Non-highlighted groups
Select
The following actions enables you to select certain files at the search result:
Select all files
Select all files which are duplicates of files from the folder
Select all files except the first file
Select all files except the file with the shortest name
Select all files except the file with the longest name
Select all files except the file with the shortest path
Select all files except the file with the longest path
Select all files with a Percent Match greater or equal to a specific percent value
Select all files with a Percent Match lower or equal to a specific percent value
Select all files except the picture with the most megapixels
Undo
Enables you to reverse the last select or deselect file action. Please note that the complete undo list will
be reset after a file is removed from the search result!
Deselect
The following actions enables you to deselect certain files at the search result:
Deselect all files which are duplicates of files from the folder
Remove all groups where the files not exist at all source folders
Remove all groups that contain only hard links with the same ID
Recursively remove all files in a specific folder from the search result
File protection
Here you can activate or remove the file protection for each source folder. The file protection enables
that no files will be deleted, moved or renamed inside the source folder.
Toolbars
Here you can show or hide the toolbars.
Toolbar 'Files'
Toolbar 'Filter'
Toolbar 'Groups'
Toolbar 'Search'
Toolbar 'Select'
Choose font
This command enables you to change the font of the search result.
Colors:
File
This context menu will be shown when you right click a file at a group.
Hint: Doing a right click on a file while pressing the CTRL key will show only the submenu with the file
actions.
Select/Deselect file
File actions:
Open
Open the file with the associated application.
Rename file
Filter lists:
Highlighted Files
You can highlight multiple files by pressing the CTRL key and clicking on each file. Click the highlighted
files with the right mouse button to show up this context menu:
Groups
Select/Deselect Group
Open all files from inside the group with external program
File Manager
The file manager enables you to delete, copy, move or rename the selected files from the search result.
Folder
Options
Make sure that the Windows Recycle Bin is activated at partition of your selected files.
Make sure that the size of the Windows Recycle Bin is set to 100%.
Selected files located on another computer will be deleted and not moved to the Windows
Recycle Bin!
5. Rename files
This action enables you to rename all selected files. You can use the following place holder to
rename your files:
PL A C E H OL D E R DESCRIPTION
%MM Month
%DD Day
%HH Hour
%NN Minute
%SS Second
NOTE: The actions 2, 3, 4 and 5 have no effect on selected files inside an archive file (ZIP, RAR...).
Create Subfolder(s)
This option causes a new subfolder to be created into which the files are copied. You can use
placeholders for Date and Time and Windows variables.
Examples:
Text: "Backup of" %USERNAME% "on" yyyy-mm-dd "at" hh_nn_ss
Folder: Backup of USER NAME on 2011-02-26 at 10_15_00
Options
Create Shortcuts
This option enables you to create a shortcut (.lnk) at the place of all files that will be deleted or removed.
The shortcut will be point to the first file that is not selected at the same group; see Example for
Shortcuts.
The Windows file system has a limit of 1024 hard links on a file.
Only files stored on the same volume can be linked together.
AllDup automatically determines a source file with a hard link count lower than 1024 in a group. All
other files in this group will be linked with this source file.
A new source file will be determined as soon as the current source file reaches 1024 hard links.
This allows you to create an unlimited amount of hard links for each group.
Only files with a hard link count of zero can be linked to the source file. The hard link count will be
checked by the value displayed at the column 'Hard links'. You need to activate the search option
Determine the hard link count for each file so that the hard link count for each file will be
determined during the search.
Exit AllDup
Example for Hard Links
A group contain the following files:
1. C:\Work\notes.txt
2. C:\User\readme.txt
3. C:\Backup\notes.txt
4. C:\Mail\new-user-readme.txt
All files inside the group are selected (?). The first file will be automatically used for the source to create
the hard links. All other files (2-4) will be deleted and a hard link to the first file will be created with the
name of the deleted file.
1. C:\Work\notes.txt
2. C:\User\readme.txt
3. C:\Backup\notes.txt
4. C:\Mail\new-user-readme.txt
The first file is not selected and will be used as the target for the shortcuts to be created. All other files in
the group are selected (?). The selected files will be deleted and for each deleted file a shortcut to the first
file will be created with the name of the deleted file.
Hot Keys
Here you can create keyboard shortcuts for all items of the menu bar and tool bar.
Click on a row at the list and press a key to set the shortcut for this item.
Press the DEL key to remove the shortcut of a selected item.
Press CTRL + DEL to delete the shortcuts of all items.
Keys
You can use the following keys to create a shortcut:
CTRL
ALT
SHIFT
A-Z
0-9
F2-F12
Num0-Num9
Backspace
Enter
Insert
Shortcut Examples
CTRL + A
ALT + 1
SHIFT + L
CTRL + ALT + M
F2
Enter
Command Line
Syntax
AllDup.exe -s[1-9,a,b,c] -c[0-5] -h[0-2] -f[0-7] -m[1-100] -l[10-999] -a[1,2,4,5] -n -p:"[profile name]"
"[Folder1]" "[Folder2]" ....
Parameters
-s[1-9,a]
This parameter enables you to specify the search method:
1 = File Name
2 = File Extension
3 = File Size
4 = File Content
7 = File Attributes
8 = Hard Links
9 = Similar Pictures
a = Similar Music
b = Audio Track
The file properties 1 to 7 can be combined together such as "-s124". The search method 8, 9, a, b and c
can only be used alone such as "-s8" or "-sa".
-c[0-5]
This parameter is optionally and can be used in combination with the parameter -s4 to specify the
comparison method for the comparison criteria File Content.
0. Byte by byte
1. MD5 (128-Bit)
2. SHA-1 (160-Bit)
3.
4.
1.
2.
3. SHA-2 (256-Bit)
4. SHA-2 (384-Bit)
5. SHA-2 (512-Bit)
-h[0-8]
This parameter is optionally and can be used in combination with the parameter -s9 to specify the
comparison method of the search method Similar Pictures.
0. aHash
1. bHash
2. dHash
3. pHash
4. MD5 (128-Bit)
5. SHA-1 (160-Bit)
6. SHA-2 (256-Bit)
7. SHA-2 (384-Bit)
8. SHA-2 (512-Bit)
-f[0-7]
This parameter is optionally and can be used in combination with the parameter -sc to specify the
comparison method of the search method Similar File Names.
0. SmartMatch
1. FuzzyMatch
2. Levenshtein
3. Ratcliff
4. MatchDiff
5. WordMatch
6. FuzzyPercent
7. Simil
-m[1-100]
This parameter is optionally and can be used in combination with the parameter -s9 and -sa to specify the
percentage match of the search method Similar Pictures and Similar Music.
-d[0-2]
This parameter is optionally and can be used in combination with the parameter -s9 to specify the
compare size of the search method Similar Pictures.
0. 8x8
1. 16x16
2. 24x24
-l[10-999]
This parameter is optionally and can be used in combination with the parameter -sa to specify the scan
length in seconds of the search method Similar Music.
-a[1,2,4,5]
This parameter is optionally and can be used in combination with the parameter -sa to specify the
comparison method of the search method Similar Music.
-p:"[profile name]"
This parameter is optionally and enables you to load a profile.
-n
With this parameter the specified folders will not added to the source folder list.
This parameter is optionally.
"[Folder]"
This parameter is necessary and enables you to specify the folder(s) to be searched for duplicates. Folder
paths that include space(s) have to be enclosed in double-quotes.
Examples
AllDup.exe -s13 "C:\Work"
Search for duplicates with the same file name and file size inside the folder "C:\Work".
AllDup.exe -s9 -h1 -m90 "C:\Pics"
Search for similar pictures inside the folder "C:\Pics" using the compare method dHash and a percentage
match of 90%.
AllDup.exe "C:\Work"
Search for duplicates inside the folder "C:\Work" with the current program settings.
* No or several characters.
The following characters have to be inserted in square brackets in order to be used for a compare
operation: left square bracket ([), question mark (?), pound sign (#) and asterisk (*). The right square
bracket (]) can't be used in a string of characters being compared. However, it can be indicated outside of
a string as an individual character.
You can also specify a range of characters in charlist by indicating the largest and smallest value of the
range, separated by a hyphen (-). For example, [A-Z] results in a match if the respective character
position in charlist is a letter between A and Z. You can specify a sequence of several ranges in square
brackets without using separators.
A hyphen (-) can appear at the beginning (after an exclamation point, if present) or at the end of
charlist in order to serve as a wildcard for itself. In any other position a hyphen serves to designate
a character range.
When a range of characters is specified, the characters have to be listed in ascending order (from
the lowest to the highest). This means that [A-Z] is a permissible pattern whereas [Z-A] isn't.
Examples:
: Time delimiter
/ Date delimiter
q Quarter (1 – 4)
The files of the scanned folder where stored on an external hard disk connected via USB2 and encrypted
with TrueCrypt. AllDup was installed at a computer running Windows 7 x64 with 8 GB RAM and an i3
CPU.
The log of the first scan using the comparison method Byte by Byte:
The Log of the second scan using the comparison method SHA-1:
14:10:31 - Search: File content SHA-1 (160-Bit)
14:10:32 - File Count: 45442
14:10:32 - Scan: R:\!P2
14:14:32 - Duration : 0:04:01, Progress : 7 %, Current File : 3422 \ 45442, Duplicates Found
14:18:32 - Duration : 0:08:01, Progress : 26 %, Current File : 12073 \ 45442, Duplicates Fou
14:22:32 - Duration : 0:12:01, Progress : 55 %, Current File : 25020 \ 45442, Duplicates Fou
14:26:32 - Duration : 0:16:01, Progress : 66 %, Current File : 30193 \ 45442, Duplicates Fou
14:30:32 - Duration : 0:20:01, Progress : 78 %, Current File : 35462 \ 45442, Duplicates Fou
14:34:12 - Found 32215 duplicates with 22.970.559.143 Bytes in source folder 'R:\!P2'
14:34:12 - Groups: 5.442
14:34:12 - File Comparison Count: 28.370
14:34:12 - Duplicates: 32215 (70%) (21,39 GB)
14:34:12 - Elapsed time: 00:23:41
As expected, the comparison method SHA-1 wins the rally. Comparing only the checksums stored inside
the RAM reduced the file read access and saved a lot of time.