Looking for unique picture/file comparison program

Topic

New Reply

Does anyone know of a file comparison program that will compare two root directories and all subdirectories, comparing ONLY file attributes (optionally the: name, size, timestamps, CRC/MD5, etc.) IRRESPECTIVE of the file’s location.

Background: Basically this is for digital picture backups. When I copy files from my SD cards, I put them into directories based on the activity and date. I have seen programs that do the copying for you, based on date, but these are not viable as I change time zones (and do not always remember to change the time on my camera) and activities can span days. This has not been that hard in the past, but I just came back from a trip with 5000 files.

What I am looking for is a file syncing program that has a split window, where you select the “source” directory on one side and the “destination” directories on the other side. On both sides the files are sorted by name and you can see which files are missing or different (based on the comparison variables specified).

Thanks.

Reply | Quote

Replies

The free program WinMerge does directory comparison. I mostly use it for text file comparison, so I don’t know how flexible the directory comparison features are, i.e., whether you can disregard the subfolder level.

http://winmerge.org/

Reply | Quote

Thanks jscher2000, but I have winmerge and it will not do this type of comparison. Most common compare/syncing programs will not, which is why I decided to ask the community for suggestions.

Reply | Quote

Hope this helps;http://www.wensoftware.com/PictureLibrary/

Reply | Quote

ExamDiff Pro should handle this for you. It has a free trial period. I’m not affiliated with them, but I’ve been using it for a very long time.

Reply | Quote

I also use and highly recommend ExamDiff Pro. However I don’t see that it will compare files without considering the file name as the first comparison item. If the file name is the same, then there are options to consider file attributes, date/time stamps, and CRC. Also there can be only one source and one destination root directory, not multiple on either side. Sub-directories are OK, just not multiple root directories. All comparisons are done in matching directory, or sub-directory, names.

What might be of help in comparisons that ignore the file name and location is something in the duplicate file finder program category. I have used AcuteFinder (http://www.acutefinder.com) for a number of years. The interface is not very good but I have not found anything better for functionality and flexibility and I have looked a number of times. Acutefinder does not have the dual pane functionality but it can move files to a specified location if found to be a duplicate. Duplicate matching is optionality based on the items you listed (name, size, timestamps, CRC/MD5, etc.) and is “IRRESPECTIVE of the file’s location”. You can specify any number of and which directories to consider.

I do not know of anything that will do what I understand from your post that you need. If you do find something, please post it back here. It sounds like a very convenient tool.

Reply | Quote

As my post showed up in today’s email, I thought I better update this thread.

While I try to stay away from any proprietary database programs, I did try Picture Library. It seems that it’s duplicate file finder tool does not traverse sub-directories so was very cumbersome to use for my issue. The problem with the duplicate file finder programs I have tried for this task is that they assume the the source directory (of a comparison) is the master and will not show you if there is a file in the source but not in the destination (i.e. not a duplicate).

I also tried ExamDiff Pro. And while the interface looks good, it did not work. It never considered the same file in the two locations as being the same. I tried many comparisons, toggling every variable in the Directory Comparison options. Even if the files had the same CRC, it still did not consider them the same. I could not find it stated anywhere, but I think it must be comparing the relative locations of the files, as almost all comparison utilities do. And unfortunately, it had the size and timestamp of the file as 1 option and I need them separated (ie. I want to compare sizes, but not timestamps)

To clarify: I noted that Ktal mentioned that he thought I wanted to ignore the file name. That is not true. Let me try to clarify what I am looking for:

I have SD cards full of pictures from a trip (in this case about 5000). After the trip I separate them into multiple directories based on date, them time of day and activity ( e.g. like day1/morning/dive-1).

I normally use a directory comparison/syncing tool to copy/backup files (I currently use, and really like, TreeCompare). However, it expects the directory trees on both sides, that is the sub-directories under the two directories to be compared, to be identical.

So, what I am looking for is a directory comparison program that will:
1. allow you to select 2 root directories
2. recursively search these sub-directories creating file lists
3. compare all the files found based primarily on file name and size
(optionally; binary, CRC, MD5 etc. comparisons are OK)
4. IGNORE the relative location of the file for the comparison

Thanks.

Reply | Quote

SyncToy 2.1 allows different root directories.

To make a file list use; Karen’s Directory printer, from karenware dot com. Lots of options for file info. Partial screen grab from that prog below.

Is that any good for you?

Edit. In fact you don’t need sync toy. Use Karen to list the directories and use, say, Word to compare the 2 lists. Or have I got this completely wrong :confused:

29124-Capture

Reply | Quote

Thanks RCL;

I had SyncToy installed, I must have tried it years ago, and no it would not do what I wanted.

However, Karen’s Directory Printer did allow me to accomplish the task.

To accomplish this task in Unix I would just use find, grep, cut and sort. Save the results in some text files, then compare these files. Directory Printer is basically a GUI to find, grep and cut. However, it does not sort correctly. It sorts each individual sud-directory and concatenates the results instead of finding all the files first then doing 1 big sort at the end. So I used Notepad++ to resort the text files saved from Directory Printer. Then compared the two text files with a GUI file comparator. So, while this is still a 5 step process, it did work.

I am still hoping someone knows of a single program that can do all this, but thanks for the info about Karen’s tools.
And, a thank you to Karen.

Reply | Quote

I too have been looking for a program to find orphan files in different folder structures, I haven’t found one, but I do have a another method.

If you’re running Windows Vista/7 with an NTFS drive, you can use symbolic links with most sync/compare programs (with limits). Symbolic links, aka symlinks, softlinks, are like enhanced shortcuts that the sync/compare program will treat as the original files. Just make a new folder for each root containing symbolic links to the originals, and compare those new folders of links.

To do this I use several programs:
– Link Shell Extension > this adds items to Windows Explorer context menu (right-click, right drag and drop) allowing you to create symbolic links (also hardlinks and junction points, not used here).
– Everything > this search program lets you easily find and select your desired files to make links to.
– A folder sync program; I prefer FreeFileSync, but most any should do. I’ve just tested these other programs and they seem to give the proper results:
SyncBack
Synkron
TreeCompare

Link Shell Extension
http://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html
http://download.cnet.com/Link-Shell-Extension/3000-2248_4-10971924.html

Everything
http://www.voidtools.com/
http://download.cnet.com/Everything/3000-2379_4-10890746.html

FreeFileSync
http://freefilesync.sourceforge.net/
http://download.cnet.com/FreeFileSync/3000-2248_4-75300470.html

The process, in short. Let’s call the root folders Root1 and Root2.
1 – Create two new folders, say, Links1 and Links2.
2 – Create symbolic links from all the files in Root1 into Links1. Do the same for Root2 to Links2.
3 – Run your preferred program, syncing between Links1 and Links2 folders.

The process, in long. Here your root folders are Root1 (unsorted) and Root2 (your sorted files and folders). If all the files in Root1 are in a single folder with no subfolders, likely on an SD card, you can skip steps 1 and 2 for that root since you can use that folder directly for comparison. Otherwise, copy Root1 to the hard-drive as Everything cannot search FAT partitions. I assume you’ll want to back them up anyway.

1 – Create two folders elsewhere on the drive, Links1 and Links2.
These need to be on an NTFS formatted drive. Symbolic links can point to files on a FAT drive, but can only be created on NTFS.

2 – To create symbolic links from all the pictures in the Root1 folder and its subfolders to the Links1 folder:
– Right-click the Root1 folder and choose “Search Everything…” from the menu. Everything will instantly list all subfolders and files throughout Root1.
– Highlight the desired files (here you can type .jpg into the search filter, click within the search results, then click edit > select all).
– Right-drag and drop the selected files to Links1, choosing “Drop Here… > Symbolic Link” in the menu. The links will have the file’s icons with green shortcut arrows. Note: having files with the same name from different subfolders is not a problem as the links will automatically be renamed as needed; if/when a sync program copies the image files, the copies will have the new name, your original files will remain unaffected.
– Repeat for Root2 to Links2. These two folders now have links to all your root images.

3- Run your preferred sync program, comparing between Links1 and Links2 folders.
Running FreeFileSync, in the compare settings, set “symbolic link handling” to “follow” (most other programs do not have a setting for this, but will work as normal). With Links1 as the source and Links2 as the destination, compare and sync using the update setting. This copies all orphan files within Root1 to the Links2 folder – where they’re easy to pick out from the symbolic links which are shown in Windows Explorer with shortcut arrows and a size of zero bytes. You can then sort them into your Root2 subfolders. Other programs will work similarly.
However, there is a quirk. It doesn’t work for cleaning out duplicate or obsolete files. While the programs can copy the original files, any delete operation will delete the symbolic link, not the original linked file (maybe a good thing?). To most programs other than Windows Explorer, symbolic links are transparent, as if the program is seeing the original file there; but some I’ve used some like XNView that don’t work correctly with them. (I use this flaw, setting XNView to only show image files, filtering out the symbolic links.) There also can be problems in how programs handle the files dates, so I usually set them to ignore those.

There’s a lot more variations on this to do with symbolic and hardlinks, I hope this helps.

Reply | Quote

You might try FolderMatch. It’s basic function of viewing the contents of folders and their files differences may not be what you want since it does compare on a folder-by-folder basis. But its duplicates function allows you to specify multiple root folders (i.e., more than 2) and will find duplicates without regard to folder structure. You can even search without regard to file name, if size and contents works. It is shareware.

Reply | Quote

The picture comparer is engaged in search and removal of duplicate files. One of the most widespread types of files which Clone Remover works with is image. The photo comparer (i.e. Clone Remover) does image comparison and picture comparison after which «makes» decision if it’s possible to consider images to be identical (i.e. duplicate files) after it compares images.

Reply | Quote

I have exactly the same problem: I have several computers, backup- and thumb-drives, and want to make sure before I delete files off a drive or merge them, that I’m not deleting anything unique or more current. I’d looked for a ready-made solution, couldn’t find one, and so am developing a tool (open source) to do this.

You specify two folders. It then analyzes the attributes of all files across the folders (including all subdirectories), looking for duplicates anywhere. It then shows directory statistics and a file-by-file comparison of the attributes (name, date, size for now) of each.

Software is at an alpha stage, but has still proven to be very useful to me for the above purpose.

If this thread’s still alive and there’s interest, I can post more info and release it to sourceforge.

–Rich

Reply | Quote

If this thread’s still alive and there’s interest, I can post more info and release it to sourceforge.

–Rich

G’day Rich,
I’d love to have access to your solution!

John

Reply | Quote

Please!

Reply | Quote

I back up my photos in 3 separate locations. Sometimers I forget to save changes I made in one location to the other 2. I’m looking for a program to compare 2 folderrs or drives to make sure they are identical.
Have you ever finished writing your program Rich?

Reply | Quote

I back up my photos in 3 separate locations. Sometimers I forget to save changes I made in one location to the other 2. I’m looking for a program to compare 2 folderrs or drives to make sure they are identical.
Have you ever finished writing your program Rich? (Knoppie)

Reply | Quote

If this thread’s still alive and there’s interest, I can post more info and release it to sourceforge.

–Rich

Have you ever comnpleted this Rich? I’m looking for a program that will compare my original photos folder with my backup photo folder to make sure they are identical.
Thanks,
Will

Reply | Quote

Thanks for the interest, John. I’ll release a version to SF as a portable executable and post a link on this thread. It’ll probably be a couple-few weeks.

Rich

Reply | Quote

There are times when I want to compare two folders with my own specs for my own reasons. What I do is open explorer to the directory1 that I am interested in. Next I right click that directory. This opens a context menu. Scroll down the context menu to; “Open in new window”. Left click this. I now have 2 explorer windows open. Scroll Explorer2 to whatever directory I am interested in comparing Explorer1 to. Sub directories are available to both Explorer windows in conventional options. The columns of each explorer can be individually displayed giving any/all options available to any single Explorer window.
Each Explorer can be sorted separately or synchronously. Each can be searched separately. All files can be dragged, copied, deleted between explorers1&2 as with any single window.
I have dual monitors which makes this easy to facilitate. It can also be done on a single monitor, but not as nicely. It is only useful for tedious manual organizing.
Perhaps this might help.
Michael

Reply | Quote

This is what I currently do to make sure my backups are identical to my original photos. It is tremendously labor intensive and time-comsuming. I have over 100 GB of files, so it would be great to find a program that would do that.

Reply | Quote

There are times when I want to compare two folders with my own specs for my own reasons. What I do is open explorer to the directory1 that I am interested in. Next I right click that directory. This opens a context menu. Scroll down the context menu to; “Open in new window”. Left click this. I now have 2 explorer windows open. Scroll Explorer2 to whatever directory I am interested in comparing Explorer1 to. Sub directories are available to both Explorer windows in conventional options. The columns of each explorer can be individually displayed giving any/all options available to any single Explorer window.
Each Explorer can be sorted separately or synchronously. Each can be searched separately. All files can be dragged, copied, deleted between explorers1&2 as with any single window.
I have dual monitors which makes this easy to facilitate. It can also be done on a single monitor, but not as nicely. It is only useful for tedious manual organizing.
Perhaps this might help.
Michael

This is what I currently do to make sure my backups are identical to my original photos. It is tremendously labor intensive and time-comsuming. I have over 100 GB of files, so it would be great to find a program that would do that.

Reply | Quote

Ok, sorry for the wait. I’ll publish what I have.

Reply | Quote

Update: The code is complete and I’m working on some basic documentation. I should have links to executables by the end of this week.

Reply | Quote

OK, I just released the program– Thank you for your patience.

Project Page (including download and screenshots): http://sourceforge.net/p/dupstat

Documentation Page: http://richk.net/wiki/DupStat. The documentation is pretty basic right now- if you run into any problems understanding things or working the program, please let me know.

I’d appreciate any feedback or questions; please post a reply to this forum or send me a message.

Cheers,
Rich

Reply | Quote

First – let me say thank you for taking the time to write DupStat! It does exactly what I need.

As a photographer, I archive my images by year and month on a server. I have a second “work in progress” archive on my editing PC. Most times, I forget if I’ve uploaded my work-in-progress images. DupStat quickly shows me files in my local archive THAT ARE NOT in the server archive. I really like the stats in the upper window. At one quick glance, I can see if I have any missing images in each month’s server directory.

DupStat is very fast checking files across my local network. With a Win7 laptop (1.6 GHz dual processor), an older XP server (1.3 GHz single processor) and 100Mb network, comparing 1500 files takes just under 20 seconds. Great job!

Thanks again

Reply | Quote

Just like every other greedy user, now that I love DupStat, I would like to ask for an enhancement! (I retired from a career where part of my job was to write (and later, manage) software – so I know what it is like for a developer to hear my first sentence – Sorry.)

However, I would like to request two more radio buttons on the “Show” tab of the lower pane:
1. Show Non-matches only (show files residing in the left directory only)
2. Show Changed only (show files changed from the original only)

I know that you can group these selections together by sorting the columns, however, this is not what I want to do.

What I want is to isolate just the Non-match (or Changed) files on the left side of the lower pane – no other files on the left side. Then when I double-click on a single file – Windows (tested on Win7) will transfer the entire set of files (listed on the left side) to my default program that opens the file type I clicked on.

As a photographer dealing with images, this is the answer to: “OK, now I have a text list of images on my PC but not stored in my archive, what do I do now?” I need to take a look at these images and if needed copy them to the archive.

If you default program opening images is like mine – Faststone’s Image Viewer (FIViewer)- then simply double-clicking an image on the left side of DupStat will send the entire set of images on the left side of DupStat into FIViewer. Then I can see all the Non-Matched images in FIViewer and use FIViewer to copy selected images to the archive.

The key is isolating the files I want to see on the left side of DupStat – so that is why I am asking for the two additional buttons.

Thanks again for DupStat!

PS – Double right-clicking on the right side does the same thing for viewing images in the right-hand directory.

Reply | Quote

Hi d70 and thanks much for the feedback, I’m glad you’re finding the program useful. I understand your first request and that should be straightforward to implement. For your second request, I presume when you say “changed”, you mean files that also exist on the right but that don’t pass the match test. If that’s not what you mean, please let me know.

While these changes should be straightforward, I’ll need to find time to do the work. That should be within the next couple weeks. I’ll post here again when I’ve made the update.

Rich

Reply | Quote

Hi Rich –

Thanks for the quick response.

Yes, you are correct about my meaning of “changed”.

I hope others in this thread provided you will positive feedback as well – DupStat is very handy tool and I am surprised that more people did not reply after you posted your work.

Reply | Quote

OK, I’ve updated DupStat. Per D70’s request, it now has the options, in the lower (file-compare) part of the window, to show only unique files (on left, not on right) or to show only files that have changed (i.e., are on both sides but are not matches).

Another feature I added in this release is the ability to (optionally) show individual files in the upper (summary) part of the window. When you click on such a file, it shows all matching and “near-matching” files in the bottom part. Near-matching means at least one attribute (time, date, or size) matches. I find this helps ensure I don’t miss things like a rotated photo when filesize is a criterion).

As before, the project page (including download and screenshots) is at http://sourceforge.net/p/dupstat, with documentation at http://richk.net/wiki/DupStat.

Cheers,
Rich

Reply | Quote

@knoppie – thanks for the update! I really appreciate it and I think others will find this a great tool.

I’ll take 0.3 for a test spin in the next few days and provide some feedback. A minor thing I noticed when I unpacked the download from SourceForge is that the executable is named “dupfinder.exe” instead of DupStat. No big deal, but it might confuse new users.

Thanks again for a fast, light, portable tool for helping keep my picture archive well organized.

Bob

Reply | Quote

@D70, appreciate the feedback. I corrected the executable name in the zip file, thanks for the catch. Appreciate any other feedback.

Rich

Reply | Quote

I am also somebody who would like the feature of seeing only unique files.
Unfortunately Chrome classifies this file as malicious, will harm my computer and does not allow download.
Anyway to circumvent this?

Reply | Quote

@julo, thanks for the heads up. I researched this and apparently chrome uses a guilty-until-proven-innocent approach to malware detection. It seems to report a lot of false positives- see the comments on google’s own blog.

All I can suggest is to either temporarily turn off safe browsing in chrome before you download, or use a different browser. If anyone is still concerned, I will post an md5sum of my own copy of the download zip file, which you can use to confirm it at least hasn’t been tampered with. I should be able to do this within the next couple days.

Hope this helps.
Rich

Reply | Quote

Thanks for the tip. I used Firefox and got the file without problems.

The reason that I spoke up was that I had no problem with v0.2 and version 0.3 is only 3/4 the size so I figured you must have done something rather major.

I will speak up again after I had some time to try it out. I had some issues with 0.2 but will wait if they have been addressed in 0.3.

Keep up the good work.

Reply | Quote

Rich – version 0.3 does exactly what I needed. Everything works fine. I’ve already used DupStat to pull more than 2.5GB out of my archive.

Thanks for implementing the changes.

Bob

Reply | Quote

I must be missing something.
Lets say I have a folder of files some of which may be in sub/sub-folders. Take 1-current as example (on the right side).
I obtain/have another folder, say 2-backup-original, which has some unique some of the same named files, but are potentially filed differently (some in root, some in different named sub-folders) on the left side.
I like to find those files which are unique to the second folder (only in 2-backup-original), no matter the tree, so I can move/copy them to the first folder.
Matching by Name I do not see the ‘unique’ button doing that.
Where do I go wrong?

Reply | Quote

@Julo,

To make sure I understand your example, you are looking for files in the “2-backup-original” folder that don’t exist anywhere in “1-current” directory structure (e.g., in the test folder, the file akbar.jpg in the aliens subdirectory)

Given that, you can search for unique option in left side folders only. Try swapping your left and right directories, rerunning the search, then selecting “Left unique” option for the lower pane. In the upper pane, folders with unique files will show up on the left as green, yellow, or gray. You can click one of those folders and get a file-by-file comparison in the lower pane.

Alternatively, if you have the “Tree includes files” box checked in the upper pane, the unique files show up there with a gray icon. You can confirm they are unique by clicking on those files and checking the lower pane.

I hope this helps.
Rich

Reply | Quote

You understood correctly.
I have 2-backup-original in the left column and 1-current in the right column, which I assume is the correct way. I also tried the other way but there were no results.

Selecting Left-unique I get 2 files (semantic and fleas) in the left bottom pane. Clicking on a sub-folder, say aliens in the top pane, I get all eight files. Unique or not.

The root folder appears as expected.

What I would like is a way to see only ALL unique files listed at once and in addition those unique files which are in a folder by selecting a folder in the top pane.

Reply | Quote

Gotcha. The way the lower pane currently works is that it shows a direct comparison between the selected folders, not including subdirectories. When you select “Left-Unique”, you’re only seeing uniqueness against whatever folder you selected on the right. That’s why when you click on the “Aliens” subfolder as you mention above, all the files on the left are unique compared directly against what you selected on the right.

What I think you’re asking for is a) to have the option to do a complementary (unique instead of match) search, i.e., tally statistics in the upper pane of files on the left that don’t have any matches on the right and b) to have the option for the bottom pane to show matches (or unique files) in all subdirectories in addition to the current directory (to show “All unique files listed at once”).

That makes sense, and would more directly address what many people might be looking for. Let me think about those and I’ll respond in this forum about how to do that.

Reply | Quote

Hi, just a quick status. I’m making my way through the updates above. There will be a option to show files from all subdirectories in the bottom pane. In that case, it will just list matching files; there won’t be any directory-to-directory comparison in this option.

There will also be an option to search for files with no matches (unique files).

I expect to have this version done in 2-3 weeks. I’ll post here when it’s published.

Reply | Quote

Auslogics Duplicate File Finder
it’s free

Reply | Quote

Auslogics Duplicate File Finder
it’s free

There are many, many programs which will find duplicate files.
However free program(s) which find unique files in one folder compared to another folder I have yet to find.

Reply | Quote

Sorry in advance for the long post.
Very nice comparison program!! I also appreciate how inclusive and fast it scanned through everything (30,000 files from one drive, 28,000 files from the first backup, most of them are the same, and my third drive the main backup only showed 15,000 files… I’m behind in updating it). I really need to find a good mirroring program.

I have over 30,000 pictures from about 10 digital cameras (since 2001) and more recently about 5 cell phones that I have been trying to backup regularly each under their own folders which sometimes has the event name(s) in the filename, but I started losing track of where I left off. My temporary solution was to create a folder named “All backed up 9-10-10” which was supposed to include all the files prior to that date, then any newer ones were under their respective camera folder, then under the relevant date as to when they were dumped from the camera’s memory card to the computer. Sometimes I have multiple cameras at the same event, so they are from the same day, but they would be saved in each camera’s folder but under the same date and sometimes the event name(s) (if I dump them at the same time). But that also created confusion. I keep them separate so I don’t have one giant folder with over 30,000 files.

My problem was that on one of the storage drives, I did not change the folder structure, but another I did as noted above. And the third, external main backup yet another structure. Basically I’m going for triple redundancy backup, but this created duplicates in multiple folders in some cases. This program will eventually help me save much time. I had looked for such a program a few years ago, and no such luck. I am glad I just came across this thread.

I might just have to create a new folder for everything that I have checked, then eventually copy/paste that across all three drives and just keep it updated with a sync program.

I do like that you can double click on a file or folder and it opens the item instantly. I also especially liked being able to Finding All Matches for an Individual File.

The only suggestions I have for how I would like to use it are (without enough thought into what I am requesting, so I apologize if they don’t make sense):
1) A quick Swap Button (or a Right Unique button) still trying to understand if this would be benefit
2) Thumbnails or separate panes for pictures to easily, quickly compare the content side-by-side.
3) bit by bit comparison for pictures (not sure if this would be helpful or not)
4) The capability to delete duplicates or to sync directories (like drag-n-drop, but could cause problems…)
5) For the column resize to remain at a width that I choose rather than resizing constantly
6) A “Quick Update” button, for if I double-click to open a folder, then delete or add something to one of the folders, so I don’t have to click “Go” again

Reply | Quote

@MagicTK, Thanks much for the comments. I’ll respond with some suggestions and plan some more app updates.

I’ve just released DupStat v0.4, which includes Julo’s suggestions from 2014-12-05 and several other refinements. You can now search for and highlight unique files via option at the top of DupStat’s window and show all files in the bottom pane by selecting the ‘Include Subdirs’ option above the right hand tree.

Again, appreciate any comments or additional suggestions.

Reply | Quote

@MagicTK, here are my thoughts on your points.

1) A quick Swap Button (or a Right Unique button)
I like that idea. I’ll also add a copy button to copy the left folder to the right side (e.g., to look for dups in a single directory).

2) Thumbnails or separate panes for pictures to easily, quickly compare the content side-by-side.
That’s something I’ve been looking for myself. There are a couple freeware image viewers that have a side-by-side comparison feature (xnview and fsview). What I’d like to be able to do is launch a comparison of 2 files from the bottom pane. Neither of those tools have command line options, which is what would be needed.

So I’ll explore creating a simple image comparison tool, then hook it up to DupStat. The tool would allow you to load images side-by-side, then zoom and pan them both synchronously. That would be a longer term project, but I agree it would be useful.

If anybody is familiar with a command-line able image comparison tool, please respond; that would be much easier than creating a new app.

3) bit by bit comparison for pictures

if you’re talking about being able to zoom in to view pixels, then that would be an option with the above viewer.

4) The capability to delete duplicates or to sync directories (like drag-n-drop, but could cause problems…)
I’ll add an option to “delete highlighted files”, “delete selected files”, etc. This action will send the file(s) to the recycle bin.

5) For the column resize to remain at a width that I choose rather than resizing constantly
Under what conditions are you seeing this? If it’s when you click the “Include Subdirs” option or quit the app then relaunch it, I can fix that. If it’s anything other than that, please let me know.

6) A “Quick Update” button, for if I double-click to open a folder, then delete or add something to one of the folders, so I don’t have to click “Go” again
Good idea, though that will take some thought.

If I didn’t understand any of the items above, let me know. I’ll work on items 1, 4, and 5, and should be able to incorporate perhaps over the next month. The others will be a longer term task.

Cheers,
Rich

Reply | Quote

Rich,

It looks like you understood my requests. Thank you for replying and working on these suggestions. I sent you an e-mail to the address shown in your http://www.richk.net/wiki/DupStat/DupStat link. If you don’t receive it, I can try again.

Here’s what I found a few days ago for command line comparison tool:

http://www.imagemagick.org/

Specifically –> http://www.imagemagick.org/script/compare.php

Also not free, but compares visual content http://www.bolidesoft.com/imagecomparer.html It almost did what I wanted, but your program seems to be much faster at finding matches or uniques. I really like your idea about he synchronous zooming too.

Reply | Quote

Lately, whenever I want to find duplicate images, I have been using http://www.duplicate-finder.com/photo.html (very fast) and http://www.joerg-rosenthal.com/en/antitwin/ (slower but allows image byte comparison.)

I have quite a few more image comparers, all portable, but do not use them too often. A google search should bring them up.

Personally, I would rather see a smaller DupStat which does only file-(name/date/size) comparison but does that well. I am especially interested in finding unique files when comparing folders.
I have to play some with v0.4 before I will comment, but I can say already that using it intuitively needs more work. One thing, for instance, I hope for a way to show only unique files. Right now they appear to be hidden among lines of folders?

Eventually I like, once only files are listed, the capability to copy them into another folder from which I then can distribute them later.
This would come handy if the list of unique files is long and could not be managed in one setting. Along the same line some statistics would help. If DupStat would say there are 5 unique files I may manage them differently than if there are 100 files.

I also like right unique better, for me that would be more intuitive.

Reply | Quote

@julo: First, thanks for the pointers on existing image diff tools; my preference would be to not reinvent that. It would be ideal to be able to launch such a tool from within DupStat, but that such tools exist means not having to reinvent it.

Next, I think v0.4 makes your use case work. First, with the ‘Find: Unique’ and ‘Include Subdirs’ options selected, the statistics against the topmost folder will show you the number of unique files for that directory and all subdirectories.

Then, in the bottom pane options, selecting ‘Show: Only Matches’ will show you a list of all unique files highlighted in yellow, along with any near matches (files that have one or more matching attributes but don’t pass the match criteria). The near-matches are to confirm whether a file is actually unique vs. just a rotated version of an image or renamed or with a slightly different timestamp.

To get to a pure ‘Just show the unique files at the bottom’, I could optionally remove the nearly matched files in the bottom pane. If showing near matches turns out to be distracting, I may do that.

Finally, copying unique files: for the next version, I’m adding actions. An easy action would be to copy matching files (in the case above, all the unique files) to a user-selected folder.

If this doesn’t address your use case, let me know.

Reply | Quote

I did not realize until too late that “Only Matches” apparently does not have the same meaning on top and bottom. On top it refers to the folders, while on bottom it refers to the settings. Do I have that right?

I do not understand ‘near matches’. Any tests I have been doing so far had as Criteria ‘Name only’.

And yes I would like to be able to see ‘unique’ files only.

Reply | Quote

I’ve just released DupStat v0.5, which includes several more updates and refinements, including an Action menu with copy/move/delete capability, drag-and-drop to specify search directories on either side, directory swap and copy buttons, file counts displayed in the file pane, tooltips for most UI controls and options, better control over what you see in the bottom pane. Several updates and refinements are also included; see the change history section of the Readme file.

Again, appreciate any comments or additional suggestions.

You can get the updated app at http://sourceforge.net/p/dupstat

Cheers, Rich

Reply | Quote

Rich,

I’ve only played with it for about 30 minutes, and I like it!! Thanks so much for putting the time into adding these things. I am amazed at how fast this program operates.

I just thought of another suggestion, although minor. I know I will eventually memorize the color codes of the Directory Icons and File Icons, but it would be nice to have those in a small key up at the top somewhere for quick reference like right next to the DupStat V0.5 name or something.

I love the Search for Uniques and Dups at the top. I love the Left Hits and Misses and Right Matches and Misses. I still need to take more time to process in my mind exactly what each combination means and shows. Like if you click Uniques then Left Hits or Misses and Right Matches and Misses. Too tired to make sense of it, but I can see it doing something, and that’s what I was hoping for.

I do like the Copy, Move, Delete under the Actions button, especially after seeing if there are uniques, and double clicking them to quickly verify what they are, then deciding if I want to copy everything or just the selected. I did notice that after I delete a file, I need to hit Go again to refresh, can it auto-refresh showing it has been deleted without hitting Go again?

And thanks for fixing the column width when changing from various modes.

It does look like I will have to go through individual folders and clean up and match, copy, delete first, then select higher up in the directories and compare overall for Uniques and Dups in multiple other folders.

I’ve already been mentioning this program to others, but I need to ramp it up because this is an awesome program and deserves lots of exposure.

P.S. thumbnail or small window comparison showing images side by side would be awesome, although, I haven’t put enough thought into each usage situation and what makes sense, but I think you understand the intent.

Tom

Reply | Quote

Tom, thanks for the feedback and the suggestions. I’ll think about a quick reference for the file/folder colors and refreshing the lower pane after moving or deleting files is a good idea.

Also, as you indicated, I would like to simplify the descriptions around the checkboxes in the lower pane. That will require some more thinking too.

Rich

Reply | Quote