Freeware Spotlight — Anti-Twin

Topic

New Reply

ISSUE 18.32 • 2021-08-23 BEST UTILITIES By Deanna McElveen Yes, those seashell pictures you took from the ocean were great, and that recipe for Frito
[See the full post at: Freeware Spotlight — Anti-Twin]

5 users thanked author for this post.

lmacri, abbodi86, KWGuy, Ascaris, doriel

Reply | Quote

Replies

Regarding duplicate files :
I have a lot of big (GBs) of duplicate files, mainly installed software, one for me the user and under ‘all users’.
I am the only user on my PC.
Can I delete all instances for ‘all users’ ?

Example :

Reply | Quote

Interesting product, and I favour the configurations.

I note the note about shortcut links towards the end , reminiscent of Sun Microsystem’s triple-tier data storage system.

I have spent some time looking at content matching, and am currently at the stage of optimizing the “binary compare” mechanism. This gets tricky with specific file types. MP3 content can be broken into chunks by the meta-data tags, but I hadn’t started on image files so “pixels” is a good hit.

Back to links: Vitally important and a principle cause of failure of any system where deleting duplicates can leave hard-coded links (say, to an image file) hanging, basically destroying a word-processing document which wakes up to find all its hard-coded image files gone!

A great deal hinges on an individual user’s definition of the term “DUPLICATE”.

One version of “Jingle Bells” in my T:\Music\ folder is enough, but I will never have enough versions of “Charles-Marie Widor Toccata from Symphony No. 5 for Organ & Orchestra”; preferably at amplified volume! For me, any version of Jingle Bells is a duplicate to be tossed out of the nest, but Widor? No!

Cheers

Chris

Unless you're in a hurry, just wait.

Reply | Quote

I’ve been using the Linux utility FSLint (file system lint) to de-duplicate files, but the devs of that program have recommended Czkawka, a reimplementation of the FSLint (which is written in Python) in Rust, with additional features (including picture deduplication when there is no checksum match). It has Windows, Linux, and Mac versions, and is free and open source.

Dell XPS 13/9310, i5-1135G7/16GB, KDE Neon
XPG Xenia 15, i7-9750H/16GB & GTX1660ti, KDE Neon
Acer Swift Go 14, i5-1335U/16GB, KDE Neon

Reply | Quote

fslint isn’t present in the ubuntu 20.04 LTS repos as it uses the depreciated Python 2.6 and some now, obsolete dependancies 🙁
Have used it in the past on 18.04 LTS and previous to skim off excess and duplication to good effect. Hope the author updates it to conform.

Keeping IT Lean, Clean and Mean!

Reply | Quote

I need and want those duplicates.   It is easier to find the file when I need it when it is filed under alternate categories/topics.  And it is safer in case one of them gets corrupted.

YMMV

 

What  windoze needs is a better way to tag files for finding them so I dont have to add tags into the name of the file when I save it.

Reply | Quote

It does – two ways. When you save a new file, there’s an option under the name “more options”. this allows you to enter a tag (amongst other things). You can save as an existing file and save the tagged file over the old.

The second is File Explorer. Right click on the file name and right click. Choose properties and click the tag line. You can enter a tag this way. You can also select a number of files and tag them all.

To find files with a particular tag, type tag:string and it will find all files with a tag with that string.

Note that only files with a tag option can be tagged e.g. all office files. If there’s no tag option in properties, this can’t be done. I’ve only just discovered this so they may be other options – searching with other keywords is possible in File Explorer

Eliminate spare time: start programming PowerShell

Reply | Quote

Thank you for a well written, and informative article.

Over the years I have  used Anti-Twin to help control my obsession for backing up my photos.  I have found it to be easy to use and very accurate.

There is one peculiarity, I have accepted, but do not understand.  Occasionally I will run the program looking for 100% duplicates in a directory, and get zero records found.  But when I rerun against the same directory looking for less than 100%, the results show files with 100%.

Do you have any suggestions for a similar program, which works with files in the cloud?

Dave. v.

Reply | Quote

My guess is that it’s rounding error.

If you search for 100% duplicates, it could be looking for the highest level of similarity it is capable of detecting, which it defines as 100%. The photos it finds in a lower match level search, and reports as 100%, might be just below that threshold, at say 99.5%, which it rounds up to 100% for the display, but is still below the threshold when you set it to 100%. Just a guess; I have never used the program, so I don’t have any direct experience.

Dell XPS 13/9310, i5-1135G7/16GB, KDE Neon
XPG Xenia 15, i7-9750H/16GB & GTX1660ti, KDE Neon
Acer Swift Go 14, i5-1335U/16GB, KDE Neon

Reply | Quote

There are many utilities to find duplicate files. But, for some unexplainable reasons I can’t find a utility to find the ‘comlementary’ of duplicate files or call it the list of all mutually exclusive files between two (and ideally more) paths, A and B. To put it in other words if not clear, list all files that exist in path A and NOT in path B and vice versa.
(Notes: I am surprise because it takes adding/changing few lines of code of a ‘find duplicates’ utility to accomplish its complementary).

As usual, there should be filters for at least file types, both for included and excluded files.

Also as usual, comparison method options should be: 1. File’s name+time-stamp(of last change)+size, 2. File’s MD5, 3. File’s SHA.

I don’t mind to pay for a good professional application.

Thanks,
David

Reply | Quote

ExamDiff Pro will show either matching or unmatched (you choose which display you want). I think (haven’t used that feature in several years) that it can also do a 3-way comparison (3 different directories), but I’m not sure about that.

Reply | Quote

I’ve just taken Anti-Twin for a spin, and it’s amazing. I have had two iPhones, separated by a brief return to Android. Using iCloud, the older one placed photos on my PC in Pictures\iCloud\Downloads, whilst the newer one separates them by year Pictures\iCloud\Downloads\2019. Where a picture had been transferred to the Android phone and back to the new iPhone, I ended up with a copy in both locations. In addition, when I got O365, I started using my 1TB OneDrive to back up my photos, so they also appeared in Camera Uploads. Then I would gradually move them from there to named folders under My Photo Library. (See the attached photo if this doesn’t make sense)

This tool has made it a doddle to trim the excess fat. Its next challenge will be to help reduce the multiple copies of files due to my keeping backups of a sequence of computers on my NAS, and potentially multiple File History copies as I have sorted my photos and other files.

Happy Days!

Steven

Reply | Quote