Finding/removing duplicate files

Topic

New Reply

Anyone using Rmlint (with or w/o gui) or Fdupes to clean up file system?  Both appear in Software Mgr in this Mint Mate 20.3 system.

Reply | Quote

Replies

I’ve never used either one. If you’re reluctant to try one of the programs and you think the duplicates have similar file names you could search using the first few letters of a file name you think might be duplicated somewhere.

Yes, that will be tedious if there are a lot of duplicates. Is your HDD/SSD filling up?

1 user thanked author for this post.

Slowpoke47

Reply | Quote

DrBonzo wrote:

Is your HDD/SSD filling up?

No, not yet.  But at some point in the past, the directories duplicated most of the files, and the dupes are not in their assigned folders.  Hoping to engage assistance without having to go thru them one by one.

Reply | Quote

[Ascaris]

I used to use fslint, but its own developer suggests using its improved,faster, and more often updated remake (by another author), czkawka.

Czkawka is available as a compiled portable executable for Linux. Just put it in whatever directory (folder) you wish, make sure it is set to executable (right click in file manager, then Properties, in the permission tab), and launch by double clicking. You can create a .desktop shortcut to put it in your main (start) menu too, though I only remember how to do that offhand in KDE Plasma.

Dell XPS 13/9310, i5-1135G7/16GB, KDE Neon 6.2
XPG Xenia 15, i7-9750H/32GB & GTX1660ti, Kubuntu 24.04
Acer Swift Go 14, i5-1335U/16GB, Kubuntu 24.04 (and Win 11)

• This reply was modified 1 year, 5 months ago by Ascaris. Reason: Added links

1 user thanked author for this post.

Slowpoke47

Reply | Quote

Found some discussion of this app on Linux Mint forum:

Downloaded app bundle, hope to get it deployed.

Edit- In Properties screen, system tells me I must choose app to open the file.

Out of a list of about 30 items, I chose several that looked like possibilities:

But before I get in too much trouble, perhaps you could suggest one (or, none) of the above?

 

 

Reply | Quote

I put links in that message! Either they were stripped, or I messed up during the post somehow. I will add them back now.

The link I gave (and that should be back in there by the time you read this) had native, non-Appimage Linux executables, which is what I use. That is the one titled linux_czkawka_gui.

Appimage is an all-in-one kind of format for Linux executables, like Snap and Flatpak. Although I have never used it personally, it seems like the best bet for all three of the universal Linux formats.

I am not sure why it is not running. I initially suggested setting the executable bit, but then I went back and saw your screenshot and I can see the box is selected. As far as I know, it should run like any other executable. Asking what you want to open it with suggests that the system is treating it like a data file, which would be the case if the executable bit was not set.

I would suggest trying the non Appimage and see how that works for you. I will try the Appimage and see how it works for me.

 

Dell XPS 13/9310, i5-1135G7/16GB, KDE Neon 6.2
XPG Xenia 15, i7-9750H/32GB & GTX1660ti, Kubuntu 24.04
Acer Swift Go 14, i5-1335U/16GB, Kubuntu 24.04 (and Win 11)

1 user thanked author for this post.

Slowpoke47

Reply | Quote

Both AppImage formats worked fine for me. I set the executable bit and double clicked each one in Dolphin (KDE’s file manager), and they started up and ran as intended. If you execute the Appimage from the command line (navigate to the place where the Appimage is located, then type ./linux_czkawka and hit TAB, and it should fill in the rest of the filename for you. Press enter, and if it fails again, please cut and copy any error messages it provides and paste them here.

 

Dell XPS 13/9310, i5-1135G7/16GB, KDE Neon 6.2
XPG Xenia 15, i7-9750H/32GB & GTX1660ti, Kubuntu 24.04
Acer Swift Go 14, i5-1335U/16GB, Kubuntu 24.04 (and Win 11)

1 user thanked author for this post.

Slowpoke47

Reply | Quote

I downloaded the same file from the same site yesterday.  The entry in my directory:

This system is set to open files via single click, but neither single- or double click produce any action.  Context menu by right-click, Properties, generates this:

This is more or less where I was yesterday.

Reply | Quote

Tried apt-get install in cli, not found.  Should have checked software repository (manager) first- found and installed czkawka (no suffixes). Has intuitive gui interface, see s/shot:
Unknown what the (hiccup) designation means, but just used the app to find and remove dupes from docs folder- some files had multiple dupes, could be from updating to newer OS or who knows why.  Docs folder had 270 dupes! App has a babysitting feature where it warned me that I had (inadvertently) marked all examples of a couple of files for deletion.

This was not a difficult process, installation or deployment.  Installation added app to main menu.  Definitely worth doing.

Reply | Quote

Hiccup is a translation of the word czkawka into English.

Glad you got it working. I am not familiar with the software manager of which you, er, type.

I don’t know why the appimage failed to run in your case. I may be missing something here, as my experience with appimages is (to say the least) very limited, but to my way of thinking, it is an executable, not a data type that needs to be opened with another application, so I do not know why your system was asking for a program to run it with. It IS the program!

Linux executables are not like Windows executables in that their .exe extension is necessary for them to be treated as executable. In Windows, if you look (for example) in the firefox folder, you will find firefox.exe, which Windows file explorer lists as “Application.” If you change its name to just firefox, Windows will warn you that changing the name might make it unusable, but if you do it anyway, Windows file explorer will just list the type as ‘file.’ Windows is not aware it is an executable anymore, and if you try to double click it, it will ask what program you want to open it with– like what your Mint OS was asking when you tried to open the appimage of czkawka.

That’s in Windows. Linux works differently.

If you look at any executable file in Linux (say, for example, Firefox again), you will see that your file manager lists the type as ‘executable’ even with no extension. Inside the Firefox program folder, you find firefox, with no extension, but if you look over in the ‘type’ column, it will say ‘executable’ even with no extension present. Of course, Linux also has an executable attribute, which is why we had to tick the little box under Properties to allow the file to be executed, and of course, firefox will normally have that attribute bit set. But even if you were to untick the box, your file explorer will still show firefox as an executable. If you try to run it with the bit not set, it will complain, but the point is that the system knows the file is an executable anyway.

That is why I suggested using the non-appimage executable, which is the version I am familiar with… the one just named linux_czkawka-gui (not appimage). Did you ever try that? I would be curious as to whether it worked. I cannot see why it wouldn’t, but then the appimage should have worked also.

Dell XPS 13/9310, i5-1135G7/16GB, KDE Neon 6.2
XPG Xenia 15, i7-9750H/32GB & GTX1660ti, Kubuntu 24.04
Acer Swift Go 14, i5-1335U/16GB, Kubuntu 24.04 (and Win 11)

1 user thanked author for this post.

Slowpoke47

Reply | Quote

Thread marked as resolved.

Reply | Quote

Ascaris wrote:

Hiccup is a translation of the word czkawka into English.

That possibility did cross my mind.  A developer with a sense of humor…

Ascaris wrote:

That is why I suggested using the non-appimage executable, which is the version I am familiar with… the one just named linux_czkawka-gui (not appimage). Did you ever try that?

No, I didn’t- once I saw the app in Software Mgr I jumped right on that.

From what I can deduce, this app was not originally gui- that was added only recently.  Finished yesterday using czkawka to find dupes in all directories- I have added several to those native to the OS.  Wound up deleting almost 800 dupes in all.  I knew there were quite a few, but still surprised at just how many.  They must have been added when transferring files from our old computer into this one, and a few years back when upgrading the OS in that old machine.

Interesting that the app shows the paths of each item back to the root folders.  Many of the duplicated pairs were in different folders and a few even had different names- meaning that czkawka compares content, not names, to find dupes.  And, some files were in triplicate or quadruplicate.

Finished this operation yesterday in time for the weekly Veeam backup today.  Today’s b/u took 12:12 compared to previous- 13:40 give or take.  If the Veeam b/u’s are incremental- meaning adding only changes from the previous b/u to the current one-  I wonder how it treats files that have been deleted since the previous b/u?

Czkawka is just what I was looking for- quite pleased to have cleaned up the directories.

1 user thanked author for this post.

Ascaris

Reply | Quote

I am glad it works for you! And as you can see from its UI, it can do a lot more too. A very useful tool indeed!

Dell XPS 13/9310, i5-1135G7/16GB, KDE Neon 6.2
XPG Xenia 15, i7-9750H/32GB & GTX1660ti, Kubuntu 24.04
Acer Swift Go 14, i5-1335U/16GB, Kubuntu 24.04 (and Win 11)

2 users thanked author for this post.

Lars220, Slowpoke47

Reply | Quote

I appreciate your pointing me in the right direction!

1 user thanked author for this post.

Lars220

Reply | Quote