I like freefileSyncs UI and used it to copy MTP files (from my phone) to my network drive, and put in for a donation.
But I'd love if it had a de-duplication function, essentially what to do if the files are the same. That would help clearing out files from my phone. There are other similar programs, dupeguru etc, but they can't seem to handle MTP very well.
Since freefilesync already checks if files are identical it doesn't seem like it would add too much complexity.
Please consider adding dedupe (delete duplicates) function
- Posts: 5
- Joined: 27 Feb 2022
-
- Posts: 3620
- Joined: 11 Jun 2019
It doesn't seem to add much complexity, but it does. FFS is NOT a duplicate finder and should never be used as one. This has been brought up many times, the answer is to find software that is designed with what you want it to do. The content comparison in FFS would be wildly inefficient as it would be (n-1)! complexity. For n files, there are (n-1)! compares you would have to do.
Also, dedupe does not utilize deduplication at all, that is a Microsoft optimizer for saving storage space at a data block level. Dedupe and other duplicate file finders use way more efficient, "fuzzier" matching than FFS.
Also, dedupe does not utilize deduplication at all, that is a Microsoft optimizer for saving storage space at a data block level. Dedupe and other duplicate file finders use way more efficient, "fuzzier" matching than FFS.
- Posts: 5
- Joined: 27 Feb 2022
sorry, i searched the forum and found scant mention of dedupe or duplication and none addressing this particular request.
in any case, I think dedupe is the wrong term for what I am actually asking for:
consider that when FFS identifies a file that exists in both target and destination, it must make an assessment as to whether the files are identical, or which of the files is newer.
What I am suggesting is that if the files are deemed identical, the file on either the target or destination is deleted.
I am not suggesting a true dedupe in that each file needs to be compared to all others, which yields the Big O complexity you describe, just an additional option for a comparison FFS has already made.
in any case, I think dedupe is the wrong term for what I am actually asking for:
consider that when FFS identifies a file that exists in both target and destination, it must make an assessment as to whether the files are identical, or which of the files is newer.
What I am suggesting is that if the files are deemed identical, the file on either the target or destination is deleted.
I am not suggesting a true dedupe in that each file needs to be compared to all others, which yields the Big O complexity you describe, just an additional option for a comparison FFS has already made.
-
- Posts: 3620
- Joined: 11 Jun 2019
Oh I think I see! You want files that exist on both sides, identically, to be deleted from one or both sides? I will say, that makes no sense as a function in a sync utility. Is there a specific reason you want this?
- Posts: 5
- Joined: 27 Feb 2022
understood, the aversion to feature creep, but I'd sort of consider it a loose, one-time "sync" and then a clean up task.Oh I think I see! You want files that exist on both sides, identically, to be deleted from one or both sides? I will say, that makes no sense as a function in a sync utility. Is there a specific reason you want this? xCSxXenon, 28 Feb 2022, 15:05
Specifically, I'll record some videos on my phone and download a few videos here and there to my PC/network drive, but then later get around to transferring the rest later
Since I've already transferred some videos, FFS helps with detecting and skipping those, then transferring the rest.
But now that the sync is complete (both sides match), I can safely delete the files from my phone. That's the use case, and it should only require FFS to offer and an additional test (Files are equivalent) and action (Delete left, delete right, do nothing), without any further computation or comparison.
-
- Posts: 2290
- Joined: 22 Aug 2012
You should still be able to do what you want in FFS, albeit partially manually.
Directly after having run a sync, you can make the "=" (left and right are equal) Compare result category visible by clicking the "="-icon mid below the (then likely empty) left and right Compare result window.
Then, at the side of your phone (left?) select those files you want to remove and press delete or delete via the right-click context menu.
Directly after having run a sync, you can make the "=" (left and right are equal) Compare result category visible by clicking the "="-icon mid below the (then likely empty) left and right Compare result window.
Then, at the side of your phone (left?) select those files you want to remove and press delete or delete via the right-click context menu.
-
- Posts: 3620
- Joined: 11 Jun 2019
That is an option /\
This is the same as many other posts where OP wants to delete files from source after copying to destination. Rabbit hole here: viewtopic.php?t=8781
Although, this may be different since you are most likely using MTP. You will need to make use of ADB and USB debugging in the phone's developer options to achieve what you want. Using those tools in a post-sync command or script can delete files from the phone.
This is the same as many other posts where OP wants to delete files from source after copying to destination. Rabbit hole here: viewtopic.php?t=8781
Although, this may be different since you are most likely using MTP. You will need to make use of ADB and USB debugging in the phone's developer options to achieve what you want. Using those tools in a post-sync command or script can delete files from the phone.
- Posts: 8
- Joined: 16 Mar 2022
Uh, I use 'Windows Server' for deduplication purposes. In SOME situations (like virtual machines) it can save me 75-90% of my disk space. I am ALWAYS running out of disk space. I get SOME copies of Windows Server for free from my M$ certifications, and I always run a couple of machines on the trial edition. 180 days isn't bad.. of course, you can renew (I mean 'rearm') the license 5-6 times. That's 3 years in total!
- Posts: 1
- Joined: 3 Oct 2023
Hi all, I love FFS and finally I ran in the same situation.
It would be great to have the choice of an action also for the "=" files.
In my special case we are exporting files to a database, but the users are still working on the files to export.
So we did two copies of all files in two temporary folders /copy1 and copy2), the export software analyzes and then deletes the files from copy1.
This means I can't sync the original files against copy1 as I cannot determine if a file which exists in "original" but not in "copy1" is a new file, a newer version or simply the same as before but deleted by the export tool.
Solution until now is sync the original files against copy2 where I have the option to determine if it's a new file, a newer version or the same as before.
The idea is "delete if it's the same" and after this process, move the rest from copy2 to copy1 and start the exporting tool again.
Will see how i can do this now. Maybe we can tell the export tool not to delete the files after copying.
Thanks in advance
It would be great to have the choice of an action also for the "=" files.
In my special case we are exporting files to a database, but the users are still working on the files to export.
So we did two copies of all files in two temporary folders /copy1 and copy2), the export software analyzes and then deletes the files from copy1.
This means I can't sync the original files against copy1 as I cannot determine if a file which exists in "original" but not in "copy1" is a new file, a newer version or simply the same as before but deleted by the export tool.
Solution until now is sync the original files against copy2 where I have the option to determine if it's a new file, a newer version or the same as before.
The idea is "delete if it's the same" and after this process, move the rest from copy2 to copy1 and start the exporting tool again.
Will see how i can do this now. Maybe we can tell the export tool not to delete the files after copying.
Thanks in advance
