I have exactly the same problem.
I got as far as using fdupe to identify duplicates and delete the extras. It was slow.
Thinking about some of the other comments... If you use a tool to create hardlinks first, then one could then traverse the entire tree and deleting a file if it has more than one hardlink. The two phases could be done piecemeal and are cancelable and restartable.
Also see the Oracle (s) of Pythia/Delphi.
Go into cave
Breathe fumes
Hallucinate
Tell local King he'll be victorious in upcoming battle
May he have itchy toenails.
System working as designed.
I, for one, welcome our new AI overlords.
Of course they relieved themselves of the resource (human) that negatively impacted the share price.
Same way they relieve themselves of a resource (bolts) that negatively impacts the share price.
Thing is, it's not actually her virginity that's being protected. It's the privilege of her husband to take it.
I'm reasonably sure it's the bullets...
I'm really appreciating your use of &c.
Are you in the nineteenth century by any chance?
Yet.
As IPv4 blocks get scarcer and ISP's get more customers, they'll all eventually have to move to IPv4 CGNAT.
And that's completely fine for most people.
If you're not one of those people, then IPv6 is your saviour.
Remember: in most cases the c-suite consider you merely as resources(human) a source of labour.
To them employees are things.
Customers are just a revenue source, they are also just things.
CEO's are just chasing their next quarterly performance bonus.
For backup or for file-level reduplication?
If the latter, how?