this post was submitted on 22 Nov 2023

1 points (100.0% liked)

Photography

24 readers

1 users here now

A place to politely discuss the tools, technique and culture of photography.

This is not a good place to simply share cool photos/videos or promote your own work and projects, but rather a place to discuss photography as an art and post things that would be of interest to other photographers.

founded 1 year ago

MODERATORS

communick@viewfinder.pro

Catalog, Culling, and Import Software for 2023? (alien.top)

submitted 11 months ago by eeeerrrppp@alien.top to c/photography@viewfinder.pro

31 comments fedilink hide all child comments

Hello! This is both a question and commentary, as I'm hoping to finally solve a dilemma that's bothered me all year.

I have a library nearing 1 million photos and videos of all formats (including RAW and heic) that I've built over several years. Nowadays, I edit with Capture One, which is fantastic, but not very useful for importing (lack of options), culling (slow render), and especially cataloging (breaks down with a year's worth of photos). I do not yet have a NAS, and store everything on two USB HDDs — broke artist with ancient mac arch etc etc.

So began the quest to find software that could organize and show my photos. What's the use of a million photos if it takes too long to look at any of them? My POIs were importing (copying files from source to library, sorting by year/month, detecting duplicates), culling (rate, color tag, keywords, and preview speed in full view), and catalog (fast gallery thumbnails, albums, sorting, small data size). Here's what I've found thus far:

-Lightroom: Does it all, but I hate it. Overpriced, lacks some professional features, doesn't play well with a lot of non-adobe apps, no duplicate finder, slow-ish.

-PhotoMechanic Plus: Excellent cataloging, okay culling and import. Quite expensive, doesn't let you assign custom keyboard shortcuts, weird-ish licensing system, hard to tell what data you have in your RAW photos (can you save under/overexposed parts? will it look good in B&W?)

-ACDSee 10 for Mac: Eh? Seems to work well, but has some... quirks. Always shows both jpeg and raw when shooting linked. Feels underdeveloped still, or like a paid version of Darktable.

-FastRawViewer: Amazing for culling, shows full RAW data and lets you view with basic "effects" like shadow boost of B&W. Lacks keywords or any kind of cataloguing or importing.

-XnViewMP: Previews seem fast enough, but software seems slow and sometimes has issues when scrolling past videos. Unsure how usable it is for importing.

-Mylio: Offers a lot of features and a nice UI for free, but also seems oversimplified, has a lot of weird restrictions, doesn't offer a good way to switch to my backup drive if my main fails, and seems to read the wrong capture date on many of my Panasonic RAW photos. I'm also worried it won't stay free forever.

-PhotoSupreme: Supposedly similar to PhotoMechanic? I could not get it to work very well, seems to lack the import to year/month folder feature so I didn't spend a ton of time with it.

-DigiKam/DarkTable: (similar experience with both) Worked okay, but both software and previews were slow. It felt like my catalog file was set up wrong and slowing things down, but everything I checked looked correct. I tested these a while ago now, maybe they've improved or I should change something?

-PhotoStructure: I'm hopeful? Seems to just be for cataloging and deleting duplicates, but paired with FastRawViewer and C1, I'm okay with that. I'm running this now and will update... so far looks like it's going to be slow, unfortunately.

-Adobe Bridge and similar DAMs without cataloging: Too slow and complex to navigate for more than like 20 pictures. At least FastRawViewer lets you see subfolder contents, unlike most of these. Great if you do low-batch work, but I shoot a lot (concerts, etc) so it's a non-starter.

-Network DAMs, ie Daminion: Sounds great. Doesn't work for me per above...

I'm left baffled. How are there all of these almost-good options at vastly different price points? Like, smash together ACDSee and FastRawViewer and I'd be a happy camper, but no. I know development is hard and I'm not saying any of these are bad software, but, surely there's something that covers these relatively basic bases?

For myself, I see two path: Drop money I don't have on PhotoMechanic Plus, or make do with FastRawViewer and C1 with a minimal catalog. I feel like there has to be a better option though. Does anyone have tips? A program I haven't tried, or settings to make the above work betetr?

Thank ya!

P.S. I'll update the above as I learn more!

top 31 comments

sorted by: hot top controversial new old

[–] sakrebs84@alien.top 1 points 11 months ago (1 children)

Wow, that's quite the collection! I've been using PhotoMechanic Plus and it's been solid for me, especially with the cataloging. It's a bit pricey, but might be worth considering given the size of your library. Good luck!

[–] eeeerrrppp@alien.top 1 points 11 months ago

I’m very close to just biting the bullet and getting it. If it had better tools to evaluate raw, i definitely would, but as is and with lack of keyboard shortcut customization, I just have this overwhelming feeling that it’s the wrong choice for me. Maybe it’ll feel better when I actually have it though 🤷🏻‍♀️

[–] TastyStatistician@alien.top 1 points 11 months ago (1 children)

I've been looking for a Lightroom alternative. I've been doing the free trial of all the major raw editing programs. I haven't decided which one to buy yet. On1 Raw 2024 has AI catalog tools. Not sure how it would run on an old mac but it could help tag your pictures.

[–] eeeerrrppp@alien.top 1 points 11 months ago (2 children)

Pretty much same. On1 2021 (i think?) was the first replacement I bought and I certainly didn’t hate it. But I remember it also chugging to handle the large catalog, and while it had more features, the quality didn’t compete with C1. Has it improved though? Without the AI tools, is the cataloging faster now?

[–] TastyStatistician@alien.top 1 points 11 months ago (1 children)

2024 version just came out. They say it has major speed and ui improvements. I haven't imported my full library yet. I've been using a small catalog for testing the free trial. I think for the price, it's good and worth a try.

The reason I'm looking for a new raw editor is because my old non-subscription version Lightroom(5.7) doesn't support my new camera's raw files.

I've tried C1 many times but I've never been happy the interface and raw rendering. I like on1 probably more than C1. C1 is a heavy program in my experience. On1 is faster on my computer.

DxO photolab 7 is currently my top choice for Lightroom replacement. It's not perfect. Raw rendering is great and it gives me a great starting point but the other tools aren't as good. My main problem is the tone curve is poorly implemented. I haven't explored the catalog tools yet. It's double the price of on1.

[–] eeeerrrppp@alien.top 1 points 11 months ago (1 children)

I’ll check it out!

What do you shoot with? I personally love the C1 rendering with my Panasonic and Olympus cameras, but my old Sony is a little bland, so it might vary by model/brand? I feel like it’s my favorite of what I’ve used, but that’s subjective… Overall I agree, DxO seems to trade blows with C1 more than On1 or even LR can. But if you’re a fellow user of curves a lot, I believe C1 has the best implication I’ve used (excluding DaVinci Resolve). If you work in small batches, you might also consider Affinity Photo?

Sounds like we’re in similar situations… I’m excited to hear what you land on!

[–] TastyStatistician@alien.top 1 points 11 months ago (1 children)

I only had a Nikon D5200 and Lightroom 5.7 for a very long time. About a year ago, I bought a Fuji X100V for travel and street photography. My Lightroom version is too old for my Fuji camera. I tried all the other competitors and decided to buy DxO Photolab 7 only because of the raw rendering. It doesn't have a catalog, only a file browser so it's not great for organizing large numbers of images. Adobe bridge is much faster.

I have Affinity Photo. I like it a lot but not for raw development. I'm using the Nik Collection to do the final adjustment. It's not the best workflow but I'm getting pretty good results.

[–] eeeerrrppp@alien.top 1 points 11 months ago

Okay yeah! Interesting approach. I develop RAW with Affinity on my ipad and am very satisfied given how portable it is. But it’s a slow workflow for large projects :/

[–] vonbauernfeind@alien.top 1 points 11 months ago (1 children)

The large catalog

Do you keep all your photos in only a single catalog? I recently started spinning off my photos into separate catalogues (by year) to help with just seeing less, myself.

I'm also hyper aggressive about deleting rejected photos every few months; I think I'm only at 20,000-30,000 photos these days, when I used to be close to 100k, simply because what's the point of saving rejected photos I'm never going to look at again?

The bigger red flag for me is your hardware. I'd consider upgrading that first. USB HDD's are not the most reliable, and storing media you're actively using on HDD's is going to be slow, especially over HDD. If the USB drives are SSD drives that will help, are they running over USB-C, USB-A, Firewire?

I keep my active library on an SSD, and backup to a home server running HDD's. But older computers with lower RAM pools are going to struggle to keep up with modern memory hungry programs. I believe C1 (amongst others) can also offload graphical processing to preview to GPU's these days, and if you're running an older machine like you said, that's not going to be an option. Previews on my version of C1 23 render in about as fast as I can click through (even the mega 61mp files taken with my A7R V), but I also have a processor that's only 2-3 years old and a high end AMD GPU to help (with a crapton of GDDR RAM).

[–] eeeerrrppp@alien.top 1 points 11 months ago (1 children)

I might start trying yearly catalogs and an ssd for the current one, that’s a good idea! Doesn’t solve all my problems, but that’s at least a major improvement to traveling with an hdd haha!

I’m okay with small previews, and my internal ssd and ram is plenty for a catalog with the right software. I intend to cull and delete stuff from years of being less vigilant, but the hope is to catalog it all first so I can see everything clearly.

It’s worth noting SSDs are only significantly more reliable if you’re moving the drive a lot. The best longevity is the cloud. For speed, it’s also foggier. My HDDs are pretty quick with USB 3.1, it’s not thunderbolt, but I’m not editing video, so it’s not really a bottleneck that affects anything I’m trying to do. Plus, due to the physical caching structure of low-mid range SSDs, the difference is smaller than it looks on paper for random access photos (to my understanding). When in FRV, I get the same speed you describe in C1, but with the actual raw. In C1, I could get that in small catalogs, but only with the compressed/low res previews (since it’s in the catalog on my machine). If you’re getting the actual raws loading that quick, I’m guessing it’s more because you’re using a newer version of C1 than because of write speed. Granted, that’s all based on anecdotal evidence so I can’t be sure. Regardless, it’s the catalog speed I’m most concerned with, so I’m personally okay with slower devices.

I think I’m gonna try your advice until I find sometime better… Thank you :D

[–] vonbauernfeind@alien.top 1 points 11 months ago (1 children)

The yearly catalogues help because it's less that your computer will need to load into active memory and manage. This will help a little with flipping through things.

I'm only talking about speed with regard to SSD's. I'm not worried about reliability. Your typical 7200RPM HDD is going to have a data transfer rate at around 80-160MB/s. USB 3.0 has a theoretical maximum of 600MB/s, USB 2.0 has a theoretical maximum of around 30MB/s, if I recall correctly. Typical SSD's vary in transfer rate, but I know mine for my C1 library is 580MB/s (though connected over SATA theoretical Max is 6GB/s, that's not realistic to hit on my PCIe version).

So if you're talking an SSD over USB 3.0 or in your computer, you'll see incredibly different transfer speeds; if you're working over 3.0 or higher (which you said you were) you should see a huge step up in individual file accessibility with SSD's; that being said, previewing is going to be based on cache and hardware render, which on an older Mac may be the bottleneck you're hitting here. C1 has different overhead and uses hardware differently from FRW I think, and C1 has a lot of other tasks it's hitting your computer with besides just rendering the previews.

Like I said, it takes less than half a second for a 60 megapixel image to render on my machine. But we're operating very different hardware too (i7-9700K & a RX 7900 XTX with 24GB GDDR6), and I know that's playing a role.

I suspect you're being bottlenecked by the older processor & rendering solution in your Mac, but getting onto using smaller catalogues with an SSD should help you get some improvement.

[–] eeeerrrppp@alien.top 1 points 11 months ago

Okay, we’re pretty much on the same page!

I admit I’m off with the SSD info, but there are scenarios where the gap is diminished. I’ve experienced it first hand, I just can’t remember what caused it… the internet can explain, though I couldn’t find anything that explained it concisely. Takeaway: SSDs are objectively faster, but not required for my personal use case atp.

We agree that C1 does something inefficient with it’s cataloging and raw rendering (I’m calling it “bloat,” and I think all of us are mostly guessing what that involves). But that’s fine, when I’m editing, neither of those are priorities.

Hence the search for other software… PM does the cataloging and importing satisfactorily, and FRV does the RAW render/cull great! So, software is the difference. Yea, better hardware will better handle C1’s “bloat” and have more headroom overall, but PM and FRV prove that what I have already has the ability to work satisfactorily with my RAW files (though, I’d definitely need an upgrade if I was regularly shooting your 60mp!) Takeaway: C1 is not optimally efficient, so compromises, like the year-to-year catalog, are necessary—doubly so on lower end hardware.

I’d just love to spend less than ~$350 + cost of upgrading in a few years. It seems unlikely that FRV and PM are the only programs without so much “bloat”, but maybe they are? I don’t know what this “bloat” is, so maybe it’s harder to remove than I know, but my impression is that it is more so something C1 added than FRV and PM took away.

So yep, for now I’m using your solution because I have C1 and can’t afford the PM and FRV combo. Hopefully a middle option comes along 🤷🏻‍♀️

[–] luke400@alien.top 1 points 11 months ago (1 children)

I still use Aperture as I am yet to find something better. Although its quite broken now.

[–] eeeerrrppp@alien.top 1 points 11 months ago

I’ve heard so many people say this! I sadly never got to try it, but it’s amazing to me that there’s stil nothing that competes. My impression is that DxO is similar, but I’m admittedly pretty clueless 😅

[–] Aroosasheikh5@alien.top 1 points 11 months ago (1 children)

Have you tried digiKam recently? I had a similar experience with it a while back, but it might have improved since then. It's worth giving it another shot. Good luck with finding the right software!

[–] eeeerrrppp@alien.top 1 points 11 months ago (1 children)

I’ll give it a go! Thanks :)

[–] eeeerrrppp@alien.top 1 points 11 months ago

Just updated to 8.1.0 and WOW - I've only tried it with a some of my library so far, but it's so much better than I remembered! Feeling good about this one!

For those keeping score, I still have these concerns:

-Can I figure out the MySQL Internal that seems necessary for large catalogs?

-Will it migrate properly when I have to upgrade computers, probably in a few months?

-I can't find a setting for thumbnail size, so I'm worried that will become an issue before long.

[–] Teams11b@alien.top 1 points 11 months ago (1 children)

Bridge can do most things you are looking for outside of duplicate detection. If your file structure is OK. Duplicate detection is not something that should be needed regularly.

You can view items in subfolders and batch process with Adobe Camera Raw and Photoshop Actions - https://imgur.com/a/ogJdffZ

I'm not satisfied with Bridge and its annoyances, but I might be missing what your goals are.

[–] eeeerrrppp@alien.top 1 points 11 months ago

Oh I think that subfolder content option might be new!

Bridge was painfully slow last time I used it… like a half second per thumbnail. And it doesn’t do any cataloguing or ingest sorting, does it?

It’s similar to FRV, to my understanding the “con” being speed and the “pro” compatibility with heic and video. Does that check out?

[–] No_Shake3769@alien.top 1 points 11 months ago

Just shoot less 😃 You probably spray and pray a lot if you have 1 millions photos.

[–] ISAMU13@alien.top 1 points 11 months ago (1 children)

Don't let the perfect be the enemy of the good.

[–] eeeerrrppp@alien.top 1 points 11 months ago

I’m just looking for usable haha 😅

[–] RedHuey@alien.top 1 points 11 months ago (1 children)

This is the inherent problem of taking a million photos. There is NO solution to this problem that won’t take many many hours to implement, if it’s even practical anymore.

Step one is turning off the burst mode of your camera.

Step two is adopting a proper filing structure when you import your photos. Not a library, but a specific place on your storage medium with a directory tree. You should at least be able to immediately drill down to a specific date with a specific camera.

Step three is tagging as aggressively as possible when you import the images. Every single image should have at least one, if not three or four tags, so that you can find those images by tag(s).

If you’ve done steps two and three, then your problem is only volume, which is solved by not taking 600 pictures of the same things (step one).

If you have not already done steps two and three, then that’s what you have to do. On your old photos and from here out. Nothing further will really matter or be helpful until you’ve done that.

A possible way to approach it, if you did do step two, is to divide the whole into manageable chunks. Start with the current year. Get that under control all by itself. Then, if that wasn’t an unmanageable amount of time, go one year, or six months, or whatever, further back. Get everything tagged and properly organized. Etc.

But no software solution is going to do any good if you don’t have this done.

[–] eeeerrrppp@alien.top 1 points 11 months ago (1 children)

I agree with step two, but otherwise, I feel like this is a rather pessimistic approach. Unless your camera is collecting dust, a million pictures isn’t unrealistic for anyone with a camera. I’ve been working in photography for years, and I back up my phone photos to the same database. Burst mode is often a necessity, I’ve met way more people who use it too seldom than too much. Also, all of that is just cataloging. Importing, culling, and mobility are crucial to many photographers as well. And cataloging is absolutely a software-solvable problem. Lightroom and Photomechanic both have it figured out, in addition to numerous network applications. Tagging is great, but you will inevitably loose and forget photos if that is the only way you identify them. A timeline of all photos is essential to many of us who capture a wide variety of styles.

Yes, it will take hours to generate a catalog. But once done, every image is immediately viewable, as opposed to spending hours searching every time you want to find an old picture. We can have different preferences, but for me a software catalog is the way to go.

[–] RedHuey@alien.top 1 points 11 months ago (1 children)

Did I say “no catalog?” I don’t think so. Of course you need one, but it has to be narrowly defined. A file cabinet with 10,000 drawers is next to useless.

Burst mode of not a necessity for many photos at all. It IS a necessity if you have the modern belief that photography is about capturing some objectively perfect shot that exists, but much be captured for some reward. So you take 50 pictures in 5 seconds of the same exact thing and figure you’ll go through them to find the one that happened to catch that elusive perfect moment. This is simply lucktography. It is meaningless because you, as the “photographer” have done nothing but drive your camera out to some spot, and aim it at something. The camera does everything else. You don’t even have to do the thinking here. It is literally just taking a snapshot with a $5000 Instamatic.

I’ve been taking pictures for probably 45 years now. And I have no where near a million pictures. Also, my ratio of throw-away bad pictures to keepers is nothing like 100:1. You make your choices in this game.

[–] eeeerrrppp@alien.top 1 points 11 months ago (1 children)

Your wording implied using catalogs exclusively as a editing tool, not a viewing/reviewing tool. I’m never going to argue against good file management, but that is largely irrelevant to this thread.

I’m assuming you’ve never shot pictures of a low light band, wildlife in action, stop motion film, sports at night, a high budget wedding, pets playing, an eventful trip, or anything similar? Any of those necessitate hundreds or thousands of pictures unless you have a really low bar for “hit.” If burst rates were useless, they wouldn’t be the major talking point of every new camera that comes out. I’m not here to argue. If you shoot all your pictures in a nice and controlled setting, awesome, but we work in completely different worlds. Also, as I mentioned, I use my library to store non-professional and video work too. The size of it is a necessity of the work I do; that’s a simple fact, and totally independent from skill.

No one has a 100% hit rate in every situation, and even if they did, they have every right to take a lot of pictures.

[–] RedHuey@alien.top 1 points 11 months ago (1 children)

Actually, I’ve done most of those things. Even back in film days before autofocus was invented. You DO NOT need everything automated at 10fps to take pictures, good pictures, even great pictures, of just about anything. If you do, then you don’t know what you are doing. That’s just a fact.

Yes, modern technology can be a great help and does wonderful things, but people were taking great pictures long before even the most rudimentary in-camera light meters came about.

The problem is in the modern belief that there IS a perfect picture that must be captured. That it exists, and that photographers will miss it forever unless they use every tool at their disposal. There isn’t. There are only the pictures you take. That YOU take. Not that the camera takes. Not that Luck takes. If you can’t take a great wildlife picture without auto-everything at 10fps, then you need to go back to First Principles and learn photography again. There is a reason why so many modern photos are technical masterpieces, but have no heart in them. The reason is that nobody took them.

[–] eeeerrrppp@alien.top 1 points 11 months ago (1 children)

I’m glad you chose to slander a fellow professional for your false belief of their techniques instead of actually help answer the question I and others are asking. If your happy with your process, keep it, it’s probably a lot more similar to mine than you are assuming. But I don’t care. I happen to agree that “burst mode” is often overused, but it doesn’t make anyone an idiot incapable of photography as you claim.

[–] RedHuey@alien.top 1 points 11 months ago

Nobody “slandered” anybody or even used the word idiot. Grow up.

[–] eeeerrrppp@alien.top 1 points 11 months ago

Moi? I am hoping to find a software solution that can catalogue everything. Technology-wise, this should be doable. I’m okay with really small previews and my computer, though older, has a good ssd and plenty of ram to host said catalogue. I’m not expecting lightning quick either, just comfortably usable.

I’m considering doing catalogs by year or half year. C1 would probably work if I did it… but in addition to inconvenience in viewing, it could also get messy with ingesting a series of photos that crosses the date line.

I can’t say you’re wrong about hardware. But, to my understanding, you overstate how severe it is. Unless you’re moving it a lot, modern HDDs have comparable longevity to SSDs. And, the way most ssds work, the difference in speed when working with a lot of pictures is noticeable, but not that huge. Only the cloud has a significant safety upgrade to dual hdds or dual ssds. All of the above are likely usb 3.1 speed, except a few of my expensive video ssds, so possibly bottlenecking, but not much. And R/W speed only really applies to culling in my case, which seems perfectly fast in FastRawViewer, and that’s off my main hdd which I admit is failing (I travelled with it a lot, oops).

I’ll probably follow your advice! If I switch to yearly catalogs, I can probably run the current one off an SSD. (More so I can travel than for speed, but I’ll take both wins!) And I definitely have a lot of photos to delete. But my hope was to catalogue everything and then delete, as that should in theory be faster. There remains the dilemma that my backup software would leave the deleted photos on the “B” drive tho…

Are your previews rendering in full when you click through? C1 (et al, except frv) load a low res cached preview while fetching the raw, which I believe stresses ram more than anything. If your full res renders are truly faster, I’m inclined to believe it’s more a C1 software improvement (I’m using an older version) more than hardware since FRV is plenty fast on my machine and hdd. It’s all a balancing act 🙃

[–] eeeerrrppp@alien.top 1 points 11 months ago

I tried Narrative, it seemed great for what it does; but it didn’t seem to have the organization tools I wanted when I tried it. And I recall it doesn’t really have a catalog, or only supports small ones per project? Great tip for the folx it works for though!