this post was submitted on 26 Oct 2026
8 points (100.0% liked)

Programming

29065 readers
413 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 3 years ago
MODERATORS
 

I am building an open source domain cataloging system, based on an XXX.XXX.XXX format.

INF.REF.DIC (information, reference, dictionaries), for example, would include: Cambridge, Webster, and Urban Dictionary. The purpose is to create buckets for comparative research -- like a library.

I want to give my users a desktop application to explore the catalog. So, I've decided to host the catalog over IPFS, and build the desktop app in Node, with Helia. This way, the catalog will be fully distributed and easily manageable.

I'm wrestling with a couple of database topographies, and would enjoy some feedback from you guys:

Blockchain

The blockchain topography exists in 4 layers:

  1. Root Index Node
  2. Index Layers
  3. Drawer For Each Call
  4. Individual cards within said drawer

Since XXX.XXX.XXX offers 101559956668416 possible combinations, each inode links to 1296 locations (CIDs for the drawers) within the catalog. As you crawl down the inode tree, you eventually hit the correct drawer for the subject matter you're seeking.

Each card has 25 entries on it, including: domain ranking (scored 0-9, based on a mixture of credibility and culture relevance), domain name, and description. The card then links to the next card's CID in the drawer.

The drawback to this system is that I would have to create a buffer of insertions during the day, and then comb the drawers backwards at midnight. The drawers are sorted by the ranking score.

My entire system would be distributed though.

Mounted File System with Key-Value Store

The alternative is something like this: https://github.com/ki4jgt/PPD

An on-disk key-value store, inside IPFS's built in file system.

The problem with it is that it'll require me to purchase hosting servers with large amounts of storage data, and eventually use the hashed location values to distribute the data between multiple nodes. And develop a manifest system, dictating which node keys are responsible for which ranges.

What I need

I need some heads to butt together and discuss alternatives, or give me the best option out of the 2 choices. Give me pointers on what to look out for. Etc.

Thanks for your time.

top 4 comments
sorted by: hot top controversial new old
[–] talkingpumpkin@lemmy.world 2 points 6 days ago (1 children)

How much data is it? How often read/written?

[–] ki4jgt@feddit.org 0 points 6 days ago* (last edited 6 days ago)

I'd be updating the data at midnight, every night. But I'd also be rewriting major portions of the blockchain every night as well. That's what I meant by "combing the drawers."

My users would be reading the individual cards from their own computers.

[–] ludrol@programming.dev 1 points 6 days ago (1 children)

From my limited testing and one paper IPFS has huge overhead in finding a file. It can take dozens of seconds.

[–] ki4jgt@feddit.org 0 points 6 days ago

But not with files you've already got cached. So, if you like spending time in a particular section of the catalog, it's already on your computer.