Technology

59135 readers

3376 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

228

Any “small-web” search engines? (lemmy.zip)

submitted 2 months ago by dch82@lemmy.zip to c/technology@lemmy.world

78 comments fedilink hide all child comments

I use Duckduckgo, but I realised these big(ish) search engines give me all the commercialised results. Duckduckgo has been going down the slope for years, but not at such a rate as Google or Bing has.

I want to have a search engine that gives me all the small blogs and personal sites.

Does something like this exist?

you are viewing a single comment's thread
view the rest of the comments

[–] phanto@lemmy.ca 2 points 2 months ago (1 children)

I tried running yacy for a while but it just ran for a bit less than a day then ran out of memory and crashed, over and over. Tried to figure out the problem, but it's niche enough that I couldn't get anywhere googling the issue.

[–] zutto@lemmy.fedi.zutto.fi 2 points 2 months ago (1 children)

This is a bit off-topic, but did you try to increase the JVM limits inside Yacy's administration panel?

Spoilering to hide wall of text related to this topic.

This setting located in /Performance_p.html-page for example gives the java runtime more memory. Same page also has other settings related to ram, such as setting how much memory Yacy must leave unused for the system. (These settings exist so people who run Yacy on their personal machines can have guaranteed resources for more important stuff)

Other things that would reduce memory usage is to limit the concurrency of the crawler for example. There's quite a lot of tunable settings that can affect memory usage. Would recommend trying to hit up one of the Yacy forums is also good place to ask questions. The Matrix channel (and IRC) are a bit dead, but there are couple of people including myself there!

Also, theres new docs written by the community, they might help as well! https://yacy.net/docs/ https://yacy.net/operation/performance/

[–] phanto@lemmy.ca 2 points 2 months ago

Yeah, I did try that. Basically, if I doubled the memory I allocated, I gave it half again longer before it crashed, but it still crashed, eventually.

It's no big deal, this was last year, I may try again one day. Loving Searxng though!