Nyfure

joined 1 year ago
[–] Nyfure@kbin.social 8 points 7 months ago* (last edited 7 months ago) (2 children)

you want a frontend, not the "service" itself.
Under "service" i usually understand the main logic part of something. In this case the LLM-processing itself.
Thats probably where the confusion is coming from here.

[–] Nyfure@kbin.social 10 points 7 months ago* (last edited 7 months ago) (4 children)

Bandwidth is one part.. Storage is theother and usually you have less storage than bandwidth anyways.

[–] Nyfure@kbin.social 1 points 7 months ago* (last edited 7 months ago)

Please include the actual calculations for energy-prices as many, you may not know, live in different locations and pay different prices compared to you.

[–] Nyfure@kbin.social 4 points 8 months ago* (last edited 8 months ago)

I like the streaming approach, but i dislike that its linked to stremio.
I want to have the option to use my server (at least for syncing) with my own auth and transcode when needed on the server.
Basically jellyfin with streaming sources and search built-in.

And most debrid services dont seed (well) and have poor retention, so thats also not that great.

[–] Nyfure@kbin.social 10 points 9 months ago

Hetzner Storagebox is $3.5/M for 1TB

[–] Nyfure@kbin.social 3 points 9 months ago* (last edited 9 months ago)

As far as i understood tailscale funnel its just a TCP-tunnel.
So you handle TLS on your own system, which makes sure tailscale cannot really interfere.

If you already trust them this far, might aswell do the same with a VPS and gain much more flexibility and independence (you can easily switch VPS provider, you cannot really switch tailscale funnel provider, you vendor-locked yourself in that regard)

I'd connect the VPS and your home system via VPN (you can probably also use tailscale for this) and then you can use a tcp-tunnel (e.g. haproxy), or straight up forward the whole traffic via firewall-rules (a bit more tricky, but more flexible.. though not that easy with tailscale.. probably best to use TCP-tunnel with PROXY-Protocol).
This way you can use all ports, all protocols, incoming and outgoing traffic with the IP-Address of the VPS.

Tailscale might even already have something that can configure this for you.. but i dont really know tailscale, so idk..

And as you terminate TLS on your home-system, traffic flowing through the VPS is always encrypted.

If you want to go overboard, you can block attackers on the server before it even hits your home-system (i think crowdsec can do it, the detector runs on your home-system and detects attacks and can issue bans which blocks the attacker on the VPS)

And yes, its a bit paranoid.. but its your choice.
My internet connection here isnt good enough to do major stuff like what i am doing (handling media, backups and other data) so i rent some dedicated machines (okay, i guess a bit more secure than a VPS, but in the end its not 100% in your control either)

[–] Nyfure@kbin.social 1 points 9 months ago* (last edited 9 months ago) (2 children)

Many systems dont support subpaths as it can cause some really weird problems.
As you use tailscale funnels, you really want incoming traffic from the internet. I am not sure thats a good idea for e.g. homeassistant that is limited in access anyways.
Might aswell use tailscale and access the system over VPN.

And for anything serious i wouldnt use something like funnel anyways. Rent a VPS and use that as your reverse-proxy, you can then also do some caching or host some services there. Much simpler to deal with and full support for such things as you then have an actual public IPv4/IPv6 address to use.
Heck, dont even have to pay for it with the Oracle Always-Free system.

[–] Nyfure@kbin.social 2 points 9 months ago

In an more ideal world, getting less money because people tip less, would push you to reconsider the job choice and ultimately switch to something more lucrative.
With less workers, the company would be forced to pay more to even get employes.

Problem with this idealised scenario is, it doesnt work in the US, because workers are getting screwed so much and have so little choices at those low paying jobs, they'd be the ones loosing massively in the short-term.
And with little support structures my the states and federal government, they would fail.. and the 2 party system would fail them even harder, noone cares about them in the government.. too much invested in fighting imaginary culture wars.

But then again, using less services of the business leads to the same outcome in the end, so even that wouldnt work well.
The business will always win in the short-term.
So as it is ineviteable, maybe its better to think long term anyways.

And everyone wants tips these days, no longer just a gratitude or paying low wage workers, but now also a 'bid'.. (sure not every worker might like relying on tips, but specially well paid servers prefer it as they make bank)
I dont see you getting iut of tipping either way very well without government intervention.. which i dont see happening, but you have orher big issues too..

[–] Nyfure@kbin.social 1 points 9 months ago* (last edited 9 months ago)

You can not only use that information for e.g. blackmail, but also to build material to manipulate you to do things without you knowing.
Information is a powerful tool.

[–] Nyfure@kbin.social 3 points 9 months ago* (last edited 9 months ago)

Yes, you need an organization which signs your certificate, so it is trusted by default. This is our trust-anchor so we know the certificate presented was validated and it was given only to the website owner.
There are numerous around the world for that.
And if that is no longer offered, you can just not have your certificate signed, which means browsers will complain about it.
But you can trust your own certificate yourself. Or create your own certificate authority which can then sign other certificates for the community as their new trust anchor.
I think we would very quickly build the web-of-trust, but for certificates.

You can even not have certificates, but keep an weak form of TLS (no idea if browsers support TLS_DH_anon_*), but its still encrypted and can only be broken by an active Man-in-the-Middle-attack. (which is theoretically detectable later on)
Diffie-Hellman is an awesome key-exchange.

[–] Nyfure@kbin.social 9 points 9 months ago

How much time do you have? Because even small models will take alot of time on that kind of hardware to spit out a long text..
And the small models arent that great. I think the current best and economic model would be a mistral, mixtral or dolphin.
If you got the power, nous-capybara is very good and "only" 34B parameters (loading alone needs like 40GB of memory).

[–] Nyfure@kbin.social 2 points 9 months ago

When i was with a customer who was using one of ther VPS offers, performance was unexpectedly low and upon contacting support it was clear the small fish dont get great support answers, but rather pushed to the FAQ.

And i personally find their offerings and marketing scummy. Big promotional prices, but always some small print with a higher price after x Months.
Or just stuff thats not included by default.
I never had that with other (also very cheap) providers.

As long as it works great for you, i wouldnt see a reason to leave.
There arent that many providers offering such small ressources at all or at such a price. To be fair, not much one can do with those specs.. 10GB storage is very limited already.
But for those specs.. always free oracle tier would work too (though requires a credit card).

view more: ‹ prev next ›