this post was submitted on 18 Jul 2024
114 points (100.0% liked)

Technology

37719 readers
279 users here now

A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.

Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.

Subcommunities on Beehaw:


This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] jarfil@beehaw.org 3 points 4 months ago* (last edited 4 months ago) (1 children)

URL shorteners in general, or just Google?

https://wiki.archiveteam.org/index.php/URLTeam

Goo.gl has a namespace for 10 billion entries, it used to keep tracking/analytics data for each link, with a user interface, and it would happily generate them for stuff like Google Maps links.

How much money would you say it takes to even maintain a system like that, plus update its security, not to mention account for changing web standards, at that scale?

[–] jonne@infosec.pub 5 points 4 months ago (1 children)

Probably like $50/month in cloud resources if you turned off all the extra stuff and only did redirects and kept it around in read only mode. You'd need to do some dev work up front and price that in as well, obviously.

[–] jarfil@beehaw.org 4 points 4 months ago* (last edited 4 months ago)

$50/month would barely scratch the surface.

Let's take a conservative approach, and say there are:

  • only 1 billion links
  • each link only points to a URL of up to 100 characters in length on average (some will be 1000 or longer, but let's hope some are 50 or shorter)
  • less than 10 billion daily hits total (that's an average of 10/link)
  • the response time should be well under 50ms.

Now you're looking at 100GB of raw data to put into a database, that needs to return 100K answers/second, in less than 50ms each, worldwide, 24/7.

What is your estimated cloud cost for something like 256GB of RAM, 128 cores, 10Gbps connect, replicated across several zones, and 1TB/day outgoing transfer?

That's only for the redirect responses in read-only mode, nothing else. You will also need some maintenance to keep it 24/7, for when the server catches fire, or gets obsoleted, and when new exploits come up against your software stack.