this post was submitted on 20 Sep 2025
45 points (97.9% liked)

Selfhosted

60967 readers
816 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

  1. Be civil.

  2. No spam.

  3. Posts are to be related to self-hosting.

  4. Don't duplicate the full text of your blog or readme if you're providing a link.

  5. Submission headline should match the article title.

  6. No trolling.

  7. Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.

  8. AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago
MODERATORS
 

I just woke up to all my lights on and my server off. It was a power cut, but... Checking the logs for Proxmox, it seems to have lost memory.. There were no logs for 10 days and when rebooting it said the system clock was off by as much (931,920 seconds = 10.7 days). What do? Anyone seen this before?

Thanks all, gonna replace the CMOS battery as suggested. Got a few in storage. πŸ–€

all 15 comments
sorted by: hot top controversial new old
[–] Brkdncr@lemmy.world 32 points 10 months ago (2 children)

Your cmos battery is bad or your grid power was running below normal frequency

[–] tal@olio.cafe 21 points 10 months ago* (last edited 10 months ago) (1 children)

I don't think that the grid frequency is used for PC timekeeping. You have internal timekeeping circuits. AC power stops at the PSU, and I don't think that there's any cable over which a time protocol flows from the PSU to the motherboard.

[–] tal@olio.cafe 8 points 10 months ago* (last edited 10 months ago)

I'd also bet against the CMOS battery, if the pre-reboot logs were off by 10 days.

The CMOS battery is used to maintain the clock when the PC is powered off. But he has a discrepancy between current time and pre-reboot logs. He shouldn't see that if the clock only got messed up during the power loss.

I'd think that the time was off by 10 days prior to power loss.

I don't know why it'd be off by 10 days. I don't know uptime of the system, but that seems like an implausible amount of drift for a PC RTC, from what I see online as lilely RTC drift.

It might be that somehow, the system was set up to use some other time source, and that was off.

It looks like chrony is using the Debian NTP pool at boot, though, and I donpt know why it'd change.

Can DHCP serve an NTP server, maybe?

kagis

This says that it can, and at least when the comment was written, 12 years ago, Linux used it.

https://superuser.com/questions/656695/which-clients-accept-dhcp-option-42-to-configure-their-ntp-servers

The ISC DHCP client (which is used in almost any Linux distribution) and its variants accept the NTP field. There isn't another well known/universal client that accepts this value.

If I have to guess about why OSX nor Windows supports this option, I would say is due the various flaws that the base DHCP protocol has, like no Authentification Method, since mal intentioned DHCP servers could change your systems clocks, etc. Also, there aren't lots of DHCP clients out there (I only know Windows and ISC-based clients), so that leave little (or no) options where to pick.

Maybe OS X allows you to install another DHCP client, Windows isn't so easy, but you could be sure that Linux does.

My Debian trixie system has the ISC DHCP client installed in 2025, so might still be a factor. Maybe a consumer broadband router on your network was configured to tell the Proxmox box to use it as a NTP server or something? I mean, bit of a long shot, but nothing else that would change the NTP time source immediately comes to mind, unless you changed NTP config and didn't restart chrony, and the power loss did it.

[–] lime_red@lemmy.world 7 points 10 months ago

I feel like other things would have misbehaved if the power frequency was too low. And I'd expect the RTC to run well while power is on, and fail to accumulate time while power is off, but still remember the time at power off.

None of what I said above explains what we are seeing with our eyes though.

[–] 486@lemmy.world 29 points 10 months ago

It is just a dead CMOS battery. So the clock had the wrong time, which in turn also causes the log entries to have the wrong time and date. Simply replace the battery. It is most likely of the CR2032 type.

[–] lime_red@lemmy.world 6 points 10 months ago

In line with the possibility of your system's clock actually being wrong, can you correlate some other events in your logs to when you know they occurred? What about inside your VMs and CTs?

[–] galacticbackhoe@lemmy.world 4 points 10 months ago* (last edited 10 months ago) (1 children)

What kind of storage is it running on? Maybe the filesystem went read-only as another possibility to cmos battery issues.

Usually a cmos battery alone wouldn't cause something like logging to stop.

[–] Lyra_Lycan@lemmy.blahaj.zone 2 points 10 months ago

It went read-only because the root drive ran out of space. I've expanded its storage now. Big whoopsie on my end

[–] thelittleblackbird@lemmy.world 3 points 10 months ago (1 children)

By any chance.... Your proxmox is not close to any source of big electromagnetic fields, right?

Close to big Transformator or electric motors, right?

[–] Lyra_Lycan@lemmy.blahaj.zone 3 points 10 months ago (1 children)

Not at all but my god that sounds like that would be recipe for disaster

[–] thelittleblackbird@lemmy.world 1 points 10 months ago (1 children)

I've seen things.. Near the gate of TannhΓ€user, things you little people wouldn't believe.

And I am not joking...

[–] shindig1457@lemmy.world 1 points 10 months ago

Tell us more, wise one!

[–] huquad@lemmy.ml 2 points 10 months ago

Prepare yourself. The dark descent is coming

[–] possiblylinux127@lemmy.zip 0 points 10 months ago

Switch to ZFS

Ext4 is way to vulnerable to power loss. It just keep a journal but it isn't nearly as robust as ZFS