Oh f❌k it, my NAS crashed in the middle of nightly backups because presumably that + Nextcloud uploads were too much.

I hate when the front LCD comes on. Because it's 40 characters out of a kpanic.

Even better, My mounts are over NFS. If I had to shut a container down right now it would hang until the NAS came back, potentially even needing a forceful host power-off from the iDRAC.

Even better: the one iSCSI initiator (backup) needs me to take the entire backup frontend offline to unmount and remount.

Even BETTER: I have to power cycle the NAS now. I legit created a script to send the SNMP commands to toggle its outlet group remotely on the UPS, just for this. I hate this f❌king NAS.

Follow

# fsck -C /dev/sdb1
fsck.ext2: No such device or address while trying to open /dev/sdb1
Possibly non-existent or swap device?

# iscsiadm -m session -P 3
iSCSI Connection State: TRANSPORT WAIT
iSCSI Session State: FREE
Internal iscsid Session State: REOPEN
Attached scsi disk sdb State: transport-offline

· · Web · 1 · 0 · 0

The superblock could not be read or does not describe a valid ext2/ext3/ext4 filesystem.

False alarm, needed to re-log the iSCSI connection, even though it said it came back up just fine.

NAS started a disk resync (md resync) so I guess I'll just have unbearably slow NAS ops for the next.... 13 days.

Seems it's all clean. Time to retry the backups.

Sign in to participate in the conversation
Tek's Mastodon

The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!