It amazes me that pretty much anything that involves interacting with physical objects around here gets added to the 'Systems Admin' queue. Right now there's a job there to replace the filter in the kitchen's chilled water dispenser. Definitely a task that requires my expertise in Unix administration.

"Hi Mike, I need immediate access to some files that are on an old laptop that was replaced a year ago, in the home directory of someone who hasn't worked here for 4 years. It is very urgent."

Just closed a ticket that'd been open for nearly five years. I'm not expecting the delay in fixing the problem to cause any complaints, as I was the one that reported the issue.

Eliminated the last (for now) cause of occasional slowdowns on a system by slapping myself on the back of the head retroactively and adding a semaphore file check to a cleanup job.

Turns out that thinking to yourself "well it can't possibly take longer than 3 hours to run this ..." absolutely guarantees that of course once every few days it'll end up running two copies at once.

The highlight of today's incident was receiving an email from someone telling me that they were unable to send email. I'm equally amazed that a) they thought that would work and b) it did.

Had some completely nonsensical issues with mail today, which uncovered a five-year-old configuration error on a critical server, but that didn't have the common decency to actually be the cause of the damned problem.

A real situation I once had to deal with: a daemon running on a box, listening on a port exposed to the internet, that recompiled and restarted itself every time code changes were checked in to its SVN repo, by anyone in the company.

There's a special kind of anxiety just for preparing to delete a virtual machine off a server, where suddenly you no longer believe your backups work at all or that your right arm will suddenly twitch and you'll delete the wrong thing.

Just spent a bit of time scratching my head and reading NFS man pages to see if there'd been major changes that I'd missed, when it turned out I'd just typed in the wrong IP address.

*visualises a calm blue ocean*






*calm blue ocean*

*calm blue ocean*

I just had to disable a user's email account because they had a really dumb out-of-office auto reply set up that replied to everything, even itself and non-delivery reports, and I was all "Is it 1996 again somehow?"

How did I reach this low point of spending my day setting up Active Directory OU structures and deploying Group Policy settings to Windows desktops? Where did it all go wrong?

It's time to go home when you discover after 45 minutes of frustrating work that the reason this desktop computer isn't picking up the network policy changes is that its network cable isn't plugged in.

Yes, look, I do want to help you solve this issue with the payroll submission, as I have a strong desire for it to work. I am very keen to get to the bottom of this error you're experiencing.

I did not need you, however, to attach a plain text copy of all the payroll data for all staff to the email you sent about the problem.

You're lucky you sent it to a sysadmin who already has access to everything. If you'd sent this off-site, my tone would be quite different right now.

I think I finally found the cause of some performance issues that have been bugging me for months. Looks like a minor config error, basically not much more than a typo. Computers are fun!

Topic of email filtering came up this morning, and reminded me of a time I had a major government body repeatedly reject a user's mail due to "profanity" which nobody could ever find in the message.

No, it wasn't a Scunthorpe problem!

I had a backend server named "fuckchop" which would appear in the email HEADERS about half the time.

My server naming scheme at the time was "slang terms of abuse" and that kinda ended that era.

One of our new people recursively chmod 777ed his hard drive. Not going to even attempt anything other than a reinstall heh.

Kiddo loses his local sudo rights though. Gonna have to earn that back.

How good is it when you wake up and the first thing you find out is that one of your database servers isn't running and you get to debug that before you're even dressed?

Hey Fediverse - if I wanted to set up a generalised authentication backend for current and future projects, where users could self register, change passwords, update details etc, and I could have that available via a bunch of methods like oauth, LDAP, whatever ... How much is possible with off the shelf software right now? What's out there that's good?

I've searched of course, but anyone out there got personal experience in this area and feel like a chat?

The five stages of debugging server issues are denial, anger, bargaining, depression and pasting content from Stack Exchange directly into the terminal window.

Show more
Chinwag Social

Consider this a friendly, local pub. Make yourself at home, bring your friends, have a good time! Meet new people, have a laugh, enjoy the ambience, and the Oxford commas.