neuhalfen.name

A random collection of posts

Your Data Is Corrupted, and You Don't Know It

Permalink

Chances are, that you have corrupted data stored on your server. And written to your backup. And you'll only find out, when there is no correct copy left. But help is on the way. Kind of.

Be warned: this is a rather long piece of work. I finally came around to put my diploma thesis on github, and this is a mixture of Introduction Into Memory Management, Computer Hardware, and my thesis.

The target audience for the first sections is the interested layman, later on it gets more technical. If you are really interested read the diploma thesis.

But don't worry, the post is structured and gradually gets more technical. I tried to keep technical mumbo-jumbo to the absolute minimum, and tried to explain as much, as possible.

TL;DR

Memory errors are bad, bad, bad. And they happen quite frequently, each year ~1/3 of Googles servers showed them. Use ECC memory in all your computers, if supported. Run a memory test program every now and then. Watch out for unexplained program crashes. Make backups.

As my diploma thesis I wrote a hybrid (kernel, userspace) memory tester for Linux that solves the main problems of current memory testprograms:

Reliably test most (~70%, YMMV) of the computers memory while still beeing able to use the computer for productive work.

If the headline made you wondering “Do I have corrupt data? Why do you think I have corrupted data?”, or if you just want to learn a bit about RAM in general, or the Linux memory management: read on.

Keeping Dotfiles Organised Over Multiple Machines

Permalink

Like many long term users of Linux/Unix/Mac, I have grown a fair amount of dot-files (.vimrc, .bashrc, .muttrc, …) over the time. Each tailored to my specific needs, e.g with a git enabled prompt, and custom colours.

As long, as there was only one (main) machine, this was good and well. As the complexity of the dotfiles grew, so did the number of machines/virtual machines I wanted these settings to be used on. I started copying over files. Then I switched to plain git to keep the history, and to rollback changes. Later I switched to homesick.

More machines also meant more environments/operating-systems. Throwing in somewhat special systems like Solaris or MacOS, my dotfiles got littered with if ... then ... else and case statements.

Things got worse when the systems had different roles, or were configured for different environments. Say I did not want my personal .muttrc on my company machine. Or I wanted to configure $PATH different for a specific system.

Long story short: I have been through that, wrote a bunch of scripts that help me to keep things organized, and put them on github.

First Aid When Gitlab HTTP 500 / Refuses to Create Repositories

Permalink

GitLab is a GitHub like git-repository manager, and more. It supports “repository management”, issue tracking, wiki, groups, and even LDAP authentication . Git is managed by gitolite .
Sometimes creating/moving repositores goes wrong and repositories are no longer available for push/pull or are not moved in gitolite (e.g. the ui shows the new location, SSH the old or both). As of 4.2 GitLab brings some helpers for this.

Empty Albums in Gallery 3.0.4: Solved

Permalink

The update to Gallery 3.0.4 leaves some albums empty, the only error is a cryptic [name] => illegal_data_file_extension in the logfiles. The culprit is that Gallery 3.0.4 disallows dots in the filename . It seems that the issue is resolved in 3.0.5 — which is yet to be released. To fix my installation I wrote a script to rename the files for me on disk, and in the database.

  1. Related issue: Dots in filenames disallowed
  2. I am not the only one

Creating and Installing an Solaris 11 IPS Package for the Zabbix Client and Server

Permalink

Besides several Linux VMs, I am using Solaris 11 for some of my home-tinkering systems. After an undetected three day outage of my mail server, I decided to install a monitoring solution for my systems.

Unwilling to go through the configure, make, make install triple jump every time I install a system, I set out to create a local IPS repository to serve my needs (and packages).

This tutorial consists of multiple steps

  1. Create and set up a local IPS in a dedicated zone
  2. Create a simple IPS package
  3. Create the Zabbix package and publish it (this article).

Creating a Simple Solaris 11 IPS Package

Permalink

Besides several Linux VMs, I am using Solaris 11 for some of my home-tinkering systems. After an undetected three day outage of my mail server, I decided to install a monitoring solution for my systems.

Unwilling to go through the configure, make, make install triple jump every time I install a system, I set out to create a local IPS repository to serve my needs (and packages).

This tutorial consists of multiple steps

  1. Create and set up a local IPS in a dedicated zone
  2. Create a simple IPS package (this article).
  3. Create the Zabbix package and publish it.

Setting Up a Solaris IPS Server

Permalink

Besides several Linux VMs, I am using Solaris 11 for some of my home-tinkering systems. After an undetected three day outage of my mail server, I decided to install a monitoring solution for my systems.

Unwilling to go through the configure, make, make install triple jump every time I install a system, I set out to create a local IPS repository to serve my needs (and packages).

This tutorial consists of multiple steps

  1. Create and set up a local IPS in a dedicated zone (this article).
  2. Create a simple IPS package
  3. Create the Zabbix package for Solaris 11 and publish it.