Archive for the 'Coding' Category

git: Rebase vs. Rebase

For a some time I’ve wondered about git rebase. At some point in the past I’ve realised that there is not one use in git rebase but (at least) two distict ones.

  • On the one hand git rebase is used to rebase a branch onto an updated source branch (source like in origin, not in source code).
  • On the other hand it’s used to rewrite commit history.

What’s rebase?

A rebase makes git rewind your commit history up to a certain point and than re-apply your patches onto another starting point. The starting point depends on the exact type of rebase you do and what you tell git.

Rebase a branch

Why would you want to rebase a branch onto another one and what does it do?

If you have a feature branch and want this to be merged with your master branch you could of course merge it. If you do so and your feature branch is not based of the last commit in master git will create a new merge commit since it has to merge two branches of a tree. If you’d rebase your feature branch onto the tip of master you’d have a linear merge history w/o a merge commit instead.

Rewrite a branch

Why would you want to rewrite your commit history? What if you make mistakes while crafting a new branch? If you’re following common VCS advice you’ll probably commit often. Probably between some mistake you’ve made and the time you did correct it.

Now you’d have your shiny new feature commit and two or more commits w/ bugfixes to your initial commit in this feature branch. Some people prefer to keep the history of their mistakes. Those could just merge this new feature branch into their master branch.

Others prefer to keep their commit history clean (you wouldn’t release a book which includes all mistakes and corrections you’ve made, would you?). Here comes git rebase -i to the rescue. This rebases the current branch onto itself. This may sound a bit silly but it allows you to drop commits or more importantly to combine (squash) commits together!

Where to got from here?

This post wasn’t meant to give a complete introduction to git rebasing. There are plenty of great tutorials out there. I did just want to highlight some points which were important to me.

Migrating to PSGI/Plack

Have you heard of PSGI/Plack?

It’s that awesome Perl meta-webframework you’re looking for. Unfortuneately it’s not that easy to get started with because the documentation is a bit too euphoric. They talk about “superglue” and “duct tape for the web” but fall a bit short of explaining how to get started with it.

Shortcut: It you’re using one of the supported frameworks (like Catalyst oder CGI::Application 4.5+) you shouldn’t need to work about anything of this. How to get those running w/ PSGI is explained in sufficient details in various docs.

This post is for those who need to do it w/o an framework or want to know the internals. Well, I’m not going into great detail in this post, only the essentials.

  • What is PSGI? PSGI is a specification of a protocol spoken between a PSGI compliant Server and the WebApp. The WebApp is talking PSGI to it’s executing server and this server is talking whatever he likes to his downstream (e.g. HTTP, FCGI, …)
  • What is Plack? Plack is a set of tools implementing PSGI. It provides some PSGI compliant Server implementations on its own and help others building their PSGI servers. It’s also some kind of meta-framework (“middleware”) that implements such things like Session management, authentication and compression. You don’t have to use those features! You don’t even need to known that they are there, but if you want you can have a look.
  • What is $env? This is a HashRef containing the environment for your request? Why env, you ask? Well, probably because CGI get it’s parameters passed via the OS ‘environment’ and they just adopted that for PSGI.
  • What is the PSGI-triplet? Well, that’s just the name I gave to the data structure PSGI expects to get returned after a request has been processed. It contains the HTTP-Statuscode, an ArrayRef with the HTTP-Header pairs and an ArrayRef containing the Body.

So, how do I migrate my framework-less perl web application to being PSGI compliatnt?

  • Throw out CGI, CGI::Carp and possibly FCGI. You won’t be using them anymore. You’ll be using Plack::Request instead.
  • Make your App “persistence-compatible”. That means you’ll have to abandon any global (class) variables thats only valid for one request. Every class variable must be valid throughout the entire runtime of your server  (because your class is instantiated only once, at the PSGI-Servers startup. There are excpetions to this, but keep it as a rule of thumb). Every information that is only valid for one request must be passed between the methods. If you have much pass around put if into a custom request class or a HashRef.
  • Make sure your class has a method handling the request. You probably have that already, but you should name it ‘run’. It will get passed the $env. Use that directly or create your custom per-request data structure from there.
  • Remove any direct output to STDOUT. Make your app return the PSGI-triplet to the caller. Everywhere.
  • Create a webapp.psgi, see below for it’s content and possible a webapp.pl if you want to support plain old CGI.
  • plackup webapp.psgi

webapp.psgi

#!/usr/bin/perl
use strict;
use warnings;

use lib '../lib';

use MyApp::Web::Frontend;

# The important part here is to instantiate your WebApp Class before
# the closure is defined. Everything inside the sub is executed on each
# request. If you instantiate your class inside you'll loose any benefit
# you get by not using CGI.
my $Frontend = MyApp::Web::Frontend::->new();
my $app = sub {
    my $env = shift;

    return $Frontend->run($env);
};

webapp.pl

#!/usr/bin/perl
use strict;
use warnings;

# Warning: This script will only work properly when invoked with
# the correct environment. Plack::Loader tries to autodetect the
# proper server and will use CGI only if certain CGI environment
# variables are set. It will most specifically not work properly
# when run from the commandline.
use Plack::Loader;

my $app = Plack::Util::load_psgi("webapp.psgi");
Plack::Loader::->auto->run($app);

Stop writing useless programs

Stop-writing-useless-programs

It’s kinda rude, but they aren’t entirely wrong …

VBoxAdm 0.1.15

I’ve just uploaded VBoxAdm 0.1.15. It includes another set of major refactorings. Please be careful when upgrading and watch your logfiles closely for any errors that may occur. Especially the Vacation module was refactored.

The time when the project will leave it’s alpha stage is drawing closer. VBoxAdm is now running on several largish sites under my direct administrative control, so I’ve got plenty of possiblities for some real-world testing. Several other migrations/installations are planned for the near future. Once it has proven sufficiently stable on these mailservers I’ll announce the the end of the alpha phase and enter beta testing.

Stay tuned!

GIT: Rewriting commit history – change author and email

Quicklink: How to change author and email in the whole commit History: http://theelitist.net/git-change-revision-author/

Memcache Sessions may cause zend_mm_head corrupted in PHP5

PHP5 (5.3.3 in this case) may break in very supprising ways if the Memcached configured as a session handler goes awry.

In my case there was a webserver (Apache2 + mod_php5) w/ two memcached configured as session handler. One of those memcached got stuck and didn’t properly reply to request. This shouldn’t happen but whats even worse was that PHP5 just “died” with the following error in the syslog:

vs-www-s01 suhosin[32504]: ALERT – zend_mm_head corrupted at 0x7fc2f41a2090 (attacker ’10.1.2.3′, file ‘/var/www/index.php’)

This resulted in empty pages delivered to the browser.

After fixing the memcached everything was fine again.

Perl UTF-8 Checklist aka Surviving the Perl Unicode Madness

Some time ago, when I wrote the first version of this post I thought I had mastered UTF-8/Unicode with Perl and MySQL. Sadly I was very, very wrong. So I had to revisit the topic and I’d like to share my findings in the hope that they can save some coders from going nuts.

First you should read “Why does modern Perl avoid UTF-8 by default?” on Stackoverflow, especially the top-voted answer. It is the best ressource on UTF-8 and Perl I’ve found so far.

The next stop would be “UTF8, Mysql, Perl and PHP” on gyford.com. Pay special attention on the “utf8::decode( $var ) unless utf8::is_utf8( $var );” part. However I’d suggest using Encode::decode and Encode::is_utf8 instead. The imporant lesson to take away here is that you still may need to “decode” the bytes coming from the database into Perls internal UTF-8 representation. Once Perl knows its dealing with UTF-8 it will probably handle them correctly. Unfortunately sometimes the conditional decode doesn’t work … in this cases you can try to decode the data w/o checking if it is already in UTF-8 first. Brave new world …

If you still need more advice I suggest the following links, in this order:

VBoxAdm: Mailinglist and API

A short status update regarding VBoxAdm.

Mailinglist

Finally I’ve created a Mailinglist: http://www.vboxadm.net/support.html#mailinglist

Refactoring

I’ve been refactoring the Code for a while to turn it more into a MVC-Shape. This means separating the Model from the Controller (former VBoxAdm::Frontend, now VBoxAdm::Controller::Frontend). The ultimate goal of this work is to support code reuse and support for multiple ways to manipulate the data. Once the Model classes are stable I’ll finish the command line interface as well as the HTTP-API. This will provide three ways to modify the underlying data:

  • Web Frontend
  • HTTP-API (no REST for now, maybe later)
  • CLI

The Mailarchive is postponed for the time being.

Auto Configuration

Most Mailclients, like Outlook, Thunderbird and KMail, support a way of client auto-configuration. When setting up a new mail account they request a certain URL derived from the mail address and if they find an XML document with the expected information there they’ll use this information to set the correct username, mailserver and protocols. Support for this was added recently. There is even support for the weird way MS Outlook does this. However Outlook support is, so far, based solely on the documentation on Technet. Due to the lack of a Outlook license I wasn’t able to test it. Please provide feedback.

Future Work

After the refactoring, API and CLI are finished I’m going to look into the Mailarchive again. After that I’ll look into Quota Support, Mailman integration and I’d like to find a way to get the Postfix Logs into the database to ease support work. Having the Log in the database in a parsed format – no raw syslog to db – would make support request more easy to handle. No more need to log into the server and grep through the mail.log.

Further feature request are always welcome. Please direct any ideas and comments to the mailinglist at vboxadm@vboxadm.net.

Handling salted passwords in Perl

While working on VBoxAdm I’ve came to a point where I did want to support salted passwords. Salted passwords however are a bit difficult since you need to extract the salt from the existing password hash and compare it to the user supplied password to hash it and compare the result. You can’t simply hash the supplied pass and use a random salt, that would produce different hashes.

If you’re not into salted hashes, I’ll try to explain how salted hashes are generated, why they are and how you can handle them.

Continue reading ‘Handling salted passwords in Perl’

reStructured Text

I’ve discovered reStructured Text some time ago and have been using it since then for several tutorials, like the ISP Mailserver or the Rootserver ones.

http://docutils.sourceforge.net/docs/user/rst/quickref.html

I’d suggest to have a look at those if LaTeX would be too much of an overkill.