During the last year, however, I came to realize that even though trac is a great tool for many projects, it might not be best for my personal projects. For one, social visibility is close to zero - in part because I like to host on my own server. Second, I was wrong to think svn is good enough for smaller projects - even when working alone git has major benefits. If only for being able to work offline, worry free distributed backups and beyond human speed. Anyway, "good enough" sometimes is just a fancy argument for being lazy.

So, last week I finally decided to get over it and to try github and git as my SCM toolset of choice. To use github is a no-brainer these days - just about every day I read great stories on HackerNews or reddit/r/programming. It's not just about free hosting, that's not a real problem anyway. It's all about social visibility these days. Not having a github link on your CV will lead to certain questions.

Being on the wrong end of this discussion can be unnerving. It's not that I can't defend my point of view. It's because everyone asking me about it has been right. After having worked with github/git the last couple of days, I conclude those tools have become mainstream for a good reason.

I'm a strong advocate of kaizen - to always improve yourself a little bit. I also encourage people to not only use the tools they know, but to always seek out for the best tool for a new job. I now have to realize, that concerning SCM it was me who had the outdated mindset. trac certainly is a great tool, I won't deny that. But there is space for improvement. Time to move on.

Turns out, I'm a pretty active coder in my spare time. I worked on four projects the last few days, three of them visible on my new github account. Just smallish hacks, but hell - that's what I love about coding: always being able to tackle new problems quickly and upfront; it is not about the big bucks - those are a side-effect.

Permalink | Leave a comment  »

You can find this script on github: https://github.com/preek/gforge-mass-move-tracker-items

Why we needed this: When you implement a new custom workflow in a legacy GForge environment, it is best to do so in a template. Every new project can then be cloned from it.

Unfortunately old projects will not get updated. You have two choices now: One is to repeat enforcing the workflow on every project (which is a tedious task and can easily take up to a day). The other is to make a new tracker which is cloned from the template. Then you can move your tracker items from the old tracker to the new one. Voila, you got the new workflow rules applied.

Permalink | Leave a comment  »

While graphical package managers like Synaptic or KPackageKit certainly are great tools for the novice Ubuntu user, I personally don't see any benefit against the various apt CLI bindings. For one, I'm not a big fan of using GUIs where not needed and secondly, the graphical package managers' search features come with extra baggage - they use Xapian in a cron job for indexing, which can be quite a CPU hog.

Even though Ubuntu has good defaults and uses ionice to schedule for IO priority, this setup caused my system to hang completely. To be fair, I should mention that I run VirtualBox instances in parallel.

To disable the regular indexing, you can remove execute permission on the cron job:

sudo chmod 644 /etc/cron.weekly/apt-xapian-index

Or you can completely uninstall the tools:

sudo aptitude purge apt-xapian-index

You can also uninstall KPackageKit or Synaptic completely, but you would lose your update manager reminding you of new packages.

Permalink | Leave a comment  »

To me, fullscreen does magic in terms of gained productivity – there’s no distraction anywhere; I can perfectly focus on the work ahead. MacVim has a nice fullscreen feature(CMD+SHIFT+F). I don’t want to miss it in my Linux environments, though.

In GVIM, there is a little manual labour involved, because in Linux, the window manager is responsible for, well.. managing windows – GVIM itself can’t implement “fullscreen”.

To achieve true fullscreen capabilities in GVIM and Linux, there are two steps involved:

• Make a shortcut in your window manager for fullscreen
• Get rid of GVIMs’ menu and toolbar

I’ll demo #1 for KDE:

Get rid of the menu:

And the toolbar:

Of course, you could edit your vimrc and set those flags globally.

Now, enjoy your editing with SHIFT+ALT+F.

Permalink | Leave a comment  »

The Airport Express is a great device when used in combination with iTunes - I thought. I could sit on my couch on my MBA, do some coding work while listening to my streamed Library (Mac Mini) not through internal speakers, but through my Harman-Kardon Hifi system.

Permalink | Leave a comment  »

Permalink | Leave a comment  »

PDF files can be opened at specific points in a browser. It is possible to open them at a given page or at the beginning of a chapter. This is done by applying HTML attributes to the link URL.

Adobe has some documentation concerning start parameters for PDFs here and here.

Open via chapter name

Append the complete chapter name to your link URL.

Example: http://blog.dispatched.ch/sps/thesis.pdf#3.4%20Programmiersprache%20und%20Paradigmen

Note: Chapter names are to be encoded as is common practice in URLs. In this example blanks are encoded to %20.

Open a specific page

Append the attribute 'page' followed by the page number to your link URL.

Example: http://blog.dispatched.ch/sps/thesis.pdf#page=38

Permalink | Leave a comment  »

Previously, I’ve been using BeautifulSoup whenever I had to parse HTML (for example in my dictionary pDict). But this time I’m working on a larger scale project which involves quite a lot of HTML parsing – and BeautifulSoup disappointed me performance wise. In fact, the project wouldn’t be possible using it. Well, it would be – if I subscribed to half of Amazon EC2(;

Since the project is in stealth mode right now, I can’t say which pages I am referring to, but let me give you these facts:

• ~170kb HTML code
• W3C validation shows about 1300 errors and 2600 warnings per page

Considering this many errors and warnings, I previously thought the job had to be done using BeautifulSoup, because it is known to have a very error resistant parser. In fact, BeautifulSoup doesn’t parse the HTML directly, but splits the tags in tag-soup by applying regular expressions around them. Opposing popular stories this seems to make BeautifulSoup very resilient towards bad code.

However, BeautifulSoup doesn’t perform well on the described files. The task: I need to parse 20 links of a particular class off the page. Here’s the code which I profiled using cProfile:

Parsing 20 pages, this takes 167s on my small Debian VPS. Thats 8s+ per page. Incredibly long. Thinking of how BeautifulSoup parses, it’s understandable however. The overhead of creating tag-soup and parsing via RegExp leads to a whopping 302’000 method calls for just these four lines of code. I repeat: 302’000 method calls for four lines of code.

Hence, I tried lxml. The corresponding code is:

On the 20 pages, this takes only 2.4s. That’s only 0.12s per page. lxml needed only 180 method calls for the job. It runs 70x faster than BeautifulSoup and creates 1600x fewer calls.

When you do a graph of these numbers, the performance difference looks ridiculous. Well, let’s have some fun(;

lxml vs BeautifulSoup performance

Considering lxml supports xpath as well, I’m permanently switching my default HTML parsing library.

Note: Ian Bicking wrote a wonderful summary in 2008 on the performance of several Python HTML parsers which led me to lxml and to this article.

Permalink | Leave a comment  »

Finally, there's no more listening to this sentence anymore: "Well, I'm using Microsoft Windows. You know, that's the better OS. If it weren't, why is Microsoft the bigger player?"

My friends, the tables have turned. According to Yahoo Finance Apple has a net worth of 230B whereas Microsoft caps at 212B (08/13/2013).

There's much joy to extract from this graph from WolframAlpha (note: WA uses data from 06/30/2010. The breakthrough was imminent back then):

Now, please don't come bothering me, because Microsoft still has Windows and Office to receive the big routine paycheck while Apple has to constantly innovate to keep its pace. Depending on those two has thought Microsoft nothing but stagnation. In the meantime Apple, Google, Facebook and Twitter arose. If Microsoft doesn't change pace, they will suffer Yahoos' fate.

If you ask me, it would be for the best of it. In the field of technology not the biggest player will win in the long run, but the most innovative. Do you remember those Kodak cameras that didn't go digital (1990:7B, 2010:0$, source)? Geocities was hip once and worth 3.6B in 1999, now dead (source).

The list continues. Even when there is market domination, there still is room to manoeuvre. Microsoft didn't equip the first personal computers - and they certainly won't be the last.

Permalink | Leave a comment  »

Google AppEngine(GAE) is great for hosting Python (or Java)  Web-Applications. They offer 1.3mio hits/d and 1GB up- and downstream/d for free. Considering that you will get access to Google infrastructure that let's you crawl the web as fast as Google does itself, choosing GAE is a no-brainer for applications doing a lot of web-crawling, screen scraping or web-indexing. You can even do cron-jobs to get your job done periodically.

I won't elaborate on how to get an account, download the SDK and get started, because Google hosts great tutorials for these itself. If you are already familiar with Python web development this will get you started in a matter of minutes.

I personally chose not to use the Google webapp framework, because I'm quite familiar with CherryPy. I fell in love with it, because it feels very sleek - very Zen-like. This comes to no surprise, because it was a deliberate design decision as can be read in The Zen of CherryPy.

Getting started with CherryPy on GAE is no trouble, either. GAE supports any Python framework that is WSGI-compliant. Those include Django, CherryPy, web.py and Pylons. Google doesn't host these frameworks themselves, so all you have to do is copy the whole framework into your GAE project to get the import to work. That's it. Same counts for any 3rd party module. Need BeautifulSoup? Just copy the py-file to your project. Easy as cake.

Now, if you want to serve images dynamically, you don't have to store them on harddisk to link to them. Just save them in the Google Datastore and serve whenever needed.

Using the following snippet you will be able to dynamically serve images with URLs like this:
    http://application/handler_name/index/[0-9]*

One last note: Processes running longer than 15-30s will be cut off from GAE with the DeadlineExceededError exception. You can catch this exception and try to divide your workload into smaller pieces.

Permalink | Leave a comment  »

It's not finished yet. People won't get from it what dispatched is ought to do.

Many thanks to my great designer Katrin from http://rocketship.cc

Comments?

See the full gallery on Posterous

Permalink | Leave a comment  »

Normally, I'm not much of a petition signing person. But looking on how this world runs, I probably should step up a bit.

The petitions I signed today were:

Stop the construction of Europe's biggest slaughterhouse
http://www.vebu.de/aktuelles/petitionen/465-aktionen-und-petitionen/?pet_id=582

This new slaughterhouse would run at 7.5 deaths per second. Seriously, do you people need even cheaper meat? At a discounter, a chicken already costs about 2€. There is absolutely no margin for a cruel free life or any decent quality in the "production process". This is madness.

Stop a new animal testing lab in Malaysia
http://www.thepetitionsite.com/1/StopPlansForAnimalTestingMalaysia

We know how cruel animal testing is in Europe. How bad will it be in a lesser developed country with even fewer laws concerning animal rights? Besides, there's lots of proof that animal testing is not only cruel, but completely unnecessary (read for example "Doctors speak out against animal testing").

End the brutal killing of more than 20,000 dolphins every year in Japan
http://apps.facebook.com/causes/petitions/252

Japan is known for killing whales to study them. Afterwards they sell their meat on the market. The profits therein are of course completely unrelated to their will of studying so many whales. Due to international pressure, Japan begins to slaughter dolphins instead. 20'000 a year as a new documentary shows.

As human being and a vegetarian, probably it is not only my right, but also an obligation to make this world a better place. There is no god to do it for us. And mankind itself often is incompetent or incapable of action. I, on the other hand, have the needed time and resources to do my part. I honestly feel bad for not doing enough.

I want to finish with a small side story. Today, during my tea break, a smoker came up to me and asked me what I drank. I had peppermint tea, because I'm still suffering a little from the flew. His reaction was completely gross. He said "No wonder you're ill. You should live more hardcore! Eat more sausage and stuff. This makes you strong."

I will remind him later this year.  In Winter he will regularly go outside to have a smoke, coughing and freezing at -10C. He is a real hardcore guy. He eats sausage and stuff.

The pigs inside the cage are brought down in groups to a space full of CO2. Taken from http://www.flickr.com/photos/igualdadanimal/3806411367/

Permalink | Leave a comment  »

Searching for a viable option for endusers to reset or change their passwords, I have come up empty. Maybe I just searched wrong.

Anyway, I normally don't do any administration using web interfaces (such as Confixx, Plesk and the like), but I have come across Webmin, which is "a web-based interface for system administration for Unix. Using any modern web browser, you can setup user accounts, Apache, DNS, file sharing and much more."

Webmin supports user administration. Any account can implement the role of "Change Passwords". Unfortunately, this will enable this particular user to change any other users password. Lucky for me, Webmin is OSS and written in Perl. So a quick four line code change does exactly what I need.

This is how the Munin front-end looks after the patch. If you're not root, you can only change your own password. End of line.

See the full gallery on Posterous

If you want to recreate this setup on your machine, you can download the patch here and apply it with

$ patch /usr/share/webmin/passwd/index.cgi < index.patch

Have fun.

Permalink | Leave a comment  »

Just a little Venn diagram that came up during lunch break.

Permalink | Leave a comment  »

If you ever lacked an idea on how to spend your time at work properly, here's a fun thing to try: 0verkill - a multiplayer 2D ASCII art shooter.

Hell yeah, an ASCII art shooter, you heard right! To hell with those flashy commercials claiming to be proper games!

If you're on a Debian box (as you should), you can install it from the commandline
$ aptitude install overkill

If you're less lucky, you can also go to the author's homepage, download and build it yourself. The package will feature the game itself, a dedicated server and a level-editor. Plenty of room for a great many gaming sessions!

See the full gallery on Posterous

Notes:
1. There were more players, but I didn't managage to get my glorifying screenshot soon enough.
2. Beware of server lag. I played it in a screen session on a Linux VPS. I got a load average of up to 19.48, 7.16, 3.00

Permalink | Leave a comment  »

Permalink | Leave a comment  »

As soon as an incompetent manager gets to have only one say during a project, hell seems to break loose and stuff just gets worse.

Finally, after many sessions of hardcore reverse engineering, I have concluded the algorithm used by senior management. Hereby, I share it with the world.

def ask_manager(question="Can I dump Windows and use Linux?"):

  """ 
    Bend the universe to the magical logic of senior
    management
  """

  if True:
    globals()['True'], globals()['False'] = False, True
  if True:
   return "Do as requested."
  else:
   return "Do the opposite."

Permalink | Leave a comment  »

My grandfather has always been what I would call an all-around techie. He has built himself three houses in his lifetime, with hard work he climbed the ladder to become head of department (100+ employees) in a company working in the metal industry and he still owns the biggest private workshop I've ever seen. He knows how to build big machinery and repair everyday electronics. Heck, he didn't want to buy handrails for his house, so he built a 5meter/5ton "gadget" to cold twist 5cm thick steel. That's the man my grandfather is - a real man's man.

He will be 76 in a few months, but that doesn't slow him down. He owns a big screen Plasma and always carries his mobile phone. The only "weak spot" considering regular use of technology is computers. It has been over ten years now since he retired. Back in the day his company just started using computers for accounting. But well, what do you have secretaries for?(;

Anyway, the first computer he bought was a brand new Pentium II. Back in the day Win98 was considered state of the art and Linux was not made for n00bs. Understandingly, he never grew into it. Neither did I. But while I switched to Linux, he didn't find any good use for his computer.

When I bought my first Apple Macbook in 2007, I installed his second machine (coincidentally my previous PC). It was an IBM dual PIII workstation/monster with lots of RAM. Before it changed owners, I operated it with 6 graphics cards and 6 HDDs. I also got my grandfather a modem, a printer and a fax. The workstation ran Ubuntu 2006/04 which was a decent improvement on Win98.

Even though Ubuntu already had a pretty good user interface, usage was still clunky. He had to go to the "computer room", switch off the phone, switch on the modem, boot the PC, connect to the internet. He had to use peripherals he was absolutely not used to. I didn't think it was this hard to use a mouse, but when you're 70 and never had to use one, this device really can be a deal killer. Honestly, who invented the single left click, the double click and the right click methodology? When he kept asking me over and over when to do what, I realized how bad a design decision this really was.

Skip a few years ahead. We're in June 2010 now. Apple reveals the iPad.

I decided pretty fast I had to get this device for my grandfather. I was sure he would love to use a computer for mail, news, his favorite croatian radio program (which is normally broadcasted via medium wave - in Germany there's only reception at night), games and reading about cooking and gardening. There was just no way he would do it with a standard PC in the back of his house. But he would do it on his couch if what he wanted was only one click away. Besides, he always fancied my iPhone as a pretty cool device.

So I went to my local Apple Store and got on the priority list. A couple of weeks later I got mail saying I had 24h time to fetch it from the store. And so I did. I wanted him to have an Google account for mail and calendar, various bookmarks, apps and games preinstalled. Of course I bought the 3G version, so I got my hands on a data plan. The first three days I configured and tested the iPad in such a way that everything would just work out of the box.

Well, what can I say. The decision to get him an iPad has been an overwhelming success! He loves the device, tells me it's so much easier to use and he keeps asking perfectly good questions on stuff I have not yet taught him. And when he get's shown what to do, he will remember it and not ask the same question again.

Being the interested and always curious techie he is, he already wrote me mail, read some stuff online and played many winning sessions in Mahjong. I'm quite interested as to where this new love goes. We already upgraded and bought the iPad case - we also wanted the camera connection kit to match his Sony Cybershot, but it is out of stock in Zürich, Stuttgart and even online!

I'm happy he finally made the transition to being part-time online. And I guess he is, too.

Just my 2c on how useful the iPad can be.

See the full gallery on Posterous

Permalink | Leave a comment  »

This is how a waitress taught the three IT guys(me and two friends) during lunch break how to enjoy rich video content these days. Kinda crazy, I didn't think that such a thing could completely elude me for such a long time.

Great surplus: Only downloading is considered legal in Switzerland.

Anyways, I will keep paying for my movies on iTunes for quality, convenience and a great service. movie2k, however, has proven to be a great source of many beloved contemporary series that, not being a US resident, I couldn't watch legally otherwise.

So, thanks Ms. ? for this hint.

Permalink | Leave a comment  »

Thanks to ifixit.com it took me a mere 2h to disassemble, clean and reassemble my MB. The process taught me a decent humbleness before those who do this for a living - this computer is definitely _not_ built with frequent modification in mind.

If only I had found a any dust to clean out, the work might have been worth the effort. In the two+ years of heavily working with the MB (also in potentially "dirty places" like couches, beds and cars [pun intended]), there was close to no dirt accumulation. Either that's due to a really good design or I can easily start to mess my apartment a little more up.

Anyway, two screws will never find their way back into the MB. One is lost due to the bolt thread being fragged by unscrewing, the other I just forgot to put back to where it came from. Well, what good (or bad) can such a tiny screw do?
At least the MB hasn't become louder in the process.

See the full gallery on Posterous

Permalink | Leave a comment  »