TLDR;

Using the woff2 format and font subsetting (eg. via glyphhanger) you can get small font files for hosting web fonts yourself.

Motivation

Lately I’ve found a beautiful mono-spaced font called Iosevka (I like the etoile variant best). For coding I still prefer Cascadia Code but for normal reading I do refer it over cascadia code.

If you actually download the font, you’ll find that the release zip file is ~120MB (or in the ballpark, depending on version), a single ttf file is around ~10MB and even the woff2 version (which is a format already compressed and optimized for usage on the web) is ~2MB in size.

As a custom font is hardly a necessary feature of any webpage and at best a nice-to-have, 2MB is waaay to big for a vanity download.

Considering that in my use case I would probably only need ~60+ chars (10 digits + 26 letters + 26 more in uppercase and a couple of special characters) I wanted a smaller file that only contains the characters that I am using.

Font Subsetting

Turns out that the process of creating a smaller font from a subset of codepoints is called font subsetting (duh!) and there are a couple of nice tools like glyphhanger, which can do that for you if you give them a website to crawl or a list of the characters required.

Putting it together

So I threw together a short Dockerfile that sets up glyphhanger with woff2 export support and a script to instruct it to generate a woff2 subset with the whitelisted characters here.

Dockerfile

The docker file installs glyphhanger via bun and the prerequisites for the enabling the woff2 export:

FROM oven/bun:alpine
RUN apk add python3 py3-pip py3-fonttools py3-brotli py3-zopfli
RUN bun i -g glyphhanger
ENTRYPOINT [ "glyphhanger" ]

Script

The following script volume maps the current directory (which should contain the input ttf files) into the container and instructrs glyphhanger to create a font subset in woff2 format:

docker run -v $(pwd):/home/bun/app glyphhanger --whitelist="/.- 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ" --subset=*.ttf --formats=woff2

Running that with my favourite font files, I get the following output files that are ~69KB in size:

-rw-r--r--   1  9.6M Iosevka-Regular.ttf
-rw-r--r--   1   68K Iosevka-Regular-subset.woff2
-rw-r--r--   1  9.6M IosevkaEtoile-Regular.ttf
-rw-r--r--   1   69K IosevkaEtoile-Regular-subset.woff2

Which I find adequate in size for some eye candy.

References

• Glyphhanger (https://github.com/zachleat/glyphhanger)
• Iosevka (https://github.com/be5invis/Iosevka)
• My Glyphhanger Dockerfile and Example script (https://github.com/8/glyphhanger-docker)

TLDR;

I’ve updated the inkscape palette file for tailwindcss 3.4.5.

Background

Some years ago, I’ve created a custom palette file that can be used with inkscape (and gimp) and wrote a short article about it.

Since writing that article, tailwindcss has evolved and changed their default color palette and so I wanted to update the palette to use the new colors in inkscape.

But meanwhile tailwindcss also changed how they generate the colors, which broke the code I wrote back then to generate the palette.

Today I rewrote that script that generates the palette.

The Script

This time the script is just a single fsharp file.

1. It will grab the latest version of tailwindcss (3.4.5 at the time of writing) from the cdn
2. and try to parse the colors and
3. create a file named tailwindcss.gpl

module Tailwind =
  open System
  open System.Net.Http
  open System.Text.RegularExpressions

  let getJs () =
    let url = "https://cdn.tailwindcss.com"
    let client = new HttpClient()
    client.GetStringAsync(url).Result

  type Rgb = {
    R: byte
    G: byte
    B: byte
  }

  type ColorValue = {
    Value: int
    Rgb: Rgb
  }

  type ColorGroup = {
    Name: string
    Values: ColorValue array
  }

  let getColorGroups js =
    let r = Regex """(?<name>\w+):{((?<weight>[0-9]+):\"#(?<value>[0-9a-f]+)\",?)+?}"""
    
    let getColor (m: Match) =
      {
        Name = m.Groups["name"].Value
        Values =
          m.Groups["weight"].Captures
          |> Seq.mapi (fun i c ->
            let rgb = m.Groups["value"].Captures[i].Value 
            {
              Value = c.Value |> int
              Rgb = {
                R = rgb[0..1] |> Convert.FromHexString |> Seq.head
                G = rgb[2..3] |> Convert.FromHexString |> Seq.head
                B = rgb[4..5] |> Convert.FromHexString |> Seq.head
            }
          })
          |> Seq.toArray
      }
    
    js
    |> r.Matches
    |> Seq.map getColor
    |> Seq.toArray

module Gimp =

  open System.IO

  type RgbColor = {
    Name: string
    R: byte
    G: byte
    B: byte
  }
  module RgbColor =
    let from name (r, g, b) =
      {
        Name = name
        R = r
        G = g
        B = b
      }

  type Palette = {
    Name: string
    Colors: RgbColor array
  }

  module Palette =
    let create name colors =
      {
        Colors = colors
        Name = name
      }

    let write (stream : Stream) (palette : Palette) =
      
      let formatColor (color : RgbColor): string =
        let formatPart p = p.ToString().PadLeft(3)
        $"{formatPart color.R} {formatPart color.G} {formatPart color.B} {color.Name}"
      
      use sw = new StreamWriter(stream, NewLine = "\n")
      sw.WriteLine "GIMP Palette"
      sw.WriteLine $"Name: {palette.Name}"
      sw.WriteLine "#"
      for color in palette.Colors do
        formatColor color |> sw.WriteLine
      ()
      sw.WriteLine ""

    let writeToFile file palette =
      use fs = File.OpenWrite file
      write fs palette


let tailwindColorGroups =
  Tailwind.getJs()
  |> Tailwind.getColorGroups

let gimpColors =
  tailwindColorGroups
  |> Array.collect (fun colorGroup ->
    colorGroup.Values
    |> Array.map(fun colorValue ->
        Gimp.RgbColor.from $"{colorGroup.Name}-{colorValue.Value}" (colorValue.Rgb.R, colorValue.Rgb.G, colorValue.Rgb.B)
      )
    )
  |> Array.append [|
    Gimp.RgbColor.from "black" (byte 0, byte 0, byte 0)
    Gimp.RgbColor.from "white" (byte 255, byte 255, byte 255)
  |]

let palette = Gimp.Palette.create "Tailwind CSS 3.4.5" gimpColors

Gimp.Palette.writeToFile "tailwindcss.gpl" palette

How to use it

1. To run it you can copy/paste the script down below into a file name tailwindcss2palette.fsx and then run:

dotnet fsi tailwindcss2palette.fsx

You need dotnet installed for this to work.

This which will generate the tailwindcss.gpl file for you.

Alternatively you can simply download tailwindcss-3-4-5.gpl from here.

2. Put it in your inkscape palette folder.

In my previous article I explain that step in more detail.

Also you might want to take a look at the inkscape manual.

TLDR;

To-do lists are a simple, efficient tool for tracking open tasks. However, (mis-)using them as the sole driver of daily work can lead to several problems, which can be mitigated by using an additional time-management technique like timeboxing.

The problem with to-do lists

To-do lists are everywhere - and with good reason; they are a very simple and effective way of keep keeping track of things. Yet, they are often used as the primary way to drive work - which is something entirely different!

If your only strategy to get work done is opening up your to-do list, staring at it and then trying to complete the items one-by-one, you may find yourself struggling with the following problems:

• Everything needs to be a task
• Struggling with tasks of varying size
• Procrastination
• Anxiety & Stress
• Staying motivated

Let’s look at each of these problems in detail and explore their origins.

Everything needs to be a task

Let’s start with the obvious one: If you’re only working from your to-do list, you are kind of required to create a task for what you are working on. But there are different kinds of work that don’t fit well with this approach.

For example, open-ended tasks that lack specific completion criteria fall outside the traditional to-do list framework. Activities such as brainstorming, research, optimization, marketing, editing and design or styling improvements are examples of tasks where the more effort you invest, the more you can achieve.

When dealing with this kind of work, it becomes challenging to determine when to consider it complete and move on.

Another example is tasks which require some amount of research before you can begin. Let’s say your task is to obtain X. First you have to figure out where you can find X and its cost. Alternatively, you may discover that X requires Y and you need to obtain Y first. Maybe it would be more cost-effective to built X yourself?

It may require an investigation task, such as “Find out how to get X”, that is put first and it’s output are the follow-up tasks. But it could also turn out that getting X is straightforward and could be completed during the investigation itself. In either case, a to-do list is not necessarily the best fit for managing these types of tasks.

Struggling with Tasks of varying size

Other problematic categories are lots of small tasks. Creating a task for each chat or e-Mail message might seem overkill - until you have to answer that 20 question e-Mail that requires careful thought.

Do you create eight tasks for each message where seven are trivial and creating the to-do item takes more time than answering them directly, while the last one turns out to be a biggie? Or do you create one task “reply to e-mails” and then get stuck on the last e-Mail making it impossible to complete the task at all? Alternatively, do you go back and dynamically split them up then in someway?

It’s not that there a no viable solutions to these kinds of problems, it’s just that those solutions are only geared towards pleasing the to-do list process. Unfortunately they don’t bring any benefit on their own and you end up multitasking between the task at hand and updating the to-do list.

Procrastination

Some tasks are more fun than others, but some are a real bother. Maybe because the task is unclear or it’s really difficult and it’s chance of success is very low. It could be that the task is gargantuan and even looking at it seems daunting. The exact reason doesn’t really matter.

If you are driving your work solely with a todo list, there will be tasks on that list that produce more internal friction and require more effort to even get started. If the effort required to start a task exceeds the available energy, the task will be procrastinated on.

Anxiety & Stress

To-do lists have a tendency to grow fast, as adding an item is far quicker than completing one. Working off a to-do list for hours on end and see it only grow can induce stress or feel overwhelming.

Continuously looking at the to-do list and thinking about future tasks while working on the current one can lead to unproductive task switching and further increases anxiety.

Staying motivated

Staying motivated (or managing your dopamine levels) is very important to continuously produce good work without a crash in motivation. Motivation is highest right before receiving a reward and lowest if the next reward is far in the future.

With a to-do list, the reward is often crossing out the item, marking it done, dragging it into the “completed” column. An additional reward could be a small break, like getting a cup of coffee or tea or a quick walk.

The issue here is, that the reward is handed out by the completion of a task. This makes the size of the task, which is to a certain degree arbitrary, really important. Too small of a task and we get to many rewards and breaks, which may also lessen the impact of them and make them feel unimportant.

But a task that takes too long to complete is also a problem. Because the reward is far in the future, motivation is low and we without rewards, missing the breaks needed to prevent exhaustion and maintain performance. One could grind a full day or more before marking completing a single task.

Using Timeboxing to Tame the To-Do list

One effective way of addressing the shortcomings of the to-do list based workflow is to switch to a timeboxing approach. Instead of committing to solving specific tasks, we dedicate pre-allocated periods of time to work on them. This results in a shift from focusing on outputs to efforts.

A focus on efforts - not outputs - has some major benefits, the biggest being that our effort is fully under our control, while completion of a task is not. That’s why focusing on efforts is the core skill needed for a growth mindset.

Additionally it comes with a couple of benefits that help alleviate the to-do list based problems mentioned earlier, because it:

• Provides the freedom to ignore the to-do list when it’s not a good fit.
• Sets time limits on open-ended tasks.
• Remains indifferent to tasks of varying sizes, eliminating the need to micro-managing the to-do list.
• Reduces procrastination by committing only to a reasonable effort, regardless of the difficulty of a specific task.
• Alleviates anxiety and stress by prioritizing effort and incorporating breaks at regular intervals.
• Keeps motivation high by rewarding you for your effort rather than random task sizes.

Summary

By using a to-do list as a daily driver of work, we are putting it in charge of our anxiety and stress, our feelings of achievement and our motivation.

If we do that, we are forced to obsess over the number and size of each task, carefully adjusting it to keep it’s effect on our psyche in check.

A better approach is to use the to-do list for what it was meant to be - a simple way to track open tasks and use a different technique to drive our daily work - like timeboxing - which does a much better job at that.

References

• Wikipedia - Timeboxing
• Wikipedia - Pomodoro Technique (https://en.wikipedia.org/wiki/Pomodoro_Technique)
• Wikipedia - Growth Mindset
• Youtube - Hubermanlabs - Episode about Growth Mindset

TLDR;

To receive global HotKeys in Windows, your application needs to process Window Messages and therefore will need a message loop. To do that, I’ve written a very small library that you can use or take a look at here. Works for applicationTypes incl. AvaloniaUI, Console, WinForms and Wpf.

Who needs Global HotKeys?

In my last blog post I started writing a small Example Application named ‘Timeboxing’. I wanted to extend it now with a couple of features, one of which is starting and stopping the timer with a HotKey.

Handling keyboard input is easy in any application, but only if the application has focus. If it has not, you usually need to give it focus first.

Getting Focus is usually not handled by the application itself, but by the Window Manager of the platform, so if we want to add that feature, we will need to make use of some platform specific APIs. As my knowledge about other Desktop APIs except Windows is limited at best, I will only touch on windows.

If we take getting focus into account for starting an action, the following options come to mind:

• ALT+TAB + Keyboard Input
• Win+[0-9] + Keyboard Input
• Create an AutoHotKey script
• Use an existing HotKey library
• Start a second application instance that notifies the first and then exits
• Use the Windows API to register a HotKey the proper way

Given those options, you can probably already guess what I have opted for, but let’s try to keep the facade where I first list perfectly good options, before I find somewhat contrived reasons that help me rationalize my decision to write some code use the best approach.

ALT-TAB

The most prominent UX concept of Windows for switching between applications is of course ALT-Tabbing between applications. It allows you to quickly switch between the most recent applications. It works well, if you are continuously switching between two or so applications.

So the idea is, you are working on a thing, you hit ALT+TAB, you hit an application wide HotKey to say, start the timer and then you hit ALT+TAB again and continue working.

The thing is, Timeboxing is not an application that you have in the foreground, you probably don’t even have it as a recent application. That’s because it’s basically a countdown timer that you start and forget about for almost half an hour before you come back to it.

So it’s unlikely to be your most recent application, it’s just somewhere in that deep stack of IDEs, text editors, console windows and the many, many separate instances of browser windows that for whatever reason must not be allowed to mingle.

And with that what the user actually needs to do is:

1. Press ALT+TAB,TAB,TAB,TAB,TAB,Shift+Tab
2. Press the application-wide HotKey
3. ALT+TAB to continue working

Win Key + Number

Another option is that the user pins the application to the taskbar and then uses Windows Key and the 1-based index of the taskbar icon to start the application - or - brings it to the front in case the application is already started.

In comparison to the ALT-TAB approach, this one has the advantage that the HotKey is not context dependant, that is, it does not change depending on what app was active before. The disadvantage is, that it only works for Win+[0-9] so there are only 10 HotKeys and it vies with other applications for a spot.

An additional issue that it shares with the ALT-TAB approach is, that you can only activate the window - you cannot transport any additional information with it. It’s not a general HotKey, it’s just a “Focus it” Action and you need to tell the application what you want it do it in an additional step.

That’s fine, if it’s your IDE, but if it’s just a single action that you want to start, let’s say a Timer that you want to start, what could have been a single HotKey is now a sequence of:

1. Press Win+Number
2. Press the application-wide HotKey
3. Press ALT+TAB to continue working

Shortcut + Single Instance

Another option is some combination of Shortcut Link with arguments, parsing said arguments a single instance check and some inter-process communication.

An example of what we would need to do could be:

• Parse the command-line arguments at startup of the application
• Check if another instance of the app is already running
  • If not, start normally and execute the actions specified in he arguments
  • If there is another instance, use some form of IPC to tell the other instance what to do, then exit.
• Setup a shortcut that starts the app with the arguments.
• Make the shortcut executable via a hotkey, for example pinning it to the taskbar or create the Shortcut on the Desktop and set it’s Shortcut key in the property page, which is basically a CTRL+ALT+[a-z0-9] Key combination.

This approach is more flexible than just activating your application - you can get specific information to the application and act on it. One downside is the dependency on external links and the limited number of usable HotKeys.

AutoHotKey

For applications that I often use, I’d like to create a AutoHotKey script that focuses them when I press a custom HotKey, e.g.:

; active windows terminal on ctrl-;
Ctrl & `;::
  WinActivate, ahk_class CASCADIA_HOSTING_WINDOW_CLASS

Once you have it active, you can extend the script to click the buttons that you want and then switch back to the original application that had the focus earlier.

That’s a nice solution! You can cut down the sequence you need to do to a single HotKey combination. It works great for third party applications that you use yourself. I would like to provide a similar experience for my little pet application, but preferable without actually bundling it with an AutoHotKey script.

Looking at HotKey Libraries

To integrate that functionality in your app, you need to receive a global HotKey event somehow and after a quick web search, I found two libraries which promised to do what I wanted.

But upon closer inspection I found that they didn’t work for my scenario: one library depended on Wpf (and had a restrictive license) and the other one was dependent on either WinForms or Wpf.

That means that if you don’t want either of those dependencies - say because you are using an alternative UI Framework or a Console App, you are out of luck.

Why WinForms or Wpf?

The question then becomes: Why do those libraries depend on WinForms or Wpf?

The answer lies in the API for handling HotKeys on Windows - which consists of two Win32 functions, one for registering and one for “unregistering” a hotkey (RegisterHotKey / UnregisterHotKey).

After calling RegisterHotKey() and supplying the function with a Window Handle and the HotKey in question, the system sends a WM_HOTKEY Windows Message to the Window specified by the Handle every time the HotKey is pressed until UnregisterHotKey() is called.

How do you create a Window?

So to actually receive the notification Message, you need a Window that handles the incoming HotKey Window messages.

To create a Window, you need to call either CreateWindow() or CreateWindowEx() and supply it with a couple of arguments, the most important being the Class of the Window that should be created.

The followup-question then becomes how do you create a Window Class?

How do you register a Window Class?

By calling either RegisterClass() or the RegisterClassEx() function and supplying it with a reference to an instance of the WNDCLASSW or the WNDCLASSEX struct respectively.

As one of it’s most important fields, the struct holds the Windows Procedure that gets called when a Window Messages is dispatched to a window of that class.

So before we can fill the struct, we need to implement the Windows Procedure in question, which is a delegate of type WNDPROC.

How do you write a Windows Procedure?

Even though a Window Procedure needs to handle a lot of different messages - not only the WM_HOTKEY one that we are interested in - the implementation is quite easy, because we can delegate (hehe) most the work to a default implementation called DefWindowProc.

So our implementation of the Windows Procedure will only handle WM_HOTKEY messages and fire a HotKeyPressed event if that is the case and forward any other kind of Window Message to DefWindowProc.

Phew, that’s almost all we need. There is one more thing that we need to do - we need to have message loop that reads messages from the message queue and dispatches it to the windows procedure.

How do you implement a Message Loop?

Luckily the process is very well documented - the message Loop itself is very simple, it basically consists of an while loop that reads messages from the message queue by calling GetMessage() and dispatches them to the Window Procedure by using DispatchMessage().

But that complicates matters somewhat - a message loop needs to handle Window messages continuously and combined with the fact that GetMessage() blocks as long no messages are available and that it can only retrieve messages that are associated with the same thread, we will need a dedicated thread.

So to summarize, what we need is the following:

• a Message Loop
• a Window Procedure
• call RegisterClass
• call CreateWindow
• register HotKeys
• unregister HotKeys
• raise an event on pressed HotKey

…and most of that needs to happen on the same, dedicated thread.

I think that kind of answers our previous question why the libraries take a dependency on Wpf or WinForms - they want to reuse the creation of Windows and the MessageLoop implementation.

How can we put it together?

Now that we finally have all the parts, we can assemble them into the separate operations that we need to handle, I came up with the following operations:

• Initialization
• Register a HotKey
• Unregister a HotKey
• Receiving a HotKey
• Cleanup

What do those Operations do?

Initialization consists of spawning a dedicated thread, creating a Window Procedure, registering a class, create a window and enter the message loop.

Registering the HotKey is basically just calling RegisterHotKey(). The only complication is, that it must happen on the thread that created the window.

Unregistering the HotKey means calling UnregisterHotKey() with the same caveat that it must be done by the thread that created the window.

Receiving the HotKey means reacting to the WM_HOTKEY message and raising an event.

Cleanup means Unregistering all Registered HotKey, destroying the Window and unregistering the class.

How does the Implementation look like?

Initialization

As for the API, I chose to write a class named HotKeyManager, which in it’s constructor handles the initialization as shown in the following diagram:

HotKey Registration

Registering the HotKey is exposed by a member called Register(). To satisfy the requirement that only the thread that created the window can call RegisterHotKey(), the Register() member does not call RegisterHotKey() directly, instead it sends a custom Windows Message containing the details of the HotKey (VirtualKeyCode and Modifiers). Calling the RegisterHotKey() function is then done by the Window Procedure running on the Message Loop Thread.

HotKey Unregistration

Unregistering the HotKey also happens by sending a custom Windows Message. But instead of a corresponding Unregister() member on the HotKeyManager class, I chose to return an IDisposable instance as the return value of the Register() member, which simplifies the API and the implementation.

HotKeyPressed event

In addition to the aforementioned custom Window Messages, the Window Procedure also handles the WM_HOTKEY message, that the HotKeyManager exposes over the HotKeyPressed event.

Visualizing the Message Loop

I’ve tried to visualize the general behavior of the message loop in the following image:

Where is the code?

Glad you asked! I’ve uploaded the code of the mini-library to github here. The library code is mostly a single F# file - only the definitions used for interfacing with the Win32 API are living in separate files.

How can we use the code?

The following is the code for a simple C# Console Application that registers 2 hotkeys and prints a message to the Console whenever a HotKey is pressed:

Example Program.cs

using System;
using GlobalHotKeys;
using GlobalHotKeys.Native.Types;

void HotKeyPressed(HotKey hotKey) =>
  Console.WriteLine($"HotKey Pressed: Id = {hotKey.Id}, Key = {hotKey.Key}, Modifiers = {hotKey.Modifiers}");

using var hotKeyManager = new HotKeyManager();
using var subscription = hotKeyManager.HotKeyPressed.Subscribe(HotKeyPressed);
using var shift1 = hotKeyManager.Register(VirtualKeyCode.KEY_1, Modifiers.Shift);
using var ctrl1 = hotKeyManager.Register(VirtualKeyCode.KEY_1, Modifiers.Control);

Console.WriteLine("Listening for HotKeys...");
Console.ReadLine();

Example Applications

There are also some example applications that show off the HotKey registration:

• Avalonia
• Console
• WinForms
• Wpf

References

• GlobalHotKey (https://github.com/8/GlobalHotKeys)
• AutoHotKey (https://www.autohotkey.com/)
• AvaloniaUI (https://avaloniaui.net)
• RegisterHotKey (https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-registerhotkey)
• UnregisterHotKey (https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-unregisterhotkey)
• Blog Post “Desktop Apps with AvaloniaUI and FSharp” (./Desktop-Apps-with-AvaloniaUI-and-FSharp)
• WM_HOTKEY (https://docs.microsoft.com/en-us/windows/win32/inputdev/wm-hotkey)
• CreateWindow (https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-createwindowa)
• CreateWindowEx (https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-createwindowexw)
• RegisterClass (https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-registerclassw)
• RegisterClassEx (https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-registerclassexw)
• WNDCLASSW (https://docs.microsoft.com/en-us/windows/win32/api/winuser/ns-winuser-wndclassw)
• WNDCLASSEX (https://docs.microsoft.com/en-us/windows/win32/api/winuser/ns-winuser-wndclassexw)
• DefWindowProc (https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-defwindowprocw)
• Message Loop (https://docs.microsoft.com/en-us/windows/win32/winmsg/using-messages-and-message-queues)
• GetMessage() (https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-getmessage)
• DispatchMessage() (https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-dispatchmessage)

TLDR;

You can write cross platform desktop applications using AvaloniaUI, F# and Reactive Extensions instead of WPF and C#. You can take a look at an example Application called Timeboxing here.

Revisiting Desktop Apps

Lately I’ve been revisiting how to best write desktop applications using dotnet. In the past, I’ve mostly used WPF and while it got the job done, there were a lot things that could have been improved upon.

I chose to focus on the two problems that I found to be the most annoying:

• Windows only
• Xaml

Windows only

In my eyes a big disadvantage of using WPF is that it only runs on Windows.

Nowadays, with dotnet core running great on Linux and Mac and WPF being available to a dotnet core application you would assume WPF to run on those platforms as well - but sadly you’d be wrong, because WPF still does not run on anything else except for Windows.

Xaml = Meh

Using Mvvm and Bindings made at least the ViewModel code testable and restricted the ugly parts to the views, which I found to be the most troublesome part of the development experience.

The reason for this is, that Views are usually declared in Xaml and by writing Xaml you have to leave your trusty programming language behind. By doing that that, you lose all your existing tools that your programming language provides and trade it in for… some xml?!

This in turn renders even trivial tasks like negating booleans or converting data into painful endeavours of figuring out how every basic thing that you have been taking for granted can be done in this new environment.

But it’s not like a speed bump, that just slows you down one time when getting started, like learning a new API or writing some glue code. It’s stays a continuous problem, because you need to consume it in a way that’s both verbose and irreducible, so it does not get better much better over time.

Take for example negating a boolean. In C# you would just prepend ! and be done with it. But in Xaml, you would need to implement an IValueConverter in a separate file, reference it in the xaml and use it in the binding.

There are of course several other solutions to that problem, all with different tradeoffs, but the mere fact that a trivial problems does not have an equally trivial solution is exemplary for Xaml.

Can we do better?

So the question then becomes, how can we do better? I started to look at the crossplatform issue first, which is more fundamental as it is dictated by the UI-Framework that you are using.

WPF -> AvaloniaUI

There are a couple of cross platform GUI Frameworks out there and doing a detailed comparison between even two such frameworks would be huge undertaking on moving targets - so I am not going to do that.

Also I don’t want do any of those projects a disservice by adding what little experience I have of using them and will therefore not even mention those that I looked at and didn’t pick, instead I just want to heave praise on the one I did.

So I’ve tried a couple of frameworks and the best solution for me was AvaloniaUI.

If you are looking for a cross-platform UI Framework for .NET, I highly recommend checking out AvaloniaUI, especially if you are coming from a WPF background. AvaloniaUI has all the best parts of WPF and in my opinion improves on a couple of them as well, e.g. styling and binding.

For example, AvaloniaUI solves the previously mentioned problem of negating a boolean value in a Binding by supporting the “Bang” (!) operator directly in the Xaml Binding Syntax.

They also ship their own Designer Extension for Visual Studio and if you are using Jetbrains Rider you’ll find that they have recently also started shipping support for editing AvaloniaUI xaml and added a control preview.

The feature set is great, the code is great - for me, AvaloniaUI is what WPF should have been.

Xml -> Code

Additionally, as I’ve mentioned earlier, I had a hunch that using normal code instead of xaml could result in a superior experience developing applications - at least for me.

To try out my theory, I’ve decided to built a small sample application using that approach to figure out what works and what doesn’t.

From View classes…

The best practice when working with xaml is to create a class for each view. So the most straighforward way to transition from xaml to code is to do just that.

public class MainView : UserControl
{
  public MainView()
  {
    this.Content = ...
  }
}

…To View methods…

But I’ll found that creating them as classes is just the default - it doesn’t add anything. Why not instatiate a UserControl and set it’s Content property directly?

public UserControl GetMainView()
{
  var mainView = new UserControl();
  ...
  return mainView;
}

That’s much better as methods are way less rigid and better composable than classes.

…With Object Initializers

Also you’d want to use a declaration like approach to create the controls, not an imperative one. So e.g. instead of writing:

var mainView = new UserControl();
var border = new Border();
mainView.Content = border;
var grid = new Grid();
border.Child = grid;
...

Which obfuscates the hierachy of the controls and is just generally really hard to read and write, you’d probably want some variant of the following:

new UserControl {
  Content = new Border {
    Child = new Grid {
      ...
    }
  }
}

Improvements

From a coding perspective this is tons better than Xaml, but sometimes you run into problems, because not all parts of the API is designed with an approach like that in mind. E.g. Collection properties that only have getters but no setters or creating and assigning styles and so one.

To work around those issues and get a better declaration-like approach, I wrote a couple of extension methods that I could invoke to keep this declaration style going without falling back to intermediate variables, which improved the code legibility a lot.

The result was a somewhat fluent API that takes in an object, mutates it and returns it again and allows you to keep going with the declaration based approach. It would probably be best to use the Builder pattern for the same approach, but that’s waaay more wrapper code than I was willing to write. So the downside was that it was rather inconsistent - there was a mix of pre-existing properties and some extension methods that you would need to chain.

C# -> F#

Then I switched to F#, which I found better suited to that approach. For example, it allows extending types with properties that can then be invoked in the constructor, which allowed me to further simplify the code. It also cuts down on some uses of brackets and commas, but far less than the marketing material would make you want to believe.

For example you can define extension properties for common operations like setting the grid row and column like so:

// extend all controls with the grid row / column properties
type Control with
  member this.Row
    with get () = Grid.GetRow (this)
    and set row = Grid.SetRow (this, row)
  member this.Column
    with get () = Grid.GetColumn (this)
    and set column = Grid.SetColumn (this, column)
  member this.ColumnSpan
    with get () = Grid.GetColumnSpan (this)
    and set columnSpan = Grid.SetColumnSpan (this, columnSpan)
  member this.RowSpan
    with get () = Grid.GetRowSpan (this)
    and set rowSpan = Grid.SetRowSpan (this, rowSpan)

Then you can just set the rows and columns properties directly when declaring the control:

Border (
  Child =
    Grid (
      ColumnDefinitions = ColumnDefinitions "*, Auto",
      Children = [
        Button (Column = 1, ...)
      ]
    )
)

ViewModels -> State

To connect the views with the actual data, you can just write a ViewModel, like usually, by deriving from ReactiveUI’s ReactiveObject class and implementing observable properties that can be bound by assigning a Binding in the View’s declaration and that works as you would expect.

But I found that declaring the ViewModel classes required a lot of boilerplate code and was also actually one of the few things that I found harder to do in F# that in C#. Also binding them in the view using a string kind of doesn’t make sense if the View is written in Code anyway, because you could have verified those bindings at compile time instead of looking for the binding errors at runtime.

Reactive Extensions

While reactive extensions do have a learning curve, I do prefer them to easier approaches like INotifyPropertyChanged or Redux. While the latter approaches are easy to get started with, I find that there a certain scenarios for which they are just a bad fit.

So in the end I stuck with using Observables directly, which makes the binding code strongly typed but I got rid of all the ViewModels boilerplate code, looking at you ReactiveObject, SetAndRaiseIfChanged, PropertyChangedHelper and all of your many friends.

AppBuilder Extensions

After getting rid of the UserControl, Window on ViewModel classes, I’ve added a couple of extension methods for the fluent AppBuilder to get rid of writing a custom Application class just to select the theme or the mainwindow, which further cuts down on the boilerplate code and you are left with:

AppBuilder.Configure<Application>()
  .AddStyleFluentDark()
  .UsePlatformDetect()
  .UseMainWindowFactory(mainWindowFactory)

Missing Preview / Designer

Now there was one major issue left: no designer preview!

The reason is that the designer preview only works for .xaml files and classes and I’ve burned that bridge. One workaround is of course embedding the view in another control class based control and opening that one in the designer, but that’s back to boilerplate central.

While I’ve never used the designer in WPF for actual editing, I was definitely missing the preview, because spinning up the application continuously while you are working on the UI just doesn’t cut it.

In WPF, you could have built a primitive previewer yourself, by instantiating the control, calling Measure() and Arrange() on it and using RenderTargetBitmap to render it to an png… and this is where the similarities between AvaloniaUI and WPF were really useful, because the same approach works perfectly for Avalonia as well!

Preview Images

I ended up writing simple integration tests that can render a control to a png.

[<Fact>]
let mainViewTest () =
  renderSizeAuto "mainView" (State.init >> mainView)

So when I am working on the controls I just mark those tests to be continually executed on a code change.

This works great in Visual Studio with nCrunch. In Rider I am using the continuous testing mode to achieve a similar effect and I guess you can do it using dotnet watch as well if your are working in vscode or something else. Then you just need to open the files you are interested in a tool that reloads them on changes, e.g. vscode.

Suprisingly I found this approach to work better in some cases than using an integrated designer. While it’s slower, it’s also way more flexible because you can have multiple views of a control with different states visible at the same time. E.g. a progress bar that is empty, half-filled and full.

This fits nicely with the fluent ApplicationBuilder approach to configure the themes, which allow you to generate previews of the controls using different themes as well.

You can an also try out how the control scales to different sizes.

Free screenshots

As an added bonus, you automatically get screenshots of your application that you can use in a blog post or your documentation and that you can review for visual regressions without running your application yourself or using UI Automation.

Helping out QA and contributing to the documentation of the project never felt so effortless!

Example App: Timeboxing

I’ve thrown together a sample application that demonstrates that approach. It’s called Timeboxing and available on github and it’s basically a glorified countdown timer. If you are interested about it’s usecase, you can take a look at the wikipedia article about timeboxing or why I find it useful.

The UI is pretty self-explanatory and looks like that:

Summary

When developing desktop applications I’ve switched out the following things:

• WPF -> AvaloniaUI
• C# -> F#
• Xaml -> Code
• Views / UserControls -> Functions
• ViewModels + Bindings -> Functions + Oberservables
• Designer -> Preview Images

I’ve found developing an application with that stack way more enjoyable, while improving the output as well.

So far so good, but this was only a small, single application and the question remains: What if the size of the application increases? Will this approach scale? I don’t know yet, but the simple architecture makes me rather optimistic and I am looking forward to find out.

References

• AvaloniaUI (https://github.com/AvaloniaUI/Avalonia)
• F# (https://fsharp.org)
• Reactive Extensions (https://dotnetfoundation.org/projects/reactive-extensions)
• Timeboxing Github Page (https://github.com/8/timeboxing)
• Inverting a property (Stackoverflow)
• Wikipedia article about timeboxing (https://en.wikipedia.org/wiki/Timeboxing)

TLDR;

I’ve converted the default tailwind color palette from javascript to an inkscape palette that you can download from here and copy to your palette folder.

Background

Tailwind CSS is my favourite CSS Framework. Although Tailwind is highly customizable, it comes with great defaults, for example a gorgeous color palette.

Inkscape is a great, free vector graphics design tool that supports custom color palettes in GIMP’s palette format (.gpl). To be able to use the tailwind’s color palette in inkscape, I’ve created a custom color palette for inkscape that you can download and use.

Why can this be useful

This can come in handy, if one of the following is true:

• You prefer to start your designs in inkscape before coding them
• You want to create a svg graphic for your webpage
• You just prefer the tailwindcss color palette to the one provided by default in inkscape.

How to use tailwindcss color palette with inkscape

1. Download the palette.
2. Copy it to your palette folder.
   You can find your palette folder, by opening the Preferences window by pressing CTRL+Shift+p in Inkscape and select System in the menu on the left. The palette folder is shown under User palettes. Click the button on the right to open it in the file explorer.
   (On Windows the default folder is %APPDATA%\inkscape\palettes)
3. Restart Inkscape so that the application picks up the new palette
4. Activate the palette, by clicking on the left-arrow icon on the bottom right and selecting Tailwind CSS from the popup as shown in the screenshot below.

To get the get a colors tailwindCSS name, you can hover over the color with the mouse.

5. (Optionally) You can also use it in the Swatches window by pressing Ctrl+Shift+w and clicking on the Left-Arrow icon and selecting Tailwind CSS from the list of palettes.

…and you are done!

How to use tailwindcss color palette with Gimp

As the file format is a Gimp palette, it can also be used as in gimp and the approach is similar.

1. Start by downloading the palette.
2. Copy tailwindcss.gpl to gimp’s palette folder, e.g. %APPDATA%\GIMP\2.10\palettes on windows.
3. Restart gimp
4. Open the Palettes window by clicking Windows -> Dockable Dialogs -> Palettes

5. Now in the Palettes window open the palette in the Palette Editor by double-clicking the icon Tailwind CSS

Source

I’ve written a small F# tool that did the conversion for me, it’s hosted on github.

References

• Tailwind CSS (https://tailwindcss.com)
• Inkscape (https://inkscape.org)
• Tailwind CSS Color Palette Reference (https://tailwindcss.com/docs/customizing-colors#color-palette-reference)
• Source Code (https://github.com/8/tailwind2palette)

TLDR;

While Visual Studio does not support a Block Caret for normal mode, you can write an extension for Visual Studio to get a Block Caret, which I did and published here.

Philosophy

So I am big fan of continuous small improvements that add up over time, which includes any investment into the developer inner loop. But the pursuit of tiny gains holds a danger of it’s own - it can lead the unwary traveller astray and into rabbit holes that one may not reappear from quickly - should they be deep enough.

In case I may be biased, hence the name of the domain, I let you dear reader, be the judge if this story falls into the former camp or the latter.

What’s with the Block Caret?

Maybe it’s because I am getting older or maybe it’s because I’ve recently upgraded to a 32” 4K Monitor, I found the visibility of the default Visual Studio caret lacking, as shown in the the following screenshot:

While it’s certainly not impossible to find the caret that sits right after the second logger, it’s certainly not obvious and takes some mental effort.

I noticed that I’ve grown very fond of the block cursor type combined with a color that has a high contrast to your used color scheme - an approach that I’ve used in vscode to great success.

For me, this approach improves upon the easy discoveryability of a blinking cursor by ridding it of the blinkrate’s time-delay and replacing it’s distracting nagging with the serenity I’ve grown accustomed to by the line cursor.

You can take a look at some cursor types that are available in vscode and try the out by hovering over their names on the right.

Cursor on a Word

line (default)

line-thin

underscore

box

box-outline

I found the visibility of a specific cursor type depends on the character it sits on. For example, the visibility of most cursor types decrease dramatically if the caret sits ontop of a bracket, the only saving grace is the aggressive color of the caret.

Cursor on a Bracket

line (default)

line-thin

underscore

box

box-outline

The specifics will of course depend on a lot of factors, including the programming language and font used and so on, but in general the bigger the highlighted area and the higher the contrast the more visible it will be.

The search begins

To my dismay, I found no such feature in Visual Studio, well with the exception of the override mode, which looks the part, but is useless to me due to the behavior change it introduces. The override mode is IMHO a mostly useless remnant of that IDE’s barbaric past, just like the tailbone in humans - and about as useful, YMMV of course.

So I looked into several other options:

• Cursor Thickness
• Text Cursor indicator
• Visual Studio Extensions

Cursor Thickness

Then there is a system-wide setting for Windows that controls the thickness of the cursor that Visual Studio also respects:

The problem with that setting is that the color of the cursor is based on the “Plain Text” color and drawn with 100% opacity and so it obscures the character it currently sits on, which is… an interesting choice of implementation and the result is accordingly disappointing:

Text cursor indicator

In a recent windows build, a new Feature named Text cursor indicator was introduced, which sets out to solve exactly the “problem of the lost cursor”.

When turned on, it looks like it’s working and the indicators appear as shown in the following screenshot:

The problem is, that it stops working as soon as you scroll (using the mousewheel or a keyboard shortcut) or use Go To Definition - what a bummer!

If it would actually work and could be restricted to apply only to certain applications, it could be a viable solution.

Visual Studio Extensions

While I didn’t find any extension that solved the problem out of the box, it bears mention that there is a very popular Visual Studio Extension that adds Vim support to Visual Studio called VsVim. As part of it’s Vim implementation it also implements a Block Caret for normal mode, as is custom for Vim. While I do think that Vim is a great solution for text editing and I use it actively, I find I am more productive with a different setup when writing code.

Thanks to Jared Parsons (@jaredpar), the author of VsVim, who graciously OpenSourced the extension, I was able to gather enough information to figure out how he does that. The logic of rendering and updating the caret is rather straightforward and what you would expect, basically you hide the original caret and add your own vanilla WPF Controls (e.g. Canvas, Rectangle, TextBlock) to an AdornmentLayer and move them around as soon as something changes. But interfacing with Visual Studio was the tricky part and I am sure having the code available saved me lot of time (there is an especially arcane part for accessing the AdornmentLayer).

Visual Studio Extension: BlockCaret

Armed with that knowledge I finally could write a very simple extension from scratch that implements a Block Caret with the result below:

I’ve published the extension, so if you are interested you can try and download it from here or search for ‘BlockCaret’ in Visual Studio Extensions.

References

• Text Cursor Indicator (https://docs.microsoft.com/en-us/windows-insider/archive/new-in-20H1#text-cursor-indicator)

• Block Caret (https://marketplace.visualstudio.com/items?itemName=MartinKramer.BlockCaret)

• VsVim Github Page (https://github.com/VsVim/VsVim)

• VsVim (https://marketplace.visualstudio.com/items?itemName=JaredParMSFT.VsVim)

• Text Cursor Indicators (https://docs.microsoft.com/en-us/windows-insider/archive/new-in-20H1#text-cursor-indicator)

TLDR;

When using [DllImport] you can only specify a constant string as the library name. If you are running dotnet core 3.0 or newer and you need more flexibilty, you can use the class NativeLibrary from the System.Runtime.InteropServices namespace.

If you need to change the name of the native library, a combination of NativeLibrary.SetDllImportResolver() and NativeLibrary.TryLoad() is all you need and you can find a code sample further down the article.

Calling Native Libraries from C#

The .NET way of calling native Libraries is P/Invoke (Platform Invoke).

A major part of Platform Invoke is the DllImportAttribute, which is used to declare where - that is, in which native library - the implementation of the extern static methods can be found.

The following example shows a perfect use case:

internal static class NativeApi
{
  [DllImport("user32.dll")]
  public static extern IntPtr GetForegroundWindow();
}

It’s a perfect use case, because the exact name of the library is known in advance and the library is also part of the operating system - only a single version exists and it’s name will never change.

Cross-Platform support

When Platform Invoke was designed, .NET only ran on windows but with Mono and later dotnetcore cross-platform support was added to Platform Invoke.

To make [DllImport] work cross-platform the runtime adds some additional logic when trying to determine the name of the native library. In particular, you can skip the file extension when using the DllImport attribute - the extension is selected automatically depending on the OS it runs on.

For example, when I want to create a C# wrapper that is able to call the method TessBaseAPICreate() from the tesseract library, I can declare it like this:

internal static class NativeTesseractApi
{
  [DllImport("libtesseract-4")]
  public static extern IntPtr TessBaseAPICreate();
  ...
}

Where the name of the library is supplied in the constructor of the DllImportAttribute.

This approach can enable cross-platform bindings, because the file extension is selected based on the executing operating system. On Windows it’s looking for a .dll, on linux it’s a .so file and on OS X it’s a .dylib.

Additionally, it’s custom to prefix the library name on linux with “lib” so that when a method is declared using [DllImport("Library")] the runtime will look for a file named Library.dll on Windows and on linux it look also for liblibrary.so.

Naming is hard

The problem is, that altough this pattern applies to many libraries, it’s not a strict rule - it may or may not apply to your favorite library as the authors are free to name them in any way they want.

An additional naming pattern for linux libraries is that the version number comes after the file extension. On windows on the other hand, there is usually no indication of the version directly in the file name.

So if you use [DllImport("Library")] this could lead to liblibrary.so.3 being found and loaded.

In the aforementioned example with the tesseract library, the platform dependent library name resolution doesn’t work unfortunately, because it doesn’t follow that pattern: eg. the Windows binary is prefixed with lib and the version number comes after the filename, separated with a dash (libtesseract-4.dll).

Customizing the library name is unfortunately only possible at compile time, as the information is stored in an Attribute, which only allows compile time constants.

Conditional Compilation

This in turn, often leads to the use of conditional compilation with preprocessor directives like #if eg:

internal static class NativeTesseractApi
{
  static class Constants
  {
    public const string LibraryName =
#if WIN
      "libtesseract-4";
#else
      "libtesseract.so.4";
#endif
  }

  [DllImport(Constants.LibraryName)]
  public static extern IntPtr TessBaseAPICreate();
  ...
}

The downside of this approach is platform dependent il-code that needs to be rebuild for all platforms, despite all the code being the same except for this single constant.

There are a plethora of additional information that is sometimes used when coming up with assembly names, eg. debug vs release builds, x86 vs. x64, single vs multithreaded, etc. Although this can be solved by using additional build flags, rebuilding all combinations and storing the resulting artifacts can get unwieldy fast and leads to artifact bloat.

Static classes

On the flip side it’s possible to write different bindings for each native assembly and then switch them at runtime to use the correct one. The problem with this approach is that this leads to code duplication, as you will need to write almost identical declarations for all methods for each native assembly, eg.:

internal static class NativeTesseractLinuxApi
{
  public const string LibraryName = "libtesseract.so.4";

  [DllImport(Constants.LibraryName)]
  public static extern IntPtr TessBaseAPICreate();

  [DllImport(Constants.PlaceHolderLibraryName)]
  public static extern void TessBaseAPIDelete(IntPtr handle);
  ...
}

internal static class NativeTesseractWindowsApi
{
  public const string LibraryName = "libtesseract-4";

  [DllImport(Constants.LibraryName)]
  public static extern IntPtr TessBaseAPICreate();

  [DllImport(Constants.PlaceHolderLibraryName)]
  public static extern void TessBaseAPIDelete(IntPtr handle);
  ...
}

Additionally you will need some kind of runtime logic to switch between those classes, eg:

internal static class NativeTesseract
{
  static bool IsWindows()
    => Environment.OSVersion.Platform == PlatformID.Win32NT;

  public static IntPtr TessBaseAPICreate()
      => IsWindows()
        ? NativeTesseractWindowsApi.TessBaseAPICreate()
        : NativeTesseractLinuxApi.TessBaseAPICreate();
  ...
}

To add insult to injury, even the glue code to switch between the different static classes is repetitive and cannot be abstracted succinctly without resorting to reflection (and giving up compile time checks) in C# as static classes cannot implement interfaces which in turn leads to code bloat.

Enter the NativeLibrary class

If you are approaching this from a native application, you would have probably used Run-Time Dynamic Linking to solve that issue, using functions like LoadLibrary() to load the dll and GetProcAddress() to get the address of the function or the linux equivalent dlopen() and dlsym(), respectively.

With dotnet core 3.0, we have that option in managed code as well, thanks to the new class NativeLibrary.

So the managed counterpart for LoadLibrary()/dlopen() is TryLoad() and GetProcAddress()/dlsym() is handled by TryGetExport(), which is shown in the following table:

In addition to that, NativeLibrary also supports registering a callback that can be used to resolve dll import request, using the SetDllImportResolver() method. With this, it’s really easy to select the library name at runtime and I came up with the following solution:

internal static class NativeTesseractApi
{
  static class Constants
  {
    public const string PlaceHolderLibraryName = "NativeTesseractLib";
    public const string WindowsAssemblyName = "libtesseract-4";
    public const string LinuxAssemblyName = "libtesseract.so.4";
  }

  static NativeTesseractApi()
    => NativeLibrary.SetDllImportResolver(Assembly.GetExecutingAssembly(), DllImportResolver);

  static string GetLibraryName(string libraryName)
    => libraryName switch
    {
      Constants.PlaceHolderLibraryName => Environment.OSVersion.Platform switch
      {
        PlatformID.Win32NT => Constants.WindowsAssemblyName,
        _ => Constants.LinuxAssemblyName,
      },
      _ => libraryName,
    };

  static IntPtr DllImportResolver(string libraryName, Assembly assembly, DllImportSearchPath? searchPath)
  {
    var platformDependentName = GetLibraryName(libraryName);

    IntPtr handle;
    NativeLibrary.TryLoad(platformDependentName, assembly, searchPath, out handle);
    return handle;
  }

  [DllImport(Constants.PlaceHolderLibraryName)]
  public static extern IntPtr TessBaseAPICreate();
  ...
}

Which basically consists for 4 steps:

1. Keep a single static class and keep the declarative approach of using the [DllImport] attribute and use a well known string as the library name.

internal static class NativeTesseractApi
{
  static class Constants
  {
    public const string PlaceHolderLibraryName = "NativeTesseractLib";
    ...
  }

  [DllImport(Constants.PlaceHolderLibraryName)]
  public static extern IntPtr TessBaseAPICreate();
  ...
}

2. Use NativeLibrary.SetDllImportResolver() to register a callback function that handles the importing.

internal static class NativeTesseractApi
{
  static NativeTesseractApi()
    => NativeLibrary.SetDllImportResolver(Assembly.GetExecutingAssembly(), DllImportResolver);

  static IntPtr DllImportResolver(string libraryName, Assembly assembly, DllImportSearchPath? searchPath)
  {
    ...
  }
  ...
}

3. Implement the library name resolution, based on it’s current environment.

internal static class NativeTesseractApi
{
  static class Constants
  {
    public const string PlaceHolderLibraryName = "NativeTesseractLib";
    public const string WindowsAssemblyName = "libtesseract-4";
    public const string LinuxAssemblyName = "libtesseract.so.4";
  }

  static string GetLibraryName(string libraryName)
    => libraryName switch
    {
      Constants.PlaceHolderLibraryName => Environment.OSVersion.Platform switch
      {
        PlatformID.Win32NT => Constants.WindowsAssemblyName,
        _ => Constants.LinuxAssemblyName,
      },
      _ => libraryName,
    };
  ...
}

4. And then implement the resolver by using the resolved library name and calling NativeLibrary.TryLoad() to load it.

internal static class NativeTesseractApi
{
  static IntPtr DllImportResolver(string libraryName, Assembly assembly, DllImportSearchPath? searchPath)
  {
    var platformDependentName = GetLibraryName(libraryName);

    IntPtr handle;
    NativeLibrary.TryLoad(platformDependentName, assembly, searchPath, out handle);
    return handle;
  }
  ...
}

In a case like this, the approach of using the NativeLibrary class feels far superior to both the conditional compilation and multiple static classes approach and it saved us from either a lot of builds and their different binaries or a lot copy/pasted code.

Summary

DllImport combined with naming conventions can often make calling native binaries trivial, but in more complex cases, you can use conditional compilation, multiple static classes and now the new NativeLibrary class.

From those approaches, NativeLibrary provides the greatest flexibility to solve issues directly in code.

References

• P/Invoke (https://docs.microsoft.com/en-us/dotnet/standard/native-interop/pinvoke”)

• DllImport Attribute (https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.dllimportattribute?view=netcore-3.1)

• Tesseract OCR (https://github.com/tesseract-ocr/tesseract)

• Conditional Compilation with #if (https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/preprocessor-directives/preprocessor-if)

• Run-time Dynamic Linking(https://docs.microsoft.com/en-us/windows/win32/dlls/run-time-dynamic-linking)

• LoadLibrary() (https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibrarya)

• GetProcAddress() (https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-getprocaddress)

• dlopen() (https://linux.die.net/man/3/dlopen)

• dlsym() (https://linux.die.net/man/3/dlsym)

• NativeLibrary class (https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.nativelibrary?view=netcore-3.1)

TLDR;

When running cron as your ENTRYPOINT in a container, make sure you start cron in foreground mode (cron -f) and consider redirecting your job’s stdout and stderr to crons (eg. */1 * * * * root /app1/test.sh > /proc/1/fd/1 2>/proc/1/fd/2). You can find an complete Dockerfile here.

Dockers Entrypoint

Docker is build for running one process per container. That doesn’t mean that you can’t run more than one process per container, it’s just that docker only cares about the first process of an image. This process is what you specify using the ENTRYPOINT in your Dockerfile and it’s the process that docker starts when you run an image.

Should that process terminate, docker will stop the container and if that process writes to stdout or stderr, docker will make that information available via docker logs [container] and if you run the image attached your keyboard input will be forwarded to your ENTRYPOINTs stdin.

If you want to run multiple processes, you can start additional ones from your ENTRYPOINT. Docker doesn’t care about those child processes. Should one of them terminate, your container will continue to run - if they die, they die. And if they write stuff to stdout or stderr, docker doesn’t know and doesn’t care.

cron is a daemon

cron is usually run as a daemon so on startup cron forks, which means it creates a running copy of itself and then terminates the original process.

Which is normally what you want, but when running inside a container docker will stop your container when your main process exits.

Running Cron inside a Docker image

The way solve that problem is to instruct cron to start in foreground mode, so that it doesn’t fork by using:

cron -f

By running cron in foreground mode you can use it as an ENTRYPOINT in your Dockerfile and it will execute the jobs you’ve specified in your crontab file.

Logging of cron jobs

There is one problem left, which is that the output of those jobs don’t show up in the docker logs, because docker only tracks the stdout/stderr of the first process.

A simple solution for this problem is just redirecting the stdout/stderr of the child processes to the main process (which will always have PID 1), e.g.:

*/1 * * * * root /app1/test.sh > /proc/1/fd/1 2>/proc/1/fd/2

An complete example

FROM ubuntu

# install cron
RUN apt-get update && apt-get install cron -y -qq

# create two test applications that we will launch using cron
RUN mkdir /app1 && echo 'echo `date +"%H:%M:%S"` - This is sample application 1!' > /app1/test.sh && chmod +x /app1/test.sh
RUN mkdir /app2 && echo 'echo `date +"%H:%M:%S"` - This is sample application 2!' > /app2/test.sh && chmod +x /app2/test.sh

# register cron jobs to start the applications and redirects their stdout/stderr
# to the stdout/stderr of the entry process by adding lines to /etc/crontab
RUN echo "*/1 * * * * root /app1/test.sh > /proc/1/fd/1 2>/proc/1/fd/2" >> /etc/crontab
RUN echo "*/2 * * * * root /app2/test.sh > /proc/1/fd/1 2>/proc/1/fd/2" >> /etc/crontab

# start cron in foreground (don't fork)
ENTRYPOINT [ "cron", "-f" ]

docker build . -t cron
docker run cron

17:03:01 - This is sample application 1!
17:04:01 - This is sample application 1!
17:04:01 - This is sample application 2!
17:05:01 - This is sample application 1!
17:06:01 - This is sample application 1!
17:06:01 - This is sample application 2!
17:07:01 - This is sample application 1!
17:08:01 - This is sample application 1!
17:08:01 - This is sample application 2!
17:09:01 - This is sample application 1!
17:10:01 - This is sample application 1!
17:10:01 - This is sample application 2!
17:11:01 - This is sample application 1!
17:12:01 - This is sample application 1!
17:12:01 - This is sample application 2!

• Dockerfile ENTRYPOINT (https://docs.docker.com/engine/reference/builder#entrypoint)
• fork (https://linux.die.net/man/3/fork)

Recently I solved a computer-vision problem that came up in a project which I found interesting and wanted to share. More specifically it boiled down to a problem of artifact removal and it goes like this:

The Problem

Given the following image which is a cutout of a scan of a sheet of paper:

There are one or more lines of text that are interested in and we want to use an OCR (Optical Character Recognition) Engine on it to retrieve the text.

Now the problem is, that there are some artifacts in addition to the text that we are interested in. If we feed the whole cutout into the OCR engine we get back the text we are looking for, but we also get the results of running the OCR on the artifacts as well.

So the process currently looks like that:

Of course it’s possible to process the text after having run the OCR and that even makes perfect sense incase the structure of the text that you are looking for is well known, like e.g. a telephone number in a specific format. In that case you can simply post process the output of the OCR with a regex or parse the text yourself.

In that case you can extend your pipeline to look like that:

But that’s not always the case. It could be that the information upon which you can decide what is an artifact and what is the content is in the relative positions they take up in the image.

If that’s the case you need to remove the artifacts before the OCR step, else you lose the information on which you would base this decision.

So the process will look like:

But enough with the abstract, let’s take another look at the example:

Here we are interested in the middle line, the one reading “This is the example text that we want to detect.”.

Notice the additional lines of text at the top and the bottom, which we don’t want. Additionally the scan itself is often rotated a few degrees, as shown in the example.

So the question is: How do we get only the relevant portions from the image?

Note: As you might have guessed, this specific example is made up by me, because the original images contain confidential information that I am not allowed to share.

A False Start

My first attempt to solve the problem was thwarted by the rotation of the image. I tried to find a Region of interest (a rectangle that is smaller than the image and contains only the relevant text) but this approach failed when it came to rotated images:

As you can see, I’ve got the option to either cut off the top of “This is…” or I still had some artifacts from the end of the top line remaining.

A better Solution

On my next attempt, I looked at the image and tried to distill the rules by which to decide if something is an artifact or not.

In this project, the artifacts where more or less continous artifacts that either touched the top or bottom of the image.

The strategy that I came up with is as follows:

0. Load the image
1. Get the positions of all objects
2. Find all objects that are near the top or bottom border
3. Find the sentences by recursively finding all objects that are next to previously found objects
4. Remove all artifacts from the image

0. Load the image

Before we can start manipulating the image, we need to load it and make sure it’s a grayscale image.

var input = File.ReadAllBytes(file);
using (var mat = Mat.FromImageData(input))
using (var grey = mat.CvtColor(ColorConversionCodes.BGR2GRAY))

1. Get the positions of all objects

Finding the positions of the objects is really easy with OpenCV. All you need to do is call FindContours().

Well almost. The thing is that FindContours() works on binary images (which is a fancy word for black and white images), where everything that is not a zero (perfect black), is treated like an object. That means before you can call Cv2.FindContours() you first need to do a black/white conversion and invert the result.

The process of generating a binary image is called thresholding, because pixels are tested against a specified threshold and then (usually) either changed to black or white. OpenCV supports thresholding with a couple of different methods, for example Cv2.Threshold.

Note: For noisy real word data (scanned images!) you might want to use the fancier Cv2.AdaptiveThreshold for better results like so:

using (var thresholded = new Mat())
{
  Cv2.AdaptiveThreshold(grey, thresholded, 255, AdaptiveThresholdTypes.GaussianC, ThresholdTypes.BinaryInv, 15, 7);
  ...

Given the input image, thresholding with inversion produces the following result:

2. Find all objects near the borders

Cv2.FindContours() allows us to retrieve the bounding boxes of all contours in the image. This makes identifying the objects near the top and bottom border trivially easy.

Now that we can find all letters that are near the border, the question is: How can we find the letters next to those at the border? And the ones close to them and so on? Or to put it in another way: How can we find all sentences that are close to border.

3. Find the sentences

Turns out, that’s not an easy problem to solve! The first thing that came to my mind was naively computing the distance between all points of the bounding boxes of the objects. But this has a couple of problems, for example you need multiple passes because you need to repeat the proximity test for all contours after a new contour is found to be in close proximity. But what’s even worse is, that it’s wrong for contours when an edge is closer to another contour than a point of it’s bounding box.

After noticing that this problem is way more complicated than I previously thought, I backtracked a little and found a nice solution by visually joining the objects before calling Cv2.FindContours().

This operation is called dilation and I am using the Cv2.MorphologyEx method for that. One of the parameters of that method is a structuring element. The structuring element is a fancy name for the width and height of a rectangle that is used to extend the existing objects.

int dilateX = 15, dilateY = 3;
using (var dilateKernel = Cv2.GetStructuringElement(MorphShapes.Ellipse, new Size(dilateX, dilateY)))
using (var morphed = new Mat())
{
  Cv2.MorphologyEx(thresholded, morphed, MorphTypes.Dilate, dilateKernel);
  ...

The following image shows the result of the dilation:

When we now call Cv2.FindContours() instead of getting a contour for each letter, we only get 3 contours - one for each sentence - which can be done with the following C# Code:

Mat[] contours; var hierarchy = new Mat();
Cv2.FindContours(morphed, out contours, hierarchy, RetrievalModes.External, ContourApproximationModes.ApproxSimple);

Note: Here I am passing RetrievalModes.External so that only the outermost contours are returned.

The following image visualizes the contours drawn over the input image:

Now we can easily find all contours whose bounding box touches the top or bottom border.

var imageHeight = mat.Height;
var yThreshold = 2;

var items = contours
  .Select(c => new { Contour = c, BoundingBox = c.BoundingRect() })
  .ToArray();

/* filter all contours that are near the top border or bottom border */
var filteredTopBottom = items
  .Where(i => i.BoundingBox.Top <= yThreshold ||
             (imageHeight - i.BoundingBox.Bottom) <= yThreshold)
  .ToArray();

4. Remove all artifacts

To remove the artifacts we simply draw over their contours and by doing so remove them from the image.

This can be done with the Cv2.DrawContours() method:

Cv2.DrawContours(fixedImage, filteredContours, -1, new Scalar(255, 255, 255), Cv2.FILLED);

And finally we are left with the clean, final image that we can pass on to the OCR engine:

Summary

In this article I tried to show how OpenCV can be used from C# to solve real-world computer vision problems.

Because the difficult part was figuring out how to tackle the problem at hand (and not necessarily the code itself), I tried to document my thought process including the dead ends.

If you want, you can find the complete example code here.

References

This article will explain how you can get started programming OpenCV with .NET in C#.

The problem is, that OpenCV is built with C/C++ and while comes with python bindings out of the box, it doesn’t have any official .NET bindings. That’s why there are lots of articles on how to get started using python but not a lot about dotnet.

TLDR;

Install the nugets OpenCvSharp4 and either OpenCvSharp4.runtime.win or OpenCvSharp4.runtime.ubuntu.18.04-x64, depending on your platform or download the example solution here.

Getting started

First you need to setup a new solution and project with the OpenCV Libraries. To access the native OpenCV libraries, you need a managed wrapper. I am using OpenCvSharp4, which you can use with dotnet core and works on both windows and ubuntu and is licensed under the BSD, just like OpenCV itself.

1. Create a new solution named OpenCvTest and move to that directory.

   dotnet new sln -o OpenCvTest
   cd OpenCvTest

2. Create a new console project also named OpenCvTest and add it to the solution.

   dotnet new console -o OpenCvTest
   dotnet sln add OpenCvTest

3. Add the nuget OpenCvSharp4 to the project.

   dotnet add OpenCvTest package OpenCvSharp4

4. Add the native runtime depending on your platform (either for windows or for ubuntu 18.04) Either OpenCvSharp4.runtime.win for windows

   dotnet add OpenCvTest package OpenCvSharp4.runtime.win

   or OpenCvSharp4.runtime.ubuntu.18.04-x64 for Ubuntu 18.04.

   dotnet add OpenCvTest package OpenCvSharp4.runtime.ubuntu.18.04-x64

   or both.

Note:
There are alternative .NET wrappers for OpenCV, but a word of caution: While OpenCV is licensed under the BSD, which makes it useable for closed source commercial applications, some wrapper libraries are using a different license. For example EmguCV uses the GPL by default, making it useless for any commercial closed source software.

This creates the inane situation that the library that does all the heavy lifting is free to use, but the wrapper library that only forwards the calls can cost quite a fortune - without any of the original authors even getting any compensation.
So make sure you check the license of the library before you start using it in a project!

Using OpenCV in C#

Now it’s time we actually use OpenCV!

1. We will test the library by creating a really simple application that grayscales images with the following code:

   using System;
   using System.IO;
   using OpenCvSharp;
   
   namespace OpenCvTest
   {
     class Program
     {
       static void Main(string[] args)
       {
         if (args.Length < 1)
           Console.WriteLine("Please specify a filename");
         else
         {
           string inputFileName = args[0];
           string outputFileName = $"{Path.GetFileNameWithoutExtension(inputFileName)}-gray.jpg";
           using (var image = new Mat(inputFileName))
             using (var gray = image.CvtColor(ColorConversionCodes.BGR2GRAY))
               gray.SaveImage(outputFileName);
         }
       }
     }
   }

   Let’s take a closer look at the important parts:

   First, we import the namespace of the library:

   using OpenCvSharp;

   Secondly, we load the image from a file into a matrix:

   using (var image = new Mat(fileName))

   Thirdly, we convert the image to grayscale:

   using (var gray = image.CvtColor(ColorConversionCodes.BGR2GRAY))

   Finally, we save the grayscaled image back to disk:

   gray.SaveImage(outputFileName);

2. Now we put an example.jpg in our solution directory, for example:

3. After we run the application like so:

   dotnet run --project OpenCvTest\OpenCvTest.csproj example.jpg

4. …we can find the resulting example-gray.jpg in the same folder:

Download the Example Solution

You can download a Zip-File with the complete solution here.

Summary

Getting started with OpenCV in C# is easy, thanks to the nice wrapper library OpenCvSharp4.

References

• OpenCV Documentation (https://docs.opencv.org/3.0-beta/modules/refman.html)

• OpenCvSharp4 (https://github.com/shimat/opencvsharp)

• Example Applikation (opencvtest.zip)

TLDR;

I’ve created a cheat sheet for the dotnet cli in different colors. It’s available as a pdf file that you can download at the bottom of the page.

Dotnet Commandline Interface

Being a fan of Visual Studio, I’ve neglected learning the dotnet commandline interface in the beginning when dotnet core came out. But now, after having setup the CI for a couple of projects, I was forced to leave my familiar and cushy Visual Studio and venture into commandline land.

Now that I’ve returned from this mouse-less land, I wanted to write down the commands that I needed more often in case someone else - like future me - may find it useful.

Because I’ve previously created a cheat sheet for the docker cli I already had all the resources to more or less quickly generate a new cheat sheet at my disposal, so without further ado: Here is my take take on a dotnet cheatsheet.

@Component({ components: { CheatSheetList } }) export default class DotnetCheatSheet extends Vue { items: CheatSheet[] = this.getCheatSheets();

getCheatSheets() { const names = [“dotnet_blue_dark”,“dotnet_blue_light”,“dotnet_blue_light0”,“dotnet_blue_light1”,“dotnet_blue1_dark”,“dotnet_blue1_light”,“dotnet_blue1_light0”,“dotnet_blue1_light1”,“dotnet_blue2_dark”,“dotnet_blue2_light”,“dotnet_blue2_light0”,“dotnet_blue2_light1”,“dotnet_darkblue_dark”,“dotnet_darkblue_light”,“dotnet_darkblue_light0”,“dotnet_darkblue_light1”,“dotnet_darkgray_dark”,“dotnet_darkgray_light”,“dotnet_darkgray_light0”,“dotnet_darkgray_light1”,“dotnet_green_dark”,“dotnet_green_light”,“dotnet_green_light0”,“dotnet_green_light1”,“dotnet_iceblue_dark”,“dotnet_iceblue_light”,“dotnet_iceblue_light0”,“dotnet_iceblue_light1”,“dotnet_indigo_dark”,“dotnet_indigo_light”,“dotnet_indigo_light0”,“dotnet_indigo_light1”,“dotnet_lightgray_dark”,“dotnet_lightgray_light”,“dotnet_lightgray_light0”,“dotnet_lightgray_light1”,“dotnet_red_dark”,“dotnet_red_light”,“dotnet_red_light0”,“dotnet_red_light1”,] return names.map(n => { return { image: /cheatsheet/dotnet/${n}.png, pdf: /cheatsheet/dotnet/${n}.pdf } as CheatSheet }); } }

TLDR;

• Use using instead of calling Dispose() manually whenever you can to keep instances from escaping their dispose call.
• Refactor to reduce the need for curly braces and indentation.
• Don’t keep Database Connections around longer than needed

For extra points:

• Implement IDisposable yourself when you see try and finally being repeated in a pattern.
• …but don’t use the old Dispose pattern

Why use using?

A using statement in the C# looks like this:

using (Resource resource = expression)
...

It is used for disposing Resources, which implement the IDisposable interface as soon as they leave the scope and without manually calling it’s Dispose() method.

The reason why this is useful is because there a ways that a disposable resource can escape the call of it’s Dispose() method.

Let’s start with an example and assume that the code in question calls Dispose() manually and starts simple:

var resource = new Resource();
resource.Dispose();

Now, that code is simple to read because it doesn’t do to much - the Resource is created and immediately disposed again.

A more reasonable snippet of code would look like:

var resource = new Resource();
// Insert a block of arbitrary statements.
resource.Dispose();

What could possible go wrong in this ominous block of arbitrary statements?

1. Protection against exceptions

Probably the most common way that resources escape their Dispose() call are exceptions.

Let’s take another look at the manual Dispose() call:

var resource = new Resource();
// Insert a block of arbitrary statements.
resource.Dispose();

If anything inside of the arbitrary statements block throws an Exception the execution stops before the resource.Dispose() call can be made.

If nothing at all catches the exception, then it’s sometimes no problem, because usually the process is terminated anyway and the OS cleans up after the process by freeing it’s resources like memory, files and sockets.

It’s usually if some catch statement earlier in the call stack handles the Exception and tries to keep on running like nothing happened that the real problems start to emerge.

Luckily the using statement protects against exceptions by wrapping the statement following it into a try block and the Dispose() method in it’s corresponding finally block.

So basically the following code:

using (var resource = new Resource())
{
  // Insert a block of arbitrary statements.
}

Is converted to something like the following:

{
  var resource = new Resource();
  try
  {
    // Insert a block of arbitrary statements.
  }
  finally
  {
    IDisposable d = (IDisposable)resource;
    if (d != null)
    {
      d.Dispose();
    }
  }
}

Thereby preventing the escape via thrown Exceptions.

I am pretty sure you already knew that, but wait, there’s more!

2. Protects against Reassignment

Another way for an instance to escape could be a wrong reassignment. Let’s assume that resource actually has some resource of finite length, for example a Buffer. An unknowing developer could swoop in and reassign a new value to the variable like so:

var resource = new Resource();
// A block of arbitrary statements
if (resource.Length < requiredLength)
{
  resource = new Resource(requiredLength);
}
// A block of arbitrary statements
resource.Dispose();

Now we have a bug - even though the resource.Dispose() call disposed an instance of the resource, it disposes only the second instance that was assigned inside the if statement - the first instance escapes.

By using the using statement (hehe) that potential error can’t happen, because the code:

using (var resource = new Resource())
{
  // A block of arbitrary statements
  if (resource.Length < requiredLength)
  {
    resource = new Resource(requiredLength);
  }
  // A block of arbitrary statements
}

Results in the following compile-time error:

Cannot assign to 'resource' because it is a 'using variable' (CS1656)

There is one more advantage, that’s kind of an extension of the reassignment protection.

3. Protects against ref shenanigans and out misuse

Additionally, the using statement prevents passing the parameter by ref, which would allow a reassignment of the variable from another method.

So the following code:

using (var resource = new Resource())
{
  // A block of arbitrary statements
  DoSomething(ref resource);
  // A block of arbitrary statements
}
...
void DoSomething(ref Resource resource)
{
  resource = new Resource();
}

…also doesn’t work and the following compile-time error is triggered:

Cannot use 'resource' as a ref or out value because it is a 'using variable'

As the error message hints, the same compile-time error is also shown if the variable is passed as an out parameter.

Which is fine by me, because it doesn’t make much sense for a method to produce already disposed objects, which commonly throw ObjectDisposedExceptions when any member is accessed after they got disposed.

With advantages like that, you would assume that everybody is using the using statement whenever they can, wouldn’t you?

I found out that’s not always the case. The reasons for that are often myths about using and IDisposable that pop up as objections from time to time. That’s why I decided to tackle them here as well.

Myths that developers believe about using

So without further ado I present to you the top 4 myths why developers don’t use using / IDisposable:

1. Multiple using statements introduce too many indentations / curly braces / nested code blocks!
2. I can’t dispose the Database Connection, I will need it later again!
3. I’ve got nothing to dispose!
4. I need to implement the dispose pattern!

Let’s look at each objection separately and see what we can do.

1. Too much Indentation

From time to time I hear developers complaining about the additional levels of indentation and curly braces that can be introduced by using a lot of using statements.

Let’s take a look at a simple example that accesses our 3rd most popular Database and loads an entry with the matching id from a table.

namespace UsingExample
{
    public class CustomerRepository : ICustomerRepository
    {
        string GetConnectionString()
            => "...";

        public CustomerModel GetCustomerById(int id)
        {
            CustomerModel customer;

            using (var tran = new TransactionScope())
            {
                using (var conn = new SqlConnection(GetConnectionString()))
                {
                    conn.Open();
                    using (var cmd = new SqlCommand("select * from Customer where Id = @Id", conn))
                    {
                        cmd.Parameters.AddWithValue("Id", id);
                        using (var dr = cmd.ExecuteReader())
                        {
                            if (dr.Read())
                            {
                                customer = new CustomerModel
                                {
                                    Id        = (int)   dr["Id"],
                                    Firstname = (string)dr["Firstname"],
                                    Lastname  = (string)dr["Lastname"],
                                };
                            }
                            else
                                customer = null;
                        }
                    }
                }
            }

            return customer;
        }
    }
}

Note: The TransactionScope is not really needed here, it only enforces that the command is really unable to modify anything, as it is automatically rolled back when it leaves the scope.

Now you can argue that those indentation levels are problematic - and honestly - I would agree, because IMHO even though the code is not doing a lot, it is already pretty hard to read.

But that’s not an issue with using using! Heck, you’ve got the same problems if you nest your ifs deeply. So the solution is not using fewer usings, but refactoring your code!

The similarity to if doesn’t end here, because using statements can skip the curly brackets, just like with an if statement. When you skip the curly braces {} after an if statement, you’ll want to indent the next statement, so that it’s clear that it is being affected by the if. But the thing is, you are not required to.

While we don’t want to take advantage of that freedom with an if statement, we maybe want to do that in the case of the using statement.

So instead of:

using (var tran = new TransactionScope())
{
    using (var conn = new SqlConnection(GetConnectionString()))
    {
      ...
    }
}

we can just write:

using (var tran = new TransactionScope())
using (var conn = new SqlConnection(GetConnectionString()))

Okay, but that trick alone does not get rid of all identations! Let’s take a look at how we use the SqlConnection:

using (var conn = new SqlConnection(GetConnectionString()))
{
    conn.Open();
    using (var cmd = new SqlCommand("select * from Customer where Id = @Id", conn))
    {
      ...
    }
    ...
}

If it weren’t this pesky conn.Open() call! It keeps us from just skipping the curly braces on the using statement with the new SqlConnection().

The reason is, that if you skip the curly braces {} the using variable will only be available for the following statement - in this case the conn.Open() call, but not for any statements after it, like the line were we actually use it in the SqlCommand constructor.

So what do we do in that case? The same thing that we do with too deeply nested if statements: we refactor them!

In this case, I am going to create a method for setting up the SqlConnection, including the conn.Open() call:

SqlConnection GetOpenConnection()
{
    var conn = new SqlConnection(this.GetConnectionString());
    conn.Open();
    return conn;
}

…and because I am on a roll, I am going to do the same thing for the SqlCommand and it’s Parameters.AddWithValue() call:

SqlCommand GetSelectCustomerByIdCmd(int id, SqlConnection conn)
{
    var cmd = new SqlCommand("select * from Customer where Id = @Id", conn);
    cmd.Parameters.AddWithValue("Id", id);
    return cmd;
}

While we are at it, we can also refactor the reading of the CustomerModel into it’s own method:

CustomerModel ReadCustomer(IDataReader dr)
{
    CustomerModel customer;

    if (dr.Read())
    {
        customer = new CustomerModel
        {
            Id        = (int)   dr["Id"],
            Firstname = (string)dr["Firstname"],
            Lastname  = (string)dr["Lastname"],
        };
    }
    else
        customer = null;

    return customer;
}

Now we can use them in the final refactored GetCustomerById() method:

public CustomerModel GetCustomerById(int id)
{
    using (var tran = new TransactionScope())
    using (var conn = GetOpenConnection())
    using (var cmd = GetSelectCustomerByIdCmd(id, conn))
    using (var dr = cmd.ExecuteReader())
        return ReadCustomer(dr);
}

I would argue that this is much better than what we had in the beginning:

• We’ve got rid of the identation and the curlies and even if we want to keep the indentation, it’s now really easy to read.
• We’ve got smaller methods that do less each and they only have one single concern instead of a big method that does all the things.
• …but wait, there’s even more! Now the code is also easy to extend!

As an example imagine that we now want to retrieve Customers in a slightly different way, for example retrieve all Customers at once. The only thing we need to do is implement two teeny-weeny methods that follow the same pattern as before:

SqlCommand GetSelectAllCustomersCmd(SqlConnection conn)
    => new SqlCommand("select * from Customer", conn);

public CustomerModel[] GetAllCustomers()
{
    var customers = new List<CustomerModel>();
    CustomerModel customer;
    using (var tran = new TransactionScope())
    using (var conn = GetOpenConnection())
    using (var cmd = GetSelectAllCustomersCmd(conn))
    using (var dr = cmd.ExecuteReader())
        while ((customer = ReadCustomer(dr)) != null)
            customers.Add(customer);
    return customers.ToArray();
}

…and we are done!

2. I can’t dispose the Database Connection, I will need it later again

This one is not about the IDisposable interface, but about specific implementations - Database Connection - e.g. SqlConnection.

So on one hand opening a Database Connections is an expensive operation.

On the other hand, you should always hold onto disposable resources for the least amount of time possible.

Okay, but who wins in that case? Luckily we don’t have to decide, because ADO.NET implements ConnectionPooling.

This means that a so-called pooler inside of the Framework holds onto the physical connection, even after we’ve called it’s Dispose() method.

As a result opening the Connection the first time is just as slow as always, but the second and all consecutive times when we open the connection we don’t pay that performance overhead - as long as the underlying physical connection is available and can be reused, of course.

This gives us the benefits of both worlds - a simple resource management strategy where we can just ask for a new Connection when we need it and dispose it immediately afterwards, but also great performance, because the underlying physical connection is being kept alive and reused, if possible.

So for Database Connections that means you should dispose them as soon as you are done with your operation, as you can see in the methods GetAllCustomers() and GetCustomerById() above.

3. I’ve got nothing to dispose

Okay, that’s a valid reason - how could you possibly call Dispose() if you don’t have a class that implements the IDisposable interface?

Well, sometimes it makes sense to implement IDisposable even if you don’t have unmanaged resources to free.

So what I’ve found is that the using statement is also useful to wrap logic that you normally would want to encapsulate in a try/finally block - with the additional advantages of reassignment protection as explained above.

For just a one-time use, this doesn’t make sense of course, but if you happen to see this as a pattern repeating in your code than wrapping the try/finally in an IDisposable makes sense.

As an example, image you want to control the number of concurrent requests to a contested resource in a multithreaded application e.g. like your webserver.

You’ve setup your contested resource and an instance of SemaphoreSlim to the maximum number of concurrent requests that you are comfortable with, like so:

this.contestedResource = new ContestedResource();
this.semaphoreSlim = new SemaphoreSlim(initialCount: 1, maxCount: 1);

And you now implement the async wrapper that wraps the calls to the contested Resource (without blocking the calling thread, thanks to SemaphoreSlims WaitAsync() method).

To not mess up the system with deadlocks if an exception is thrown when calling the contested resource, you make sure to wrap the call in a try block and release the semaphore in the finally block, like so:

public async Task<Result> GetFoo(CancellationToken ct)
{
  await this.semaphoreSlim.WaitAsync(ct);
  try { return this.contestedResource.GetFoo(); }
  finally { this.semaphoreSlim.Release(); }
}

public async Task<Result> GetBar(CancellationToken ct)
{
  await this.semaphoreSlim.WaitAsync(ct);
  try { return this.contestedResource.GetBar(); }
  finally { this.semaphoreSlim.Release(); }
}

public async Task<Result> GetBaz(CancellationToken ct)
{
  await this.semaphoreSlim.WaitAsync(ct);
  try { return this.contestedResource.GetBaz(); }
  finally { this.semaphoreSlim.Release(); }
}

...

When you look at that code, you can see that a specific pattern repeats itself:

1. It starts with the resource acquisition, which is always the same call: await this.semaphoreSlim.WaitAsync(ct)
2. Then custom code inside a try block is executed
3. Then the resource is released inside a finally statement: finally { this.semaphoreSlim.Release(); }

Isn’t that exactly what using provides? So let’s create code that wraps the semaphore usage in an IDisposable interface so that we can apply using:

The implementation will be split in two parts, the first one being the class that keeps track of the SemaphoreSlim instance and is able to hand out disposable tokens. The second class is the disposable token.

First I’ll start with the class that generates the tokens, I’ve called it AsyncLock:

public sealed class AsyncLock : IDisposable
{
    private readonly SemaphoreSlim semaphoreSlim;

    public AsyncLock()
        => this.semaphoreSlim = new SemaphoreSlim(1, 1);

    public async Task<IDisposable> WaitAsync()
        => await LockToken.WaitAsync(this.semaphoreSlim);

    public void Dispose()
        => this.semaphoreSlim.Dispose();
}

AsyncLock is able to do 3 things:

1. Initialize the SemaphoreSlim in the constructor.
2. Return an IDisposable instance by creating a new LockToken instance.
3. Propagate the Dispose to it’s SemaphoreSlim instance.

Next comes the class that is returned from a call to AsyncLock.WaitAsync(), which I’ve called LockToken:

sealed class LockToken : IDisposable
{
    private readonly SemaphoreSlim semaphoreSlim;
    private bool isDisposed = false;

    private LockToken(SemaphoreSlim semaphoreSlim)
      => this.semaphoreSlim = semaphoreSlim;

    public static async Task<LockToken> WaitAsync(SemaphoreSlim semaphoreSlim)
    {
        await semaphoreSlim.WaitAsync();
        return new LockToken(semaphoreSlim);
    }

    public void Dispose()
    {
        if (!this.isDisposed)
        {
            this.semaphoreSlim.Release();
            this.isDisposed = true;
        }
    }
}

This class is basically a fancy wrapper for calling this.semaphoreSlim.Release() that is called with using.

Note: The LockToken class guards against calling this.semaphoreSlim.Release() multiple times, by keeping track if it was already disposed.

Now we can rewrite our wrapper like that:

public async Task<Result> GetFoo(CancellationToken ct)
{
  using (var _ = await this.asyncLock.WaitAsync())
      return this.contestedResource.GetFoo();
}

public async Task<Result> GetBar(CancellationToken ct)
{
  using (var _ = await this.asyncLock.WaitAsync())
      return this.contestedResource.GetBar();
}

public async Task<Result> GetBaz(CancellationToken ct)
{
  using (var _ = await this.asyncLock.WaitAsync())
      return this.contestedResource.GetBaz();
}

And in addition we get this nice utility class that we can use for async locking.

4. I need to implement the dispose pattern

First of all, what is the dispose pattern? Basically, it’s a pattern of how to implement the IDispose interface.

But instead of just implementing the single method of this interface, namely void Dispose(), it also consists of an additional overload, usually protected virtual void Dispose(bool disposing) and a finalizer implementation, which in C# is Done via the ~Classname().

Now, in the past, this pattern was the recommended directly by Microsoft, but they changed that recently and now the recommendation is, that you wrap unmanaged resources in an SafeHandle instance and don’t implement the finalizer at all.

They still recommend to keep the protected virtual Dispose(bool disposing) overload so that derived classes can also get rid of their resources.

Okay, so we got rid of the Finalizer and if we don’t have unmanaged resources to track we need zero SafeHandles!

What about the protected virtual Dispose(bool disposing) overload? Luckily there is a very simple solution to that: stick sealed on the class to deny any inheritance and you’re done!

So in a situation, were you have only Managed Resources that implement IDisposable themselves your implementation can be done in two steps:

1. Implement IDisposable and dispose managed resources in your Dispose() method.
2. Make the class sealed.

The lengthy and slightly confusing documentation for implementing IDispose can be found here and if you made it through this document, be sure to check out the discussions in the associated issues, like this one.

Summary

The using statement in C# prevents an instance escaping it’s Dispose() call by protecting against exceptions and reassignment (either directly or by forwarding by ref).

Objections against using/IDisposable are common but most are easily solved:

Indentation and deeply nested code blocks can be solved via refactoring, Database Connection can be freely disposed, because Connection Pooling speeds up their recreation and the recommended dispose pattern has changed and can be shortcutted in most cases.

Additionally, using/IDisposable gives access to a nice abstraction over try/finally that we can use ourselves whenever we see that pattern repeating.

So please keep on using using!

References

• C# Language Specification, page 265 (https://www.ecma-international.org/publications/files/ECMA-ST/ECMA-334.pdf)

• ObjectDisposedException (https://docs.microsoft.com/en-us/dotnet/api/system.objectdisposedexception?view=netframework-4.8)

• Connection Pooling (https://docs.microsoft.com/en-us/dotnet/framework/data/adonet/connection-pooling)

• SemaphoreSlim (https://docs.microsoft.com/en-us/dotnet/api/system.threading.semaphoreslim?view=netframework-4.8)

• Implementing Dispose (https://docs.microsoft.com/en-us/dotnet/standard/garbage-collection/implementing-dispose)

• Github Issue about “Implementing Dispose” (https://github.com/dotnet/docs/issues/8463)

TLDR

Take a look at the summary.

Workplace Happiness through Autonomy

Being able to choose what, when and how to do things at work - working autonomously - is one of the most important aspects in experiencing happiness at work.

To achieve that kind of independence one must be willing to shoulder additional responsibilities, like choosing worthwile goals, making plans, defining and completing those tasks, measuring progress, managing time spent and re-evaluating those goals.

No matter if you are an employee or self-employed - failure to meet those requirements will results in stress, anxiety and could finally end in you losing that autonomy.

Common Problems

So what could possibly go wrong? The most common problems that might arise and endanger autonomy are:

• Procrastination
• Anxiety
• Distractions
• Getting lost in details

Not everyone suffers from all of those problems, but I consider them to be intrinsic dangers that can come and go and so it makes sense to be aware of them and the effects that they have.

Procrastination

Procrastination is one of the most common problems and it means avoiding a task that needs to be accomplished. The reasons for avoidance are plentiful and may differ not only from person to person but from task to task.

For me personally it is often a task that requires a skill that I am not good with (accounting!) or a task that needs a big context switch that I am unwilling to do (accounting again!) or something I find just plain distasteful (okay, you guessed it, I am really not too fond of accounting).

The risk of procrastination is that high value tasks get postponed or not done at all. In my case that would be not reaching out to a potential customer, but instead implementing yet another feature that noone really asked for or scrambling to file my taxes at the last possible time.

Anxiety

Anxiety is often connected to subconsciously knowing or thinking that something is wrong - for example it could be triggered by a task that you know is important, but are still procrastinating on.

It can also be triggered by exhaustion - working long periods of time without taking any breaks or without any measure of progress can also lead to anxiety.

While being a possible effect of procrastination, Anxiety can also be a cause of further procrastination. If a task is dreaded then it is more likely to be put off for a more opportune moment when our batteries are fully charged and we are in the right mood - a moment that might as well never come.

In addition to that being anxious just doesn’t feel good.

Distractions

When people think about distractions most of the time they think about social media and pings that they get from facebook, whatsapp or twitter (or diaspora, signal and mastodon, depending on your crowd). But they don’t really count - those are beginner level distractions! Why? Because they are so easy to ignore or turn off. The reason is that the source of the distraction is external and the distraction does not feel urgent and so we don’t have to take immediate action.

But what about work e-Mails from a customer or your colleage that has a question or two? Those are grade A distractions! You cannot just ignore them - it’s kind of your job to answer them even if they force a complete context switch.

The result is that those distractions keep us from making progress - especially in those parts of our work that requires deep thought and concentration.

Getting Lost in Details

While distractions keep us from focusing on deep work, there is also the opposite danger: getting lost in the details and not seeing the forest for the trees.

This can happen if no planning is done at all, but in my experience it happens more often when a previously defined goal changes or new facts appear and an existing plan is not adjusted accordingly. An example would be trying to find a way to solve a technical problem in a way that’s not applicable to the situation at hand - something that works in theory but just not in this particular case.

The outcome is often that a wrong path is followed for too long without recognizing it’s actually a dead end. What really should have happened would have been a re-evaluation of the existing plan.

What is Timeboxing?

Timeboxing in a nutshell is a time management technique that pre-allocates time periods for specific tasks.

At the beginning of the timebox, work on the task is started and work on the tasks stops not only when the task itself is completed, but also when the duration of the timebox elapses - whichever comes first.

So timeboxing at it’s core is a very simple time management strategy, it but still provides a couple of advantages over not having any time management strategy at all.

So the core rules are something like that:

• When a timebox begins, work on a task is started.
• Work on a task is stopped when either the task is completed or the duration of the timebox elapses.

Also one can add additional rules to (hopefully) improve it further.

For example I’ve found the following rules useful:

• Timeboxes are always the same length, they are the unit of measurement for all work durations. E.g. a Timebox is 25 Minutes. This task took 2 Timeboxes.
• After a timebox, there is (usually) a break for a couple of minutes. Sometimes when I am feeling in the zone and know what needs to be done, I skip a break and start the next timebox right after the first.
• No distractions are allowed in a Work Timebox. If the distraction can be averted in a couple of seconds and flow is not lost, it’s not counted as a distraction. If you give in to the distraction, the current work timebox is cancelled and must be started from the beginning.

For an example of a timeboxing technique you can take a look at the pomodoro technique, on which most of those rules are loosely based.

No self-directed, autonomous being likes to follow arbitrary rules - especially from a random guy on the internet. That’s why in the following sections I will show how they can help with the aforementioned problems.

Tackling Procrastination

Speaking from personal experience, tackling procrastination is all about starting the dreaded task. It could be something that you haven’t done before and you’ve got no idea how to even start it. Or it’s something you have done in the past and remember it being a real slog.

And that’s where the basic timeboxing rules come in handy because they don’t expect you to solve this unclear, exhausting or impossible seeming task. They only want you to try for a fixed amount of time. That’s quite doable! After that you are free to go, you get a break and you can then try again for another no-strings-attached timebox.

By demanding effort, not results timeboxing frees you to try and give it your best shot. And once you are in the middle of it, you realize that it’s not that bad as you have imagined, so you often keep at it once you have your momentum going.

Alleviating Anxiety

Alleviating anxiety depends on the source of the anxiety, there is no single thing that works in every case, so the question is: what is causing the anxiety?

One source of anxiety could procrastination, so tackling procrastination can instantly relieve some anxiety that has been built up by tasks that were previously put off.

Taking Breaks

Another cause for anxiety are long working hours without breaks. That’s where timeboxing’s rule to stop working after the timebox elapsed is put to good use. When I worked non-stop (except for lunch break) I was often feeling stressed out and anxious at the end of the day. Being continuously reminded to take breaks - even just for a couple of minutes to take a quick walk, take some deep breaths and get some water greatly reduced my stress levels and got rid of my feelings of anxiety.

Tracking Efforts, not Results

In addition to taking breaks, timeboxing’s focus on efforts rather than results helps with anxiety that is connected with feelings of inadequacy like imposter syndrome. So for example instead of being stressed out because you couldn’t finish this single task that you set out to do for the day, you are reminded that you were able to complete 6 timeboxes all the way staying focused.

This subtly changes the thing you worry about from an external factor that you have no control over (the task at hand) to something that you can actually control (your time commitment and effort).

This is btw one of the oldest tricks of stoicism - logic dictates that if you want to be consistently happy than you need to derive pleasure from things that are in your control and not external factors.

Reducing Distractions

Distractions are a common stressor that result in anxiety. So reducing them will in turn reduce that part of anxiety that is being fed by stressful distractions.

Handling Distractions

After you’ve gotten rid of all notifications by switching your mobile to do-not-disturb and closing all social media apps you are only left with the most difficult kinds of distractions: e-Mails and Co-Workers.

Timeboxing offers two great tools that work together to combat the most negative effects of distractions by suspending and batching them.

Suspending Distractions

The first part is suspending distractions by allocating the timebox and then defending the work that is done inside the timebox against intrusions.

Defending the work in the timebox can be as simple as only checking your e-Mails when the allotted timebox for e-Mail based work starts. Or you could check your e-Mails in every break. If something important comes up, then you can dedicate the next timebox to answer that e-Mail or solve that problem.

As the timeboxes are as long as you see reasonable, for example 25 minutes, the longest an e-Mail has to wait is 24 minutes and 59 seconds. IMHO every e-Mail can wait that long to be answered. If not, then I would argue it shouldn’t have been an e-Mail in the first place. If you insist that in your case it’s totally different, that’s fine too - in that case I would suggest you use the power of modern e-Mail Clients and create a ruleset that only notifies you in very specific cases.

With co-workers and colleagues that can question you at any moments notice, it’s a little more difficult. The best way that I’ve found to handle those situations is to clearly communicate that you are in the middle of something and that you will get back to them in X Minutes. Once again, there may be situations where that is not an option, for example if something is actually on fire and you need to run for the exit. In all of those case feel free to stop what you are doing and give in to the distraction.

Getting back to the distractions

After you’ve suspended most distractions the next important thing is actually getting back to them when you can (and try not to procrastinate about it!).

By waiting for the allocated timebox (or the next break) before tackling the problem you benefit from two things:

The first is that people often solve their “urgent” problem themselves or at least add vital information to the problem at hand that was missing in their first inquiry. If you had dropped everything at once, you would have gotten stressed out and lost your progress thanks to the disruption.

The second advantage is that you are able to solve problems in a batch. What if a colleague asks you for info and you wait for your break to answer him and a second colleague needed something as well 5 minutes later? Now instead of being interrupted twice you are able to respond to them in one go.

Getting Lost and Finding your Way

When it comes to removing your blinders and figuring out an alternative, superior approach to the one your currently following, nothing is as effective as taking a break. Taking a break puts your mind into diffuse mode and allows you to connect the dots that your deep work previously created.

Let me point out that that’s just another very good reason to incorporate breaks in your routine, even if you up to this point felt like “yeah, yeah, sure - for those who need them”.

So it’s often when we take a break that we will recombine the new information we previously gathered in the focused work with the old. That’s when we notice that we need to re-evaluate our approach to solving a problem or that we missed a more elegant way to achieve our goal.

Summary

Happiness at work is greatly influenced by autonomy, which makes attaining and keeping autonomy highly desirable. Autonomy itself requires a minimum amount of self-management to maintain and if unaddressed leads to a couple of common problems:

• Procrastination
• Anxiety
• Distractions
• Getting lost in details

Timeboxing is a time-management strategy that can solve or soften most of those problems with just a couple of simple rules:

• Timeboxes have a fixed duration of e.g. 25 minutes
• Working on a task is only done inside a timebox
• When a timebox stops, work on the task stops as well
• No distractions are allowed inside a timebox
• A break follows a completed timebox

References

• Wikipedia Happiness at work (https://en.wikipedia.org/wiki/Happiness_at_work)
• Wikipedia Procrastination (https://en.wikipedia.org/wiki/Procrastination)
• Wikipedia Pomodoro Technique (https://en.wikipedia.org/wiki/Pomodoro_Technique)

TLDR;

I wrote a small C# dotnetcore commandline tool for windows to generate svg badges like:

That you can install over chocolatey:

choco install badger
badger.exe -o result-success.svg -l "Testresults" -r "100/100" --lc #444444ff --rc #00ff00ff

Background

I wanted my CI build results to contain nice svg badges like the ones above and I didn’t want to use some internet service, because the build itself shouldn’t depend on internet access and external services.

An option would be to design the svg using a vector design tool like inkscape and export an svg and then use a script to replace the corresponding colors and text.

But that has it’s own drawbacks - it’s very difficult to modify later on and positioning the items before you know the text is tricky - you might need to make your script smart enough to adjust the size of items as well.

Why not go all the way then and write a small commandline application that can create svgs from scratch? So that’s what I did.

Creating Svgs with SkiaSharp

For generating the svg files, I chose the excellent SkiaSharp library (about which I’ve written already here), which has a built-in Svg Backend. Thanks to SkiaSharp, creating a small commandline app that can create an Svg File consists basically of parsing the input and calling the right drawing methods.

I’ve called the tool badger and uploaded it to github.

If you are interested in the code, check out SvgService.cs which handles the Svg-File export and calls BadgeService.cs in turn, which handles the actual drawing.

You can download the source code from github and build it yourself:

git clone https://github.com/8/badger
cd badger
build

Installation with Chocolatey

Now the only thing that’s missing is a nice way to install it. I wasn’t in the mood of writing an WiX or an nsis installer, because their overkill for a simple commandline app that you can basically xcopy deploy.

On the other hand that doesn’t solve the question of where to upload the installer and how to find the link when you later on need it.

For installing applications and upgrading them on my windows machines, I am a fan of chocolatey, which is package manager for windows, that’s based on nuget. Chocolatey puts the installed executable automatically on your PATH, which is really useful for all commandline tools.

For Windows, I’ve created a package of the compiled executable and pushed it to chocolatey. You can install it using:

choco install badger

It’s based on the .NET 4.7 Framework, so you need to make sure that it’s installed.

Using the tool

After you’ve build the tool or installed it with chocolatey you can invoke it like that:

badger.exe -o result-success.svg -l "Testresults" -r "100/100" --lc #444444ff --rc #00ff00ff

Which will create the following svg file:

Resources

• badger (https://github.com/8/badger)
• SkiaSharp (https://github.com/mono/SkiaSharp)
• chocolatey (https://chocolatey.org)

TLDR;

I’ve created a docker cli cheatsheet in different colors. If you want to download the pdf scroll to the bottom.

Background & Goals

I’ve been working with docker extensively for the last few months and while I find the technology really useful, I’ve noticed that newcomers are often overwhelmed by the slew of commands and options available and I am only talking about running some containers - not to mention orchestrating them or building images.

As more and more people have been asking me what they can do with docker and what’s the command for that, I’ve thought about writing up the commands that I found most useful.

The solution

This week, I finally had enough time and created a 1-page docker cli cheatsheet that you can download as a pdf and print.

If you happen to spot a mistake or I am missing your favourite docker command/option, please feel free to message me or comment, maybe I can squeeze it in.

Because I am not a designer, I’ve generated the docker cheatsheet in a couple of different color themes in hope that you might find a color that you’ll like.

TLDR;

If you need to access OAuth 2.0 protected resources, than go ahead and use the official SDK from your OAuth 2.0 provider, if available.

If you want to implement the server-side you might want to reconsider, as OAuth2 is both complicated and doesn’t do what you need on it’s own.

Background & Goals

Recently I’ve taken a look at OAuth2, to find out if it’s a good fit for projects I am working on. To be specific, I was wondering if I should implement it as my means of Authentication.

I was looking for a framework that:

1. should be simple to understand / straightforward to implement
2. should allow my app to receive basic information about the user
3. should allow my server to deliver content based on the users identity

In this article I present why I don’t think that OAuth 2.0 meets those requirements. Should your requirements look like mine, then I suggest you rethink if you really want to use OAuth 2.0.

Disclaimer: Every use case is different - so if you find that it still makes sense to deploy OAuth 2.0 in your system then don’t feel discouraged to do so!

Note: If you are only interested in accessing OAuth 2.0 protected content from another service provider - that is if you are only implementing the client, then you have no choice anyway and can go ahead!

First let’s take a look and what OAuth 2.0 is and who is using it.

So let’s get started!

What is OAuth 2.0?

The short version:
OAuth 2.0 is a delegated authorization framework for the enterprise.

For the long version, you can read all about OAuth 2.0 in it’s 70+ pages rfc 6749.
It also has a dedicated homepage at https://oauth.net/2 and a wikipedia page.

When talking about resources to read about OAuth 2.0, I think it’s worth mentioning that the lead author Eran Hammer was unhappy about the path that it has taken and resigned and had his name removed from the project. You can read up on his reasons for doing so on his blog.

Also we will find out that there are 2 additional RFCs that you may need to get familiar with,

Who is using it?

If you are going to use a technology, it doesn’t hurt to take a look at who is also using it. In this case, it’s the big players:

• Amazon
• Facebook
• Google
• Microsoft
• Twitter

If you want to take a look yourself, here is a wikipedia page that lists a number of known OAuth service providers. But not all of them implement 2.0 - many are still using 1.0 and 1.0a.

On one hand this looks rather promising, as a lot of big companies have embraced OAuth 2.0. On the other hand quite a few decided to not upgrade from an older implementation like 1.0 to the newer 2.0 version, even though Version 2.0 was published in October 2012.

How does it work?

At it’s heart, OAuth 2.0 specifies a flow that defines:

• which party is calling which party
• in what order those calls are made
• and what is the data that is being exchanged

To understand it, you need to learn it’s terminology and learn about the specific flows it supports. The flows take advantange of the terminology, so you need to understand the terminology first - that’s why I’ll start with those.

OAuth 2.0 Terminology

To make sense of it, you’ll need to learn it’s terminology.

Involved Parties

First you need to know about the involved parties:

Client : A client application, e.g. your web-, mobile- or desktop-app.

Resource Owner : The user of your app who also happens to have access to some content on an OAuth 2.0 protected server.

Resource Server : An OAuth 2.0 protected server that serves some content if you hand it a valid Access Token.

Authorization Server : A server that the Resource Server trusts. It hosts a login page, that the user logs into. After the user logs in successfully and confirms the request, it will issue an Authorization Grant. It will also exchange an Authorization Grant for an Access Token for the client.

Exchanged Objects

Then you need to know about the objects being being exchanged between those parties:

Resource : A Resource is some user content protected by OAuth 2.0 and provided by a Resource Server.

Authorization Request : The Authorization Request is what happens when your App requests access to a Resource of the User.

Authorization Grant : The Authorizaton Grant is returned as a response on a Client’s successful Authorization Request. It can be exchanged for an Access Token.

Access Token : A token that is used to retrieve content from a Resource Server instead of user credentials. It is issued by the Authorization Server.

Refresh Token : An optional token that can be returned by the Authorization Server in addition to the Access Token. It can be used to retrieve additional Access Token if for example the original Access Token expires.

Authorization Grant Types

Then there are different Authorization Grant Types and the choice of the Grant Type influences how the Access Token can be obtained. There are four base Authorization Grant Types specified in OAuth 2.0:

1. Authorization Code
2. Implicit
3. Resource Owner Password Credentials
4. Client Credentials

The specifics are not terribly important at the moment - you may want to skim them, as I will call out the differences between them later on. The first one, Authorizaton Code, is the most important one as we will see.

Authorization Code

This is the classic OAuth flow.

1. The Client redirects the Resource Owner to the Authentication Server.
2. The Client logs in, confirms the requested access rights.
3. The Authentication Server redirects back to the Client.
4. The Client exchanges the Authorization Code to the Access Token by calling the Authentication Server.

Note: This is the preferred Authorization Grant Type.

Implicit

1. The Client redirects the Resource Owner to the Authentication Server.
2. The Client logs in, confirms the requested access rights.
3. The Authentication Server responds directly with the Access Token.

Resource Owner Password Credentials

1. The Client asks the user for username and password.
2. The Client sends the username and password to the Authentication Server.
3. The Authorization Server responds with the Access Token.

Client Credentials

1. The Client sends his own username and password to the Authentication Server.
2. The Authorization Server responds with the Access Token.

OAuth 2.0 Flow

An OAuth flow lists the number and order of steps that are executed between the involved parties. The specific steps depend on the authorization grant type being used.

Let’s start with a simple case, as shown in the abstract protocol flow that is also sketched out in the RFC. It can be visualized in a sequence diagram like this:

This flow would be used if the Authorization Grant Type Resource Owner Password Credentials is used - that’s why the Client would ask the Resource Owner directly.

This flow, although simple, is not the recommended approach. In this flow the Authorization Request is sent directly to the Resource Owner, but in the recommended approach the Client should send it to the Authorization Server as an intermediary for enhanced security of the user credentials.

The preferred flow uses the Authorization Grant Type Authorization Code and looks like that:

But wait, that’s not all - the previous sequence diagramm was still missing the flow for using Refresh Tokens. So here we go with Refresh Tokens added to the mix:

Now we know a little about how the flow looks like, let’s take a closer look at one of the objects that is central to the flow, namely the Access Token.

What are Access Tokens made of?

If you expect that the spec defines how the Access Token should look like, you will be disappointed, because the Access Token is defined only very vaguely as a string.

The RFC does not get any more specific than that - quite the opposite, it states that Access Tokens can have different formats and that those are outside the scope of this RFC but it links to another RFC for further reading.

Bearer Tokens

To get to know more about them you need to read the RFC6750 (The OAuth 2.0 Authorization Framework: Bearer Token Usage) which specifies how the Tokens should be used to access the Resource Server.

Even though RFC6750 specifies how Bearer Tokens are to be used, it does not tell us how a reasonable Access Token should be implemented.

JWT - JSON Web Token

This is were JSON Web Tokens come in!

In yet another RFC7519 we learn about an implementation of an Access Token that can be used in OAuth 2.0, called JWT (JSON Web Token).

JWTs are a topic on their own and discussing them in detail is out of scope of this article.

Their main characteristics are:

• they hold certain predefined parameters
• they are extensible with custom parameters
• they can be signed and/or encrypted
• they can be serialized to a string

Those properties make them a perfect fit for Access Tokens. In addition to that, those properties allow them to be self-contained. That means they can contain all necessary information for the Resource Server to verify if the Resource Request of the Client is valid and should be served - the Resource Server doesn’t need to call the Authentication Server or look up additional information.

Now we know enough about what OAuth 2.0 is and how it can be implemented.

Is OAuth 2.0 simple to understand?

To come back to the first question I posed at the beginning: Is it simple to understand and simple to implement?

Well, when I started reading up on OAuth 2.0 I didn’t find a nice succinct writeup on the internet for how it works and what it does.

Now after having read through a few resources, I feel like I know why - because it’s complicated.

Basically, you would have to read, understand and implement at least those three RFCs.

So I would answer that question with: No, I don’t think so.

But that’s not all, IMHO it’s also missing some basic features…

OAuth 2.0 is missing Authentication

When you start reading about OAuth 2.0 you soon will find out that the “Auth” part of it’s name stands for Authorization (and not Authentication) as made clear by both the homepage’s first sentence:

“OAuth 2.0 is the industry-standard protocol for authorization.”

I don’t know if I agree with the first part of the sentence, but the “for authorization” part is spot on.

And the rfc’s header:

“The OAuth 2.0 Authorization Framework”

Authorization vs. Authentication

But why the hair splitting, what’s the difference between Authentication and Authorization?

According to Stackoverflow:

Authentication is the process of ascertaining that somebody really is who he claims to be.
Authorization refers to rules that determine who is allowed to do what. E.g. Adam may be authorized to create and delete databases, while Usama is only authorised to read.

or:

Authentication = login + password (who you are) Authorization = permissions (what you are allowed to do)

OAuth 2.0 doesn’t solve the problem of Authentication, only the problem of Authorization.

That’s too bad, because getting some base info about a user is almost always a requirement for an application. At least you want to show the user as signed in and you want to have some basic info that you can display, like a username, so that the user knows under which account he is signed in.

When I started looking at OAuth 2.0, I was under the assumption that it would provide a solution for this very common problem.

But with a vanilla OAuth 2.0 implementation you are out of luck if you expect that it provides a standard way of handling user profile information.

Because those misconceptions are a widely held belief, there is a nice article on the official site that explains common pitfalls.

In OAuth, the token is designed to be opaque to the client, but in the context of a user authentication, the client needs to be able to derive some information from the token.

This torpedoes my goal that I want my app to receive basic information about the user.

On the other hand, the server that is serving the protected content has the same problem as the provided Access Token is not inherently linked to a specific user. Therefore the Resource Server has no idea which user has originated the request.

Or to quote the article once more:

The protected resource is not generally going to be in a position to tell if the user is still present by the token alone, since by the very nature and design of the OAuth protocol the user will not be available on the connection between the client and protected resource.

Which means that my last goal - that the server should be able to provide info based on the users identity - also doesn’t work out of the box.

Solving the User Information Problem

We can of course solve the problem and add the support for the missing features. One option that comes to mind is:

• Add a User Info Token
• Host a User Info Endpoint

Adding a User Info Token

One workaround to provide that information, would be extending the Access Token Response.

If you take a look at the example of a complete Access Token Response as specified in th RFC:

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Cache-Control: no-store
Pragma: no-cache

{
  "access_token":"2YotnFZFEjr1zCsicMWpAA",
  "token_type":"example",
  "expires_in":3600,
  "refresh_token":"tGzv3JOkF0XG5Qx2TlKWIA",
  "example_parameter":"example_value"
}

You can see that additional parameters, like example_parameter can be part of the response. This is specifically supported by the spec by requiring that clients must ignore unrecognized response parameters.

So we are free to add an additional parameter. For example we could add a parameter and set it’s value to another serialized JWT that holds the users profile information. Let’s call this additional token User Profile Token.

The response could then look like this:

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Cache-Control: no-store
Pragma: no-cache

{
  "access_token":"2YotnFZFEjr1zCsicMWpAA",
  "token_type":"example",
  "expires_in":3600,
  "refresh_token":"tGzv3JOkF0XG5Qx2TlKWIA",
  "user_profile_token":"..."
}

Now a Client Application is handed profile information of the user in addition to an Access Token and can mirror that information back to the user of the app.

Host a User Info Endpoint

Additionally we could implement a Resource Server that answers a request containing a valid Access Token with our new User Profile Token.

That way, should the User Info returned when retrieving an Access Token have gone stale and is not up to date anymore, a new info can be retrieved.

Open ID Connect

But if that’s such a common use case, wouldn’t it be great if the approach of using a second token and additionally supplying a Resource Server that works as a User Info Endpoint would also be part of a standard?!

That’s why a similiar approach is already standardized in yet another spec called Open ID Connect. Therefore Open ID Connect is basically an authentication layer on top of OAuth 2.0.

So before going down the custom route, it may make more sense to implement Open ID Connect instead - but either way, it’s not part of OAuth 2.0.

In the end having no support for Authentication out of the box is a solvable, although annoying problem.

Use Cases

Now we know what OAuth 2.0 can’t do, what about actual use cases? Let’s go through some usage scenarios based on different application types.

Previously, I’ve talked about the different Authorization Grant Types and looked at abstract flows. Now I want to map them to typical application types that I often implement:

1. Web App
2. Mobile App
3. Desktop App
4. Console App
5. Privileged Service

Web App

Consider building a Web App that wants to access a Resource on an Http Server that is protected via OAuth 2.0.

That’s the flow for which OAuth 2.0 was made!

In this diagram I’ve modelled the authorization code flow including the browser, because there are two important things going on:

1. The Web App initiates the loading of the Authentication Server’s login page and cannot get a hold of the credentials of the user. It only receives the Authorization Code.
   That’s especially nice in case of third party apps as you wouldn’t trust them with your users login and password.

2. The Web App’s Backend server exchanges the Authorization Code against the Access Token. Therefore the Browser (and also the User) does not get a hold of the Access Token.

That’s also nice, because the token is not exposed in an internet cafe or on an unlocked laptop.

Mobile App

Consider building a mobile app that wants to access a Resource on a Http Server that is protected via OAuth 2.0.

There are basically two options:

1. Either use a WebView to load the login page of the Authentication Server inside of the app and therefore follow the Implicit Code Flow.

2. Or ask the User directly for his username and password and follow the Resource Owner Password Credentials flow.

Both of these approaches are problematic. In the second case, it’s clear that the user credentials are exposed to the app, but even the first case is also dangerous, because:

1. Instead of using a WebView the app could simply spoof the login page, as the user can’t be sure that he is inside a sandboxed browser.

2. Even if the Application is trustworthy and uses a WebView and really loads your login page, you are training your users to put trust in your login ui even if they cannot validate the origin. Which in turn increases the likelyhood of them falling for phishing attacks later on.

Desktop App

As far as accessing OAuth 2.0 protected resources, building a desktop app is very similiar to a mobile app.

You can either use the Implicit Flow or the Resource Owner Password Credential Flow.

The main difference is that WebViews are usually more cumbersome to implement, so it’s way easier to go for the Resource Owner Password Credential Flow.

Console App

If you are building a console app you will be unable to redirect to the Authentication Server’s login page, therefore you will need ask the user for his or her username and password and follow the Resource Owner Password Credentials flow.

Privileged Service

When you are developing a privileged service that runs without user interaction you would implement the Client Credentials flow.

For example you could create a Resource Owner for the Service and provide the credentials in a configuration file.

Summary

In all but the third party Web App scenario OAuth still depends trusting the Application to do the right thing. Either to not spoof the login screen or to handle the user credentials carefully.

Implementations

I am not going to go into much detail about specific implementations as there are many different options and they depend on usage scenario and language preferences. Instead I’ll give only some general recommendations for picking a client library based on what I’ve found.

Because there a lot of different flavours of OAuth, you probably can’t reuse the same client library for all providers.

So if you need to integrate with a specific OAuth Provider, probably the best option is to first check if there is an official client library or SDK for your favourite programming language available that already wraps the OAuth calls.

An additional advantage of that approach is that you get all the other capabilities of their API wrapped as well.

For example Facebook provides an official PHP based SDK.

Using Unofficial Client SDKs

If there is no SDK available for your favourite programming language, the next best thing is probably an unofficial opensource SDK for a specific OAuth Provider.

Using OAuth Client Libraries

In case you want to integrate with multiple OAuth Providers, you could also take a look at client libraries that support multiple OAuth Providers. As they are focused on the multiple providers, they usually handle only the OAuth part, so you still have to handle the provider specific resource requests yourself.

An example is the OAuth2 client library for C#. You can see that to support all those different providers a custom implementation is used for each.

As an alternative you can write the OAuth calls yourself, but you are probably saving time by not reinventing the wheel.

Server Libraries

First of, there is no “default” or “goto” implementation of OAuth 2.0. The reason being that the different companies that are part of the OAuth 2.0 working group wouldn’t agree on one. Everyone wants their language and implementation to be the “correct” one.

If you are looking for general purpose client or server OAuth libraries, a good place to start is the official OAuth 2.0 Website’s list of libraries.

Conclusion

If you are developing a Client Web Application that wants to access Resources that are protected by a third party OAuth 2.0 provider you will have to implement the client side of OAuth, but with a matching SDK that’s rather straightforward.

Things look different on the server-side, as there are a couple of problems with OAuth 2.0 - for me the following are the most pressing:

• Too complicated for what it does, you need to implement at least three RFCs
• Missing basic functionality for common scenarios like Authentication
• Limited usefulness outside of specific scenarios like third party Web Apps
• Extensibility and vagueness cause differences in implementation

All of the above make me wary of implementing OAuth 2.0 server-side and my advise is to stay away unless you have very specific reasons not to.

What might those reasons be?

• You are working in an enterprise environment
• You expect lots of custom built apps
• Many of them are WebApp
• You plan to implement Open ID Connect
• You are willing to spent more time and money than is strictly necessary to solve the problem

If that’s not you, than you are probably better off skipping OAuth 2.0.

References

• OAuth 2.0. RFC (https://tools.ietf.org/html/rfc6749)
• OAuth 2.0 Homepage (https://oauth.net/2)
• OAuth Wikipedia Page (https://en.wikipedia.org/wiki/OAuth)
• OpenID Connect (http://openid.net/connect/)
• OAuth 2.0 Lead Author resigns Blogpost(https://hueniverse.com/oauth-2-0-and-the-road-to-hell-8eec45921529)
• OAuth 2.0 Common Authentication Pitfalls (https://oauth.net/articles/authentication)
• OAuth 2.0 Authorization Framework: Bearer Token Usage RFC 6750 (https://tools.ietf.org/html/rfc6750)
• OAuth 2.0 Client and Server Libraries (https://oauth.net/code/)
• [ (https://github.com/titarenko/OAuth2)]

TLDR;

Abstract: This article is about getting contributor stats from a git repository.

Solution:

1. To get the number of commits for each user execute git shortlog -sn --all
2. To get the number of lines added and delete by a specific user install q and then execute: git log --author="authorsname" --format=tformat: --numstat | q -t "select sum(c1), sum(c2) from -"

Conclusion:

1. q is cool, put it in your toolbelt.
2. Don’t use those stats as a base for calculating salaries.

Overview

Last week I wanted to retrieve simple stats about contributors from a git repository.

I came up with the following two stats:

1. Commits per Contributor
2. Lines changed per Contributor

Stats from git

Commits per Contributor

Getting the number of commits for each contributor is easy with git, just execute:

git shortlog -sn --all

and you will get an output like this:

  3  author

Which will show you the number of commits for each user.

The command is broken up as follows:

• git shortlog summarizes git log
• -s suppresses the description of the commits and shows only the commit count
• -n sorts the output by most commits descending
• --all show it for all branches

Lines changed per Contributor

The second thing I was interested in, was the number of lines changed.

Git is able to tell you the number of lines changed per file for each commit. When we now restrict the shown commits only to a specific author, we will be able to get a list of all changes he or she did for all files.

This can be accomplished with the following git command: git log --author="authorsname" --format=tformat: --numstat

The command can be broken down like so:

• git log shows info about commits
• --author="name" shows only info about a specific author. You could also use --committer="name" if your author and committer are always the same.
• --format=tformat: is a nice one, it uses an empty tformat string to basically get rid of every information, so this outputs an empty string for each commit.
• --numstat adds the number of lines added and delete for each file of the commit.

This leaves us with one remaining problem: How to sum up all those single lines in case we don’t care about the name of those files? Well, we will need to use another tool to do it, the question is, which one?

I had the following ideas:

1. Write a custom shell script by glueing together different commandline tools
2. Write a custom commandline tool
3. Use Excel

So I looked at all of them:

1. A shell script would need to use additional tools and would probably be pretty brittle - it would break on different versions of the tools used, depend on a specific shell and have very slim chances of working cross platform. If I am forced to write something, new I’d rather write a small commandline tool then.
2. Nah, on second thought, I don’t want to do that either, I was looking for something more out of the box for such a generic task.
3. Are you kidding me, we are at the commandline here?!

Then it dawned on me: What I wanted to do was calculate aggregates over columns - and there is already a great way to express that: SQL!

The question is, how can I run a sql statement against that output without the overhead of pushing it to relational database first?

This is were the great little command tool q (https://github.com/harelba/q) comes in!

It can run a sql query against data coming from csv and STDIN. The command that I used (including the sql query that calculates the sums of all added and deleted files) for q is q -t "select sum(c1), sum(c2) from -".

The query can broken down as follows:

• -t use tab as the separator between columns
• sum(c1) the sum of the first column
• sum(c2) the sum of the second column
• from - from STDIN

Now we only need to pipe the output of the git command into our call of q.

Therefore to get the number of lines changed per contributor you need to:

1. Install q as explained here.
2. Execute

git log --author="authorsname" --format=tformat: --numstat | q -t "select sum(c1), sum(c2) from -"

3. And you will get an output like this:

4       1

which is the number of added and deleted rows.

Svn

If you are working with svn, you could try the git svn bridge and clone a repo from svn first, like so: git svn clone url_to_your_repository

Conclusion

While getting stats like commits per contributor and lines changed is rather easy using git, those stats may not be as useful as you think for measuring developer contribution:

1. They are not comparable in and off themselves. Comparing the number of added F# lines with added lines to a XML Config file is maybe not the best idea.

2. They can easily be gamed. For example by adding and removing additional lines, and splitting changes over multiple commits and multiple lines - you will get what you measure.

3. Last but not least: They measure almost always the wrong thing. Nobody cares about lines and commits - imho far more important are code coverage, issues solved, good communication, on boarding new team members and so on.

So take stats like this with a grain of salt, or better a spoon and don’t base any important decisions upon them!

On one hand, maybe those stats aren’t so great after all, on their own, they seem more like vanity metrics instead of actionable data. But on the other hand, I’ve put a new and flexible tool into my toolbelt today :)

References

• git shortlog (https://git-scm.com/docs/git-shortlog)
• git log (https://git-scm.com/docs/git-log)
• q (https://github.com/harelba/q)
• install q (http://harelba.github.io/q/install.html)

TLDR;

1. For repeated multiple choice data entry you might want to consider alternative forms of input, like e.g. a gamepad.
2. Gamepad support is provided by XInput on Windows.
3. SendInput() allows sending of Keyboard events to foreground applications
4. Combine both in a simple application
5. Add Hotkey support to your Angular2 app
6. Now you are able to control your app with an XBox Controller

Who would want that?

A customer had an interesting idea - he asked if there were alternative user input devices available that could be used for data entry for an Angular2 Intranet Application that I was working on.

His reasoning was that the WebApp was used for data entry - and lots of it! After login, the application basically consists of a main loop that would present it’s user with visual and textual information and would ask the user to take a decision based on it. As the possible input is limited and speed for data entry is paramount, hotkeys support was introduced. I’ve wrote my take on hotkey support here. But additionally the customer was worried about the amount of stress that an employee could suffer from repeated usage.

The customer asked for ergonomic input devices that could be used in addition to a keyboard. The idea was to allow the users to freely switch between different devices if he felt uncomfortable using one.

Why the XBox Controller?

An ergonomic input device in widespread use that is also within budget are Gamepads. They are basically made for varying methods of input and allow extended usage with minimum strain.

Additionally I knew that the WebApp was used as an intranet application and the customer was using Windows Machines, which do have native support for XBox Controllers and on which additional software could be installed.

Support for gamepads and joysticks on Windows is provided via XInput or the now deprecated DirectInput API - both of them are a part of DirectX.

The XBox Controller works with XInput and DirectInput and most of the third party Gamepads provide a hardware switch for changing between XInput and DirectInput. This comes in handy if your game or app supports only one API, but not the other.

Getting access to the XBox Controller

So the first obstacle was to get access to the XBox Controller, as you cannot get access to DirectX (and specifically XInput) from an Angular Application directly. For this, a native Windows Application was necessary.

I opted for a C# Wpf Application and chose SharpDX (http://sharpdx.org), which is a thin wrapper for the C++ DirectX Api.

Polling the Controller

XInput comes with a poll based API, which is perfect for it’s natural audience, that is games. They usually poll the state of the controllers inside of their main loop and do immediate mode rendering - it all fits together.

Example of polling the Keystroke:

var controller = new Controller(UserIndex.One);
Keystroke keystroke;
var result = controller.GetKeystroke(DeviceQueryType.Gamepad, out keystroke);
if (result.Success)
    ...

or the more concise :

var controller = new Controller(UserIndex.One);
if (controller.GetKeystroke(DeviceQueryType.Gamepad, out Keystroke keystroke).Success)
    ...

From Polling to Pushing

In this case however, I am creating a normal Windows Desktop Application using retained mode drawing and I would prefer a push based interface for the gamepad events. That’s why I’ve chosen to wrap the poll based API inside of my application with push based events.

For exposing the push events to the rest of the application, you can use c# events or plain callbacks using delegates, but I’ve chosen Reactive Extensions as I greatly prefer the interface.

So my interface looks like this:

public interface IXInputService
{
    bool IsListening { get; set; }
    IObservable<Keystroke> Keystrokes { get; }
    IObservable<ControllerConnected> Connected { get; }
}

For implementing the continuous polling there are also a couple of options, for example a timer, a delayed task, a thread pool thread or the Application Idle event.

I’ve chosen to spin up a dedicated Thread, as the thread is both long running and the timing between each Poll request is fixed.

To test it out, I created a small Wpf prototype app (XInput2Key), that is able to read out the state of the gamepad’s buttons and shows if they are currently pressed.

Interfacing with Angular2

Now I was able to read out the gamepad input and the only thing that remains is to interface with the angular2 application. As I mentioned previously, the application already had hotkey support, so the obvious choice was to just send keyboard input to the angular application.

This has the additional advantage that the applications are very loosely coupled and you can use one without the other. This allows using the XInput application to work with any other application as well, as long as the target app has somekind of hotkey support.

As I already took a dependency on the host operating system being windows (for the DirectX support), I took a quick look at the native Windows Api and chose the SendInput() function. This function allows sending of keystrokes and mouse events to the foreground application, which in my case would be a browser with the loaded angular2 application.

To access the native C WinApi Method, I needed [DllImport] declarations for SendInput() which I luckily found on PInvoke.net.

Thanks to both SharpDX and PInvoke.Net I was quickly able to throw together a prototype application that:

1. reads in a config file for mapping the gamepad buttons -> keyboard keys
2. listens for gamepad button presses using XInput
3. maps them to keystrokes
4. sends the keystrokes to the foreground application using SendInput()

Sample App

I’ve pushed the resulting app to github (https://github.com/8/XInput2Key) incase you want to try it yourself.

If you check the Checkbox ‘Is Emulating Keys’, then the mapped keyboard input is sent to the foreground window. If you open up an instance of your favourite text editor, you can try it out.

The only caveat is that due to security constraints of SendInput() it cannot send input to applications running with administrator rights, if the app isn’t started with administrator rights itself.

Setting up Angular2 to deal with Hotkeys

For hotkey support in angular I am basically using the javascript library mousetrap with an angular2 wrapper. If you’re interested in how I’ve done that, you can check out my blog post about using Hotkeys in Angular2.

References

• SharpDX (http://sharpdx.org)
• XInput and DirectInput
• PInvoke.net (http://pinvoke.net)
• SendInput() function (https://msdn.microsoft.com/en-us/library/windows/desktop/ms646310(v=vs.85).aspx)
• Mousetrap Javascript Library (https://craig.is/killing/mice)
• XInput2Key Sample App (https://github.com/8/XInput2Key)
• Blog Post: Hotkeys in Angular2 (http://lostindetails.com/blog/post/Hotkeys-in-Angular2)

Background

For the app that I was building for a customer, I needed hotkey support for Angular2. For a plain old javascript web app, I’ve used the excellent javascript library mousetrap (https://craig.is/killing/mice) to great success and I wanted to use it in angular2 app as well.

Mousetrap for angular => angular2-hotkeys

As it turns out, somebody already created an nice angular2 wrapper for mousetrap called angular2-hotkeys (https://github.com/brtnshrdr/angular2-hotkeys) that wraps mousetrap and allows you to import a HotkeysService and register keys for it.

To install it, simply follow the instructions in the README.

Now a component can just request the HotkeysService in it’s constructor and register a hotkey for itself by invoking the HotkeysService.add() method.

Additionally, the component should also remove the hotkey once it gets destroyed. To do this, we store the returned value of the HotkeysService.add() method and supply it as an argument to the HotkeysService.remove() method when the component is destroyed.

In Angular, this can be done by implementing OnDestroy and it’s ngOnDestroy method. When the component gets destroyed, angular invokes the method and and the previously registered hotkey is removed.

A complete example could look like this:

import { Component, OnDestroy } from '@angular/core';
import { HotkeysService, Hotkey } from 'angular2-hotkeys';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent implements OnDestroy {
  title = 'app works!';
  hotkeyCtrlLeft: Hotkey | Hotkey[];
  hotkeyCtrlRight: Hotkey | Hotkey[];

  constructor(private hotkeysService: HotkeysService) {
    this.hotkeyCtrlLeft = hotkeysService.add(new Hotkey('ctrl+left', this.ctrlLeftPressed));
    this.hotkeyCtrlRight = hotkeysService.add(new Hotkey('ctrl+right', this.ctrlRightPressed));
  }

  ctrlLeftPressed = (event: KeyboardEvent, combo: string): boolean => {
    this.title = 'ctrl+left pressed';
    return true;
  }

  ctrlRightPressed = (event: KeyboardEvent, combo: string): boolean => {
    this.title = 'ctrl+right pressed';
    return true;
  }

  ngOnDestroy() {
    this.hotkeysService.remove(this.hotkeyCtrlLeft);
    this.hotkeysService.remove(this.hotkeyCtrlRight);
  }
}

Beyond “Hello World!”

Now this works fine for a simple app, but there are a couple of problems:

• If one component registers a hotkey and a second component also registered the same hotkey, the previous subscription would be overriden.
• Additionally the subscription / unsubscription logic leakes into each and every component that wants to register a hotkey.
• The Hotkey events are not the flexible Observable<T> as we have come to expect in angular.
• Keys are hardcoded inside of each component and therefore difficult to change

Wrapping Hotkeys in a CommandService

To solve that problem, I’ve introduced a CommandService. It’s basically an EventAggregator, that upon initialization reads in a config.json that specifies which keys should be mapped to which commands. It exposes an Observable and registers all the hotkeys specified in the config.json.

Everytime one of those keys are pressed, it triggers the corresponding commands. Instead of importing the HotkeysService itself, all components import the CommandService and subscribe to it’s observable. If the user presses a registered hotkey, a Command is triggered and the components check if they are interested in the comand and if so, take action.

Besides allowing easy updating of the hotkeys by editing the config.json, this moves the hotkey registration code to one place, which makes switching the hotkeys library a breeze (in case that should be necessary in the future). This approach also captures the essence of what the hotkeys are doing - they are issuing a command to components. It also allows reusing the CommandService to explicitedly raise those commands from other components.

An implementation of the CommandService looks like that:

CommandService.ts

import { Injectable } from '@angular/core';
import { Http } from '@angular/http';
import { HotkeysService, Hotkey } from 'angular2-hotkeys';
import { Subject } from 'rxjs/Subject';
import { Observable } from 'rxjs/Observable';

class HotkeyConfig {
  [key: string]: string[];
}

class ConfigModel {
  hotkeys: HotkeyConfig;
}

export class Command {
  name: string;
  combo: string;
  ev: KeyboardEvent;
}

@Injectable()
export class CommandService {

  private subject: Subject<Command>;
  commands: Observable<Command>;

  constructor(private hotkeysService: HotkeysService,
              private http: Http) {
    this.subject = new Subject<Command>();
    this.commands = this.subject.asObservable();
    this.http.get('assets/config.json').toPromise()
      .then(r => r.json() as ConfigModel)
      .then(c => {
        for (const key in c.hotkeys) {
          const commands = c.hotkeys[key];
          hotkeysService.add(new Hotkey(key, (ev, combo) => this.hotkey(ev, combo, commands)));
        }
      });
  }

  hotkey(ev: KeyboardEvent, combo: string, commands: string[]): boolean {
    commands.forEach(c => {
      const command = {
        name: c,
        ev: ev,
        combo: combo
      } as Command;
      this.subject.next(command);
    });
    return true;
  }
}

An config.json example:

{
  "hotkeys": {
    "left": [ "MainComponent.MoveLeft" ],
    "right": [ "MainComponent.MoveRight" ],
    "ctrl+left": [ "AppComponent.Back", "MainComponent.MoveLeft" ],
    "ctrl+right": [ "AppComponent.Forward", "MainComponent.MoveRight" ]
  }
}

A consuming Component would look like that:

MainComponent.ts

import { Component, OnDestroy } from '@angular/core';
import { Command, CommandService } from './command.service';
import { Subscription } from 'rxjs/Subscription';

@Component({
  moduleId: module.id,
  selector: 'main',
  templateUrl: 'main.component.html'
})
export class MainComponent implements OnDestroy {
  command: string = 'None';
  subscription: Subscription;
  constructor(private commandService: CommandService) {
    this.subscription = commandService.commands.subscribe(c => this.handleCommand(c));
  }
  handleCommand(command: Command) {
    switch (command.name) {
      case 'MainComponent.MoveLeft': this.command = 'left!'; break;
      case 'MainComponent.MoveRight': this.command = 'right!'; break;
    }
  }
  ngOnDestroy() {
    this.subscription.unsubscribe();
  }
}

Sample App

I’ve pushed a simple sample app that uses the CommandService to github (https://github.com/8/hotkey-sample).

References

• Mousetrap (https://craig.is/killing/mice)
• angular2-hotkeys (https://github.com/brtnshrdr/angular2-hotkeys)
• My Sample App with the CommandService (https://github.com/8/hotkey-sample)

TLDR;

To get OCR in C# Console- Wpf- or WinForms-App:

0. run on a modern Windows Version (e.g.: Win10)
1. add nuget UwpDesktop
2. add the following code:

var engine = OcrEngine.TryCreateFromLanguage(new Windows.Globalization.Language("en-US"));
string filePath = TestData.GetFilePath("testimage.png");
var file = await Windows.Storage.StorageFile.GetFileFromPathAsync(filePath);
var stream = await file.OpenAsync(Windows.Storage.FileAccessMode.Read);
var decoder = await Windows.Graphics.Imaging.BitmapDecoder.CreateAsync(stream);
var softwareBitmap = await decoder.GetSoftwareBitmapAsync();
var ocrResult = await engine.RecognizeAsync(softwareBitmap);
Console.WriteLine(ocrResult.Text);

OCR Troubles

When UWP (=Universal Windows Platform) Apps were introduced, I was interested in what new APIs came with them. Soon the OcrEngine (https://docs.microsoft.com/en-us/uwp/api/windows.media.ocr.ocrengine) peaked my interest, because it promised a simple and quick way to retrieve text from images.

A simple OcrEngine was something that I was looking for as the alternatives are big and cumbersome to use (I am looking at you Tesseract), discontinued (MODI; was included with Office), in the cloud and/or expensive.

Back then the problem was that you needed to create a UWP Application to access the UWP APIs, but at the same time an UWP Application was completely sandboxed! You couldn’t even use any cross process communication (with the exception of using the cloud and a very basic file based approach).

That meant I couldn’t use the OcrEngine in a WindowsService or WebService or even over a commandline!

So with that being the case, I put together a quick solution using Tesseract, but I never got around to tuning it and it never performed well.

UwpDesktop

Time went by and then the great Lucian Wischik (https://blogs.msdn.microsoft.com/lucian) published the library uwp-desktop (https://github.com/ljw1004/uwp-desktop) as a nuget package called UwpDesktop.

This package made UWP APIs available to Applications based on the normal .NET Framework. When I read the announcement, I was instantly reminded of my previous failure to make use of the OcrEngine and finally today I took it out for a spin and it worked great!

Example Code

The following code reads in the supplied file and prints out the detected text:

var engine = OcrEngine.TryCreateFromLanguage(new Windows.Globalization.Language("en-US"));
string filePath = TestData.GetFilePath("testimage.png");
var file = await Windows.Storage.StorageFile.GetFileFromPathAsync(filePath);
var stream = await file.OpenAsync(Windows.Storage.FileAccessMode.Read);
var decoder = await Windows.Graphics.Imaging.BitmapDecoder.CreateAsync(stream);
var softwareBitmap = await decoder.GetSoftwareBitmapAsync();
var ocrResult = await engine.RecognizeAsync(softwareBitmap);
Console.WriteLine(ocrResult.Text);

Example Application

I’ve put together a very simple example app and pushed it to github (https://github.com/8/ConsoleUwpOcr) that makes use of the OcrEngine.

Example Output:

ocr.exe ..\..\..\ConsoleUwpOcr.Test\TestData\testimage.png
Welcome to Thunderbird Donate to Thunderbird Thunderbird IS the leading open source, cross- platform email and calendaring client, free for business and personal use. We want it to stay secure and become even better. If you like Thunderbird, please consider a donation! By donating, you Will help us to continue delivering an ad-free top-notch email client. Make a donation » Other ways to contribute to Thunderbird Now IS a great time for you to get involved: writing code, testing, support, localization and more. Join a global community! Share your skills and Pick up a few new ones along the way. Volunteer as much as you like. Or as little. It's totally up to you. Learn more » Why we need donations You might already know that Thunderbird improvements are no longer paid for by Mozilla. Fortunately there IS an active community keeping it running and developing it further. But to survive long term, the project needs funding. Thunderbird IS currently transitioning to an independent organization. Being independent, we can shape our own fate, but there IS significant infrastructure that must be majntajned to deliver the application to our tens of millions of users. For Thunderbird to survive and continue to evolve, we need your support and ask for your donation today. All the money donated Will go directly to funding Thunderbird development and infrastructure.

References

• OcrEngine (https://docs.microsoft.com/en-us/uwp/api/windows.media.ocr.ocrengine)
• Lucian Wischik (https://blogs.msdn.microsoft.com/lucian)
• UwpDesktop (https://github.com/ljw1004/uwp-desktop)
• OCR Example App (https://github.com/8/ConsoleUwpOcr)

A bug appears

A few days ago a friend asked me to help him figure out a bug that was reported. Thanks to the error report he was already able to trace the bug to a specific code fragment, but he was wondering why it failed - the code looked perfectly fine.

The cause

The code was interfacing with an oracle database and was calling a stored procedure. As a stored procedures can’t return a value by itself, the usual way to retrieve data is to use an output parameter. The C# code sets the parameter up by declaring it’s type, size and direction and the stored procedure is then able to access and update it. After the control returns back to the caller, you can access the parameter and use it’s filled value.

The error message was: ORA-06502: PL/SQL: numeric or value error: character string buffer too small exception

In this case the calling code was:

cmd.Parameters.Add("param1", OracleDbType.Varchar2, 512, Direction.Output);

When we took a look at signature of the offending method, we quickly spotted the bug:

public OracleParameter Add(string name, OracleDbType dbType, object val, ParameterDirection dir)

The problem was with the third parameter - the caller thought he was initializing the size of the output parameter, but instead he was supplying the initial value - which isn’t used for an output parameter anyway.

Fixing the code was easy by setting the size of the parameter explicitedly and we could have called it a day, but I’ve seen a bug just like this before and so I wondered how this mistake could have happened and I took a second look.

The root cause

Digging deeper, I found the following overload:

public OracleParameter Add(string name, OracleDbType dbType, int size)

And then it dawned on me! Can you see what happened?

When the caller of the method started typing, he saw the overload where the third parameter is the size of type int and the correct overload is chosen.

It looked something like this:

cmd.Parameters.Add("param1", OracleDbType.Varchar2, 512

But he didn’t stop here, he continued on, because he wanted to supply the ParameterDirection as well and the moment he did so, the other overload was chosen, where the third parameter is now the value and not the size!

The caller didn’t notice, as an int converts nicely to an object and the signatures match up.

Bad API design

The culprit in this scenario was bad design on oracles behalf when they added overloads that change the semantics of a parameter at a certain position.

If the types of the parameters would have differed sufficently it would only be a nuisance for the developer, as the compiler would have caught the error, but to make matters worse the types used were implicetly convertable from one to the other and therefore the compiler was of no help.

General guidelines for Member Overloading are documented nicely on the msdn and while you may freely ignore those design principles in your own applications (even if they make sense) in a professional public facing API you really should follow them, your customers will thank you for it.

References

Member Overloading on msdn (https://msdn.microsoft.com/en-us/library/ms229029(v=vs.110).aspx)

Overview

When you are using Mvvm you need a way to bind your view to the ViewModel.

While this is always done by binding the DataContext Property of a View to an instance of the specific ViewModel class, there are generally two different scenarios:

1. The ViewModel can be retrieved using the current DataContext.
2. The ViewModel needs to be retrieved from a global source or created on demand.

While the first first scenario is straightforward, the second is a little more a tricky and in this article I’ll show a simple pattern that you can use in your applications to simplify the binding process.

Different Scenarios

If the ViewModel can be retrieved using the current DataContext of a View, then I’ll call this scenario “Hierarchical ViewModels” and if this is not the case, then I’ll call them “Independent ViewModels”.

Hierarchical ViewModels

Background

In the first scenario, you are already within a view that is bound to a ViewModel and you want to bind a child view to a ViewModel that is different than the one in the current DataContext, but one that it holds a reference to.

Often the parent is created by a DependencyInjection container that supplies the child ViewModel to the parent inside it’s constructor on instantiation.

The parent than exposes the child ViewModel via a public property.

In the parent view, you can just create a binding that sets the DataContext of the child to this property.

Example

In the following section, I’ve included an example for a hierarchical setup.

Consider this example consisting of two Views:

• MasterView
• DetailView

that are bound to these two ViewModels:

• MasterViewModel
• DetailViewModel

MasterViewModel.cs

public class MasterViewModel
{
  public DetailViewModel Detail { get; set; }
  ...
}

DetailViewModel.cs

public class DetailViewModel
{
  ...
}

Then the following code can be used to bind the DetailView inside of the MasterView to the DetailViewModel contained inside the MasterViewModels Detail property:

MasterView.xaml

<v:DetailView DataContext={Binding Detail} />

Note: As the Source of the Binding defaults to the current DataContext, the Source property of the Binding does not need to be set explicitly and the the Binding needs only to contain the path to the ViewModel.

Independent ViewModels

Background

Then there there are Independent ViewModels, which don’t know of each other. This is the case for all base ViewModels, who are the “entry points” for all hierarchical setups.

Some examples:

• The first ViewModel that is bound to a view, eg. MainViewModel, when there is no existing DataContext
• Independent ViewModels for cross cutting concerns like
  • navigation eg. the Menu
  • information eg. the Statusbar and Notifications

In those cases, there current DataContext of the View does not contain a property that we can use, so we need to access some kind of central Locator or Factory, which is probably backed by an IoC Container that knows how to retrieve or create the requested ViewModel.

Example

Now setting up independent ViewModels or the first ViewModel when no DataContext is yet available, requires a little more work.

I’ve come with this simple pattern to make the ViewModels available for binding inside each view.

1. Create a Locator that exposes the ViewModels that your views require via properties.

2. Add an instance of the locator as a static resource to your Application. This should be done on Application Startup, for example by:

   • Creating an Instance declaritively in your app.xaml file
   • Subscribing to your applications Startup event and setting it using the Resources property.

3. Bind the View to the ViewModel by using the Locator as a Source using the StaticResource Binding.

Locator

As the first step, we need to create a locator that exposes the ViewModels that will be requested by from within a view.

The Locator exposes the ViewModels as properties, that makes binding against them easy using the normal binding syntax.

You can take the manual approach, where you implement each new ViewModel as a new property of the Locator yourself or you can use a dynamic approach where the ViewModels are lookup by your dependency injection container.

An example for the manual approach would look like something like this:

Locator.cs (manual)

public class Locator
{
    public MainViewModel { get { return new MainViewModel(); }}
}

As the manual approach gets tedious really fast, I’ve opted for the dynamic approach.

My implementation is based on DynamicObject that allows me to forward the property accessor to a dependency resolver for fulfillment.

Locator.cs (dynamic)

/// <summary>Locator that forwards property access to the Dependency Resolver</summary>
public class Locator : DynamicObject
{
  /// <summary>Gets or sets the resolver that is used to map a property access to an instance</summary>
  public Func<string, object> Resolver { get; private set; }

  public Locator(Func<string, object> resolver)
  {
    this.Resolver = resolver;
  }

  public override bool TryGetMember(GetMemberBinder binder, out object result)
  {
    bool successful;

    string property = binder.Name;

    var resolver = this.Resolver;

    if (resolver != null)
    {
      try
      {
        result = resolver(property);
        successful = true;
      }
      catch { result = null; successful = false; }
    }
    else
    {
      result = null;
      successful = false;
    }

    return successful;
  }
}

The locator is supplied with a Func in it’s constructor that resolves the request for the ViewModel based on the requested property name.

For example, let’s say you are using Autofac as a dependency resolver and you’ve configured Autofac to resolve your ViewModels by looking them up from your ViewModel namespace using a simple convention like this:

ContainerBuilder builder = new Autofac.ContainerBuilder();
builder.RegisterAssemblyTypes(Assembly.GetExecutingAssembly())
  .InNamespace("WpfMvvmExample.ViewModel");
var container = builder.Build();

Now you can create an instance of the Locator like this:

var locator = new Locator(property => container.Resolve(Type.GetType($"WpfMvvmExample.ViewModel.{property}")));

Note: As the binding path used inside a view is just a string and evaluated at runtime and not strongly typed, using a dynamic object fits nicely, as we don’t lose any type information.

Add the initialized Locator

Now it’s time to add the Locator to someplace where your views can access it. The Application_Startup method inside the App class is a good place for that.

App.xaml.cs

ContainerBuilder builder = new Autofac.ContainerBuilder();
builder.RegisterAssemblyTypes(Assembly.GetExecutingAssembly())
  .InNamespace("WpfMvvmExample.ViewModel")
  .SingleInstance();
        
var container = builder.Build();

this.Resources["Locator"] = new Locator(property => container.Resolve(Type.GetType($"WpfMvvmExample.ViewModel.{property}")));

Bind the View to the ViewModel

Finally we can use the Locator inside of a view to use it to bind to a ViewModel. We reference the Locator as the bindings Source property and point the path property to the required ViewModel type.

MainWindow.xaml

<v:MainView DataContext="{Binding MainViewModel, Source={StaticResource Locator}}" />

Example Code on github

I’ve uploaded an small example application to github that contains the Locator and the setup in case it is useful for anybody else.

If you have any comments please feel free to drop me a line in the comments below, thanks!

Take care,
-Martin

References

• Mvvm (en.wikipedia.org/wiki/Model–view–viewmodel)
• Example Application Github Source Code (https:// github.com/8/WpfMvvmExample)
• Autofac (https://github.com/autofac/Autofac)

Background

After SkiaSharp was announced by Miguel de Icaza on his blog, I downloaded the nuget and took it for a spin and used it for some image manipulation.

While the sample code got me started, it was written for System.Drawing/GDI+ and when I later wanted to use it in a Wpf app, I didn’t find any sample code for that. So I wrote some code and this blog post, in case someone else might find that useful.

Drawing a Bitmap in Wpf

ImageSource and WriteableBitmap

Basically, when you’re using Wpf you most often want to use an ImageSource, for example to display it within an Image control. When creating an ImageSource yourself, the WriteableBitmap comes in handy. It is not only a subclass of ImageSource, it’s also double buffered, which allows a smooth udpate process.

Sourcecode

I’ve written the following code to do that:

public WriteableBitmap CreateImage(int width, int height)
{
  return new WriteableBitmap(width, height, 96, 96, PixelFormats.Bgra32, BitmapPalettes.Halftone256Transparent);
}
public void UpdateImage(WriteableBitmap writeableBitmap)
{
  int width  = (int)writeableBitmap.Width,
      height = (int)writeableBitmap.Height;
  writeableBitmap.Lock();
  using (var surface = SKSurface.Create(
    width: width,
    height: height,
    colorType: SKColorType.Bgra_8888,
    alphaType: SKAlphaType.Premul,
    pixels: writeableBitmap.BackBuffer,
    rowBytes: width * 4))
  {
    SKCanvas canvas = surface.Canvas;
    canvas.Clear(new SKColor(130, 130, 130));
    canvas.DrawText("SkiaSharp on Wpf!", 50, 200, new SKPaint() { Color = new SKColor(0, 0, 0), TextSize = 100 });
  }
  writeableBitmap.AddDirtyRect(new Int32Rect(0, 0, width, height));
  writeableBitmap.Unlock();
}

Basically, what we want to do is:

• Create a WriteableBitmap of the appropriate size
• Update the WriteableBitmap with Skia
  1. Lock the Backing Buffer
  2. Use Skia with the matching pixelformat to draw into the backing buffer
  3. Mark the Bitmap as dirty
  4. Unlock the Bitmaps Backing Buffer again

Note: Don’t forget to mark the updated region of the bitmap as dirty, else nothing is going to happen!

Example Wpf App

Now that I was able to render an Wpf image with Skia and the WriteableBitmap class supports double buffering, I wanted to create a quick app that updates the Image once per frame.

For that, I’ve subscribed the CompositionTarget.Rendering event and updated the render method to draw the number of elapsed frames. You can see the output on the screenshot below:

Screenshot

Sourcecode on Github

If you’re interested in the example app, I’ve uploaded the source of the SkiaSharp Wpf Example Application to github at https://github.com/8/SkiaSharp-Wpf-Example

If you find any of that useful or I am missing something, please feel free to drop me a comment below, thanks!

Take care,
Martin

References

• System.Drawing/GDI+ (https://msdn.microsoft.com/en-us/library/system.drawing%28v=vs.110%29.aspx)
• ImageSource on msdn (https://msdn.microsoft.com/en-us/library/system.windows.media.imagesource%28v=vs.110%29.aspx)
• WriteableBitmap on msdn (https://msdn.microsoft.com/en-us/library/system.windows.media.imaging.writeablebitmap%28v=vs.110%29.aspx)
• SkiaSharp (https://developer.xamarin.com/guides/cross-platform/drawing/)
• Cross-Platform 2D Graphics with SkiaSharp (https://blog.xamarin.com/cross-platform-2d-graphics-with-skiasharp/)
• SkiaSharp Wpf Example App on github (https://github.com/8/SkiaSharp-Wpf-Example)

Background

I’ve been using and enjoying knockout.js for some time now.

It’s a great library that allows you to use MVVM in web applications and keeps you from writing spaghetti code to manipulate the DOM without requiring a switch to a monolithic framework and the associated downsides like lock-in and too many abstractions from plain html.

Using knockoutjs, you are still free to use DOM manipulation yourself if and when you need it. The great thing is, it’s also easily extendable.

Extending Knockout

Why is being extendable a big plus and why would you want to extend knockout? Is something essential missing from knockout?

Nope, I don’t think so.

Instead of growing to a monolithic framework, it just solves a particular problem, namely factoring out the UI glue code into reusable bindings. It comes with almost all bindings you could think of by default, but it doesn’t try to be everything for everyone - and thats where custom bindings come in.

Using custom binding handlers, it offers you the chance to stick to DRY and to use declarations instead of repeating javascript snippets over and over again.

That often comes in handy, when you need to reuse some javascript code in multiple places and the code is tied to an element defined in html.

In the next few paragraphs, I am showing some small, exemplary binding handlers that have proven useful to me, nothing fancy.

Example BindingHandlers

I’ve been using some small knockout bindings that uses jquery’s fadeIn() / fadeOut() methods and slideDown() / slideUp() to achieve simple animations on an element.

FadeVisible

The binding is defined in the following few lines:

ko.bindingHandlers.fadeVisible = {
  init: function (element, valueAccessor) {
    var value = valueAccessor();
    $(element).toggle(ko.unwrap(value));
  },
  update: function (element, valueAccessor) {
    var value = valueAccessor();
    ko.unwrap(value) ? $(element).fadeIn() : $(element).fadeOut();
  }
};

SlideDownVisible

The definition for the slideDown binding looks almost identical:

ko.bindingHandlers.slideDownVisible = {
  init: function (element, valueAccessor) {
    var value = valueAccessor();
    $(element).toggle(ko.unwrap(value));
  },
  update: function (element, valueAccessor) {
    var value = valueAccessor();
    ko.unwrap(value) ? $(element).slideDown() : $(element).slideUp();
  }
};

In turn, they both are very similar to the example knockout binding in knockouts custom-binding documentation which also provides a binding that uses slideDown() and slideUp().

Usage

As for usage, you’d replace the default ‘visible’ binding with ‘fadeVisible’ or ‘slideDownVisible’ respectively.

    <div data-bind="fadeVisible: isVisible">
    ...

Nuget package

I’ve used the slideDownVisible binding already in a couple of projects and I’ve finally gotten sick of copy/pasting them, so I’ve packaged them as nuget packages named ‘knockout-fadeVisible’ and ‘knockout-slideDownVisible’ and uploaded them to nuget.org so that I can add it faster the next time I might need it. The (very short) source is on github as well.

Bootstrap Modal

Another example of transforming javascript glue code to a declarative knockout binding would be the following modalVisible binding:

ko.bindingHandlers.modalVisible = {
  init: function (element, valueAccessor) {
    var value = valueAccessor();
    /* init the modal */
    $(element).modal();
    /* subscribe to the 'hidden' event and update the observable, if the modal gets hidden */
    $(element).on('hidden.bs.modal', function (e) {
      if (ko.isObservable(value)) { value(false); }
    });
  },
  update: function (element, valueAccessor) {
    var value = valueAccessor();
    ko.unwrap(value) ? $(element).modal('show') : $(element).modal('hide');
  }
}

It wraps bootstrap javascript code in a tidy, nice to use knockout binding after using the binding like:

<div class="modal fade" data-bind="modalVisible: isVisible"...

This takes care of initializing the modal and allows controlling it’s visibility using an observable. It handles hiding and showing of the modal and therefore removes the need to manipulate the DOM from my ViewModels javascript code.

Conclusion

Knockoutjs is nice and flexible library that is not only easy to get started with, but also easy to extend.

Creating custom binding handlers may save you from writing repetitive and error prone code and allows you to stick view specific code declaratively right on the target html element, which makes reasoning about your view easier.

Decoupling jQuery and other DOM manipulation code from your normal code also makes that code simpler to test.

Take care,
Martin

References

• Knockout.js (http://knockoutjs.com)
• Creating a custom binding for Knockout (http://knockoutjs.com/documentation/custom-bindings.html)
• Knockout’s ‘visible’ binding (http://knockoutjs.com/documentation/visible-binding.html)
• jQuery slideDown() (https://api.jquery.com/slideDown/)
• jQuery slideUp() (https://api.jquery.com/slideUp/)
• jQuery fadeIn() (https://api.jquery.com/fadeIn/)
• jQuery fadeOut() (https://api.jquery.com/fadeOut/)
• fadeVisible binding (https://github.com/8/ko-bindings/blob/master/ko-bindings/Scripts/knockout-fadeVisible.js)
• slideDownVisible knockout binding (https://github.com/8/ko-bindings/blob/master/ko-bindings/Scripts/knockout-slideDownVisible.js)
• Bootstrap Modal Javascript (http://getbootstrap.com/javascript/#modals)

TL;DR

1. Start chrome in remote debug mode: chrome.exe --remote-debugging-port=9222
2. Attach Visual Studio: “Debug” -> “Attach to Process…” -> select the chrome instance
3. Done.

Justifying a use case

So you are still reading? Fine, than I can do some rambling. I was developing a JavaScript WebApp with some complicated client code - it’s built like a game loop using requestAnimationFrame and canvas to render multiple videos onscreen and playing synced audio - like most software it worked, but sometimes it would glitch and I was trying to figure out what caused it.

Now what I wanted was to debug the code, preferably from the comfort of my IDE, which happens to be Visual Studio, but while Visual Studio supports debugging JavaScript via Internet Explorer out of the box, it does not support any other browser.

More often than not, that’s not a big problem, you just fire up IE, wonder why you never changed the startup page to something reasonable and use it just once for debugging.

But not in this case as I was making use of AudioContext and other shiny new WebApi stuff that are available in Chrome and Firefox already but - you guessed it - not in IE.

You could of course do what everyone else would do and use the built-in chrome developer tools, which are great imho, but that would incur the cost of mental task switchs for using a different IDE that does not share the same syntax highlighting, hotkeys and general workflow you have come to be so productive with. So for the sake of this article I count switching to a different tooling as giving up.

Wondering if it’s possible…?

So I started wondering, if the big V is able to debug javascript running in chrome.

The first hint, that the consensus is that it won’t work, was for me that out of the box selecting chrome as your browser in visual studio and starting your debug session does not work, while it does for IE.

Quick Robin, to the googlemobile!

Almost every IT-SuperHero

But the googlemobile failed hard this time, top search results was talking about a Native Client and C++ code and a thread from last year said that it’s not possible and Visual Studio’s integrated Extension search turned up nothing.

But on the other hand, I’ve already tried Visual Studio’s Node.js Tools and I remember vividly being amazed that debugging just worked. Okay, so because Node.js and chrome both use V8 as their JavaScript engine, Visual Studio must already be able to debug it.

Fiddlin’ around

So I ignored that Visual Studio does not start debugging if you are using chrome and I simply tried to attach it to chrome using DEBUG -> Attach to Process… and while that did not work, I noticed something interesting:

In the “Code Type” selection I found a listing for Webkit!

Now I knew that Visual Studio could do it and even expects me to use Debug and Attach and so it’s probably chrome that doesn’t cooperate, which makes sense as a sane default.

Solution

So when I returned to google again I knew what to look for and a search for chrome remote debugging brought me to this page where my the missing part for my answer was waiting:

1. Start chrome in remote debug mode: chrome.exe --remote-debugging-port=9222
2. Attach Visual Studio: “Debug” -> “Attach to Process…” -> select the chrome instance
3. Done.

References

• Visual Studio Node.js Tools (https://www.visualstudio.com/en-us/features/node-js-vs.aspx)
• Chrome Debugger Protocol (https://developer.chrome.com/devtools/docs/debugger-protocol)

Background

I’ve written about how to do a mass file import into M-Files here and here before, but recently I was contacted by a client who had quite the opposite problem - he wanted to export a lot of files out of M-Files.

Getting Info

After a quick skype call to get to know the client and the details of the project, the following facts were available:

• The data resides in a single M-Files Vault
• It’s backed by a whopping 230+ GB SQL Server Database
• Other ways to export, direct access to the SQL Server and manual exporting had failed
• An export of all document files (.docx, .pdf, …) is needed
• All properties of the files should be exported to a CSV file
• Time was of the essence (when is it not?)

What didn’t work

It seems that the client tried to export the data directly from the SQL Server, but I heard that this approach failed as they couldn’t make out what goes where. From a software engineering perspective, this is fair, as the data storage is an implementation detail that can be changed anytime (for example by using another database backend).

Next they tried to export the files manually. That’s not only slow, but also an awfully error prone process and when you’re interested in the metadata as well, then you really really shouldn’t go down this route, even with a few documents, but in this case we had a few hundred thousand.

So what’s the alternative?

You probably know what’s the right approach: a small custom application that uses the M-Files API to access all documents programmatically.

Solution

Armed with this knowledge we can formulate the characteristics of the solution:

• Create the files
• Create metadata
• Reliable
• Fast
• Inexpensive

Considering the fact, that the tool needed to be done quickly and keep the development cost low, I decided on writing a Commandline Application. Another reason was, that it did not need to look fancy and would be operated by skilled IT personal who preferred a simple commandline interface anyway. I would have been happy to create a Wpf Application like Chronographer but that would have been overkill.

Exporting the files

As I had prior experience with the M-Files API, it didn’t take me long get the file export running. In the screenshot below you see the results if run against the Sample Vault.

Exporting the Data to CSV

A little more interesting but still straight forward was the csv export, as you need to know all properties to create the csv header and put them in the right column. To do this, I enumerate all classes in the Vault and collect their properties and then they are written in the header row.

A feature of M-Files is, that all M-Files documents can contain 0 or more files, which meant for me that a found document could result in zero exported files (if it didn’t hold any) or more than 1 file, in which case the csv export also needed to repeat the properties accordingly.

I settled on creating 4 fixed csv columns followed by the properties of all classes. The 4 columns are:

1. FilePath
   Allows mapping to the exported files
2. FileId
   The id of the document in the M-Files Vault
3. SourceFileId
   The id of the binary file document (.docx, .pdf, …)
4. ClassName
   The name of the class that the exported file belongs to

Below you’ll find a screenshot of the result when run against the Sample Vault.

Enumerating the files

An interesting problem was enumerating all the files to export. I settled on creating a search for the documents that skips deleted objects. As the maximum number of search results is capped at 100.000 items, you are not able to fetch all documents with a single search. I solved that by adding an additional search condition, namely searching for ids with a specified segment, where segment 0 for example means items 0-9999 and segment 1 returns 10000-19999. By repeatedly searching for files in this way and incrementing the segment, I was able to traverse the whole vault.

Complications

As always, nothing works perfectly on the first try and while the CSV Export completed successfully, when we were exporting the files for a few hours M-Files threw an Exception with the error message “An SQL update statement yielded a probable lock conflict. Lock request time out period exceeded.” which seemed like a M-Files glitch to me.

Anyway, as we needed to try again we were loath to start the export from the beginning, so I added another commandline argument that allowed us to specify a starting segment so that we could continue our export instead of starting it from the beginning.

So that the final commandline interface looked like this:

Additional parameters like the Vault Name and the Credentials are stored in a config file in the same folder and read by the application at startup.

On the second run, we didn’t encounter any errors and we had exported over 200K files and produced a nice 120MB CSV file. In the end, we had a nice, repeatable process that was able to save the client a lot of time, money and headaches.

References

• Blog post: M-Files mass file import Part I (http://lostindetails.com/blog/post/“M-Files-mass-file-import-Part-I)
• Blog post: M-Files mass file import Part II (http://lostindetails.com/blog/post/“M-Files-mass-file-import-Part-II)
• Chronographer, a Wpf Application I wrote (http://lostindetails.com/chronographer)

Background

M-Files always stores it’s data in a Sql Database. As of this writing two Database vendors are supported:

• Firebird (default)

  Firebird is an open source Sql Server, you can find out more about it on http://firebirdsql.org. As it is free, it’s the default option when you install M-Files.

• MS-SQL Server

  Microsoft’s Sql Server is the second option. Because it’s rather expensive, it’s not the default option and the M-Files customer has to buy and install it themselves.

Database Engines in the wild

Both Firebird and MS-Sql should be able to handle M-Files Vaults of considerable size, but in practice Firebird gets used more often in smaller businesses and MS-Sql Server gets used more often in bigger firms.

While the price tag certainly does matter, more often than not it depends on whetever there is a preexisting investment into MS-Sql Server or not. If the company already has a MS-SQL Server on premise, it’s a no brainer to tack on an additional Database. Even if they don’t have a DBA who already has experience with administrating the server, a lot of companies already depend on the MS-Sql Server if they run on the Microsoft Stack at all for different reasons, maybe they already have a CMS or Website that depends on it.

MS Sql Server Express

Although a free version of Microsofts Sql Server called “Sql Server Express” is available for download (it even comes with reporting Services and an ok GUI for Administration if you pick up the SQL Server Express with Advanced Services) it’s often a poor choice for M-Files, because of the 10 GB limit on database.

Don’t get me wrong - 10 GB is not a small amount of data, but in this case remember that M-Files does not only store your customers and invoices as numbers and strings, but it also stores all binary files in the database as well - that means all word documents, powerpoints and pdfs with their high resolution cat pictures. If you throw in revisions of the same file and multiply that by a couple of users you get to a lot of data very quickly.

If you think that that’s not an issue in your case, you should be able to use the express version as mentioned in this thread on the M-Files Forum.

Backing up and Restoring a M-Files Vault

Backing up and Restoring an M-Files Vault depends on your backend Database.

If you’re using the default firebird sql server, than the backup is done using the M-Files Admin tool.

But if you are using the Microft Sql Server as the backend, then you’ll need to a tool like the Sql Server Management Studio to create a backup of your vault and restore it back again, which is rather simple to do for anyone that has used the tool once before.

A problem when restoring a MS-SQL based Vault

What prompted this quick writeup is that yesterday a client had a problem restoring an M-Files Vault on the SQL Server.

Restoring the Database using the Management Tools worked fine, but the target system had a newer version of M-Files running and was trying an upgrade but failed with the following error message:

Upgrading the document vault ‘Vaultname’ failed. ALTER ASSEMBLY for assembly ‘MFMSSQLCLRObjs’ failed because assembly ‘MFMSSQLCLRObjs’ is not authorized for PERMISSION_SET = UNSAFE. The assembly is authorized when either of the following is true: the database owner (DBO) has UNSAFE ASSEMBLY permission and the database has the TRUSTWORTHY database property on; or the assembly is signed with a certificate or an assymmetric key that has a corresponding login with UNSAFE ASSEMBLY permission. (ERROR: 10327, SQLSTATE: 42000)

After checking that the dbo had the correct permissions, I found that the problem was that the restored database did not have the TRUSTWORTHY property set.

I fixed that by excecuting the following command in the Sql Management Studio as explained in this Microsoft Article:

ALTER DATABASE Vaultname SET TRUSTWORTHY ON;

After that I was able to attach the document vault without problems.

References

• Firebird Sql (http://firebirdsql.org/)
• MS-SQL Server (http://www.microsoft.com/sqlserver/)
• SQL Server Express Edition (https://www.microsoft.com/en-us/server-cloud/products/sql-server-editions/sql-server-express.aspx)
• Thread about SQL Server Express Support in M-Files 9.0 RC (https://community.m-files.com/index.php?topic=280.0)
• TRUSTWORTHY Database Property (https://msdn.microsoft.com/en-us/library/ms187861.aspx)

If razor intellisense stops working suddenly, try deleting your C:\Users\username\AppData\Local\Microsoft\VisualStudio\14.0\ComponentModelCache folder.

The Problem

When I was opening Visual Studio the other day, I was greeted with the following error message:

Closing and reopening Visual Studio didn’t fix the error - it occurred every time I opened a razor view (.cshtml) page. While Visual Studio itself did not crash, highlightning and intellisense was broken inside the view. E.g. razor comments “

@* this is a comment! *@

were not highlighted correctly (just as in this post) and my favourite spellchecker (intellisense) had stopped working as well.

Looking at the error

I did as I was told and took a look at the ActivityLog as the error message suggested and opened: C:\Users\username\AppData\Roaming\Microsoft\VisualStudio\14.0\ActivityLog.xml Which in turn revealed the following stack trace:

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.ArgumentException: Item has already been added. Key in dictionary: 'RazorSupportedRuntimeVersion' Key being added: 'RazorSupportedRuntimeVersion' at System.Collections.Hashtable.Insert(Object key, Object nvalue, Boolean add) at System.Collections.Hashtable.Add(Object key, Object value) at System.Collections.Specialized.HybridDictionary.Add(Object key, Object value) at Microsoft.VisualStudio.Utilities.PropertyCollection.AddProperty(Object key, Object property) at Microsoft.VisualStudio.Html.Package.Razor.RazorVersionDetector.Microsoft.Html.Editor.ContainedLanguage.Razor.Def.IRazorVersionDetector.GetVersion(ITextBuffer textBuffer) at Microsoft.Html.Editor.ContainedLanguage.Razor.RazorUtility.TryGetRazorVersion(ITextBuffer textBuffer, Version& razorVersion) at Microsoft.Html.Editor.ContainedLanguage.Razor.RazorErrorTagger..ctor(ITextBuffer textBuffer) --- End of inner exception stack trace --- at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, StackCrawlMark& stackMark) at System.Activator.CreateInstance(Type type, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes) at System.Activator.CreateInstance(Type type, Object[] args) at Microsoft.Html.Editor.ContainedLanguage.Common.ContainedCodeErrorTaggerProvider`1.CreateTagger[T](ITextBuffer textBuffer) at Microsoft.VisualStudio.Text.Tagging.Implementation.TagAggregator`1.GatherTaggers(ITextBuffer textBuffer)

Looking at the stacktrace confirmed my suspicion that on a big picture view something razor specific was messed up (I am looking at you ‘RazorSupportedRuntimeVersion’), I also got the info that an exception was being thrown, because a duplicate hashtable entry was being inserted.

Google to the rescue

The best part was of course, that an error message gave me something to google for and so I did and finally came to a Visual Studio Feedback Item with the same error message.

The issue is of course marked as closed, but that’s neither here nor there.

A first workaround appears

Under workarounds I found out, that resetting your user data seems to fix the issue. To do that you execute devenv /resetuserdata and sure enough it did! Intellisense and Highlightning started to work again in the razor view!

But the workaround also has some drawbacks, it well, resets your user data (who would have thought?!), which means it removes all your extensions as well.

Well, whatever. But then it happened to me again a few days later. :|

Improving the workaround

Okay, so I thought, I could probably do better and not reset everything but only the part that causes the problem. So I took another look at the StackTrace and judging from the StackTrace RazorVersionDetector.GetVersion() seems like the culprit. According to the StackTrace RazorVersionDetector sits in Microsoft.VisualStudio.Html.Package.Razor and luckily I found a dll with the same name under: C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\Microsoft\Web Tools\Editors\Microsoft.VisualStudio.Html.Package.dll.

So I fired up trusty ILSpy and looked at the strange looking interesting interpretation of the C# code of the RazorVersionDetector that ILSpy showed me. There I noticed that it’s private methods were called GetCachedVersion() and SetCachedVersion() which was why it worked at first, but broke later on and why resetting the user data worked - it cleared the cache.

Armed with the knowledge I looked for cache folder and luckily I found it: C:\Users\username\AppData\Local\Microsoft\VisualStudio\14.0\ComponentModelCache

Removing or renaming this folder forces Visual Studio to recreate it and Razor intellisense starts working again and you get to keep your extensions and stuff.

References

• Visual Studio Connect Item (https://connect.microsoft.com/VisualStudio/feedback/details/1572978/no-intellisense-in-razor-views-argumentexception-razorsupportedruntimeversion-already-added-to-dictionary)
• ILSpy (http://ilspy.net)

Abstract

In this article I’ll encourage you to keep an open mind about your options when building a software solution.

Motivation

Why write or read about how to choose between different solutions? Shouldn’t it be obvious? I think it’s well worth our time to dwell on that, especially considering two reasons:

1. Choosing the wrong solution is one of the most expensive mistakes to make. E.g. adding an unplanned feature to an application may seem expensive, but it pales in comparison with the realization that your mobile app should have been website or the other way round.
2. Often a decision about a platform or solution is not made deliberately, because we are simply not aware of all our options. E.g. consider if Kahneman’s WYSIATI (What You See Is All There Is) theory comes into play here or the “If all you have is a hammer, everything looks like a nail” quote rings true.

Why don’t we evaluate our options more carefully?

There are a couple of obstacles that keep us from taking a closer look.

• Missing Expertise It often requires in-depth knowledge and experience of different technologies, which is hard to come by. E.g. if you’re only skilled in developing Web Pages for customers you won’t necessarily see that a Windows Service or a commandline tool on a Parallela Board fits the requirements better.
• Emotional Bias We’re often biased towards some technology not based upon it’s merits and flaws for our specific use case, but rather on our feelings and emotions based on a past project. Not only was it a different project and the technology should be reevaluated in the context of the new one, but also our emotions were influenced by other factors like a too short deadline, mismanagement, unrealistic expectations or difficulties with the involved parties.
• Lots of Options Evaluating a lot of options is difficult and straining, consider for example the book The paradox of choice or the TED Talk given by Barry Schwartz.

Removing the Obstacles

But how can you remove those obstacles?

Missing Expertise

Missing Expertise can be battled in the following ways:

1. Learn about it The most obvious, although hardest solution is to learn about the qualities of certain technologies. This does not mean that one needs to become an expert in every new technology that pops up, but knowing if it’s a good match for a certain use case is important.
2. Get Help If you’re a consultant or developer and the technology that is most appropriate or required for the task does not match your expertise, you should acknowledge the fact and inform your client or boss. If you have a colleague or acquaintance that specializes in that technology or field you should refer to them.

Emotional Bias

1. Stressfree Environment A good way to combat emotional bias is to get to know a technology in a stressfree environment. This may mean, that you take a look at it in your spare time, without the clock ticking.
2. Tiny steps Start step by step and don’t try get everything done at once. Sometimes you’ll feel tempted to try something big at once that you are able to do in another language or with another framework that you’ve already mastered - but don’t give in. Small incremental successes keep your motiviation going for a longer time.
3. Ask a friend If you really dislike a certain technology, you should talk with a friend or someone you respect that is fond of it. As he or she is your friend, you are unlikely to completely dismiss them and get a glance why someone would want to use that and advantages it has.

Lots of Options

And finally, to get a quick overview I’ve compiled the following list that can be used to get the juices flowing on what parts could make up the ideal solution. My hope is that by spelling them out, they’ll be on the radar the next time when a problem needs solving. Many solutions do not fall strictly within one category and more often than not consist of more than one part.

Disclaimer

This list is neither exhaustive nor can it be, it’s just a reminder of what’s out there and should be seen as an inspiration to get you to explore your options.

• Web Solutions
  • Static Web Pages (consider online/offline help, manuals)
  • Content Management Systems (CMS)
    • Wordpress (php)
    • Umbraco (asp.net)
  • Mobile first
  • Server Side Frameworks
    • Asp.Net
      • NancyFx
      • Asp.Net MVC
      • Asp.Net WebApi
    • Ruby
      • Ruby on Rails
      • Sinatra
    • Php
      • phpcake
      • Symfony
    • Node.js
  • Forums
    • phpBB (php)
    • MVCForum (asp.net mvc)
  • SPA
  • Web API
  • Front End Frameworks
    • Bootstrap
    • jqueryUI
    • Skeleton
  • javascript frameworks
    • angular
    • aurelia
    • knockout
    • react
  • visualization
    • d3
    • Raphaël
    • processing
  • CSS Languages
    • Less
    • Sass
• Mobile Applications
  • Platform specific
    • Android
    • iOS
    • Windows Phone
    • Blackberry
  • Cross platform
    • Apache Cordova
    • Xamarin
    • RemObjects
    • Unity
• Desktop Applications
  • Wpf (windows only)
  • WinForms (cross platform using mono)
  • GTK and it’s wrappers like GTK#
  • DirextX and it’s wrappers like SlimDX or SharpDX
  • OpenGL and it’s wrappers like OpenTK
  • Unity
• Console Applications
• Windows Services
  • topshelf
• Linux Daemons
• Datastorage
  • FileSystem
  • SQL Databases
    • MS-Sql
    • MySql
    • Postgresql
    • Oracle
    • SQLite
  • No SQL Databases
    • RethinkDB
    • MongoDB
    • LMDB
• WebServers
  • Apache
  • nginx
• Media Processing
  • imagemagick for images
  • sox for audios
  • ffmpeg for videos
• Virtualization
  • Virtual Box
  • Hyper-V
  • Virtual Machine
  • Xen
• Operating Systems
  • Linux
  • Windows
  • BSD
• Hardware platforms
  • Raspberry Pi
  • Beagle Bone
  • Parallela
• Cloud platforms
  • Amazon
  • Azure
  • Digital Ocean
  • Linode
• Payment Providers
  • stripe
  • braintree
  • paypal

As always feel free to leave a comment, especially if you want to remind me of an important option that I completely missed when I typed up the list, as I most certainly did.

Background

In this article I explore a simple mechanism to check if an update is available by querying a server. My use case was that I’ve been writing a C# Application and I wanted to include a quick “Check for Updates” functionality that would inform the user if an update is available.

Additionally, I wanted a simple upgrade process, so that I can upload a new setup file and it’s picked up by the webserver automatically, so the requirements were quickly formulated:

Requirements

• The client application should be able to detect if a new version is available
• Publishing a new update should be painless

Formulating a plan

1. The Client App reads it’s version number
2. Client App sends the version number to the Webserver
3. Webserver compares the version number with the latest available version and returns the result
4. Client displays the result and asks the user to upgrade

Step 0 - Versioning the Application

As it turns out, there is a Step 0 associated with this process and that is versioning the application in the first place.

The simplest way to version your app is probably using the System.Reflect.AssemblyVersionAttribute as used in the auto-generated AssemblyInfo.cs file in your application.

I personally prefer to let the build process increment the build and revision parts of the version, so I’ve changed the use of the AssemblyVersion attribute in AssemblyInfo.cs to:

[assembly: AssemblyVersion("0.7.*")]

That way, even if I forget to manually increment the version number, the versions differ between each build which adds an additional safety net when diagnosing problems.

On top of that I removed the [assembly: AssemblyFileVersion("1.0.0.0")] declaration, as I prefer to keep them in sync.

After rebuilding the application I checked the generated version by right-clicking the file and selecting “Properties” -> “Details”, as you can see in the screenshot.

Step 1 - Reading the Application Version

Reading the version of the Application is a one-liner:

var version = Assembly.GetExecutingAssembly().GetName().Version;

Step 2 - Sending the Version to the Webserver

Sending the version to the webserver and retrieving the result is done using a simple WebRequest:

var baseUrl = "http://yourdomain.com/yourapp";
var url = Path.Combine(baseUrl, string.Format("isupdateavailable?v={0}", version));
var request = (HttpWebRequest)WebRequest.Create(url);
if (((HttpWebResponse)request.GetResponse()).StatusCode == HttpStatusCode.OK)
{
  /* update is available... */
}

The above code snippet assumes that you have a webserver that returns OK when an update is available. We’ll see about that in the next step. In a production ready case you might want to use GetResponseAsync and return an Task.

Step 3 - The Webserver compares versions

The webserver retrieves the version from the parameter, creates a Version object and compares the Version object with that of the current file.

In my case, I chose to encode the version in the setup file name like so: ‘setup_0.8.5742.26637.exe’ and read it back in using a regular expression. My reasons for not using the same process as on the client were the following:

• You can upload multiple setup files, which is handy if you want to be able to keep older versions
• As the generated setup file is native code (I am using NSIS to create the installer), not a managed assembly, reading the product version breaks down on a linux server (which I happen to use). The reason is, that the Product Version is encoded in the PE-Header of the file and while mono allows reading the version from a .NET assembly it does not work with a native file as it would on windows.

So the code to retrieve the current version looks something like this:

private static Regex SetupFileRegex = new Regex("Setup_(?.*?)\\.exe");
private Version GetVersionFromFileName(string fileName)
{
  var m = SetupFileRegex.Match(fileName);
  return m.Success ? new Version(m.Groups["Version"].Value) : null;
}
private Version GetLatestVersion()
{
  string[] files = Directory.GetFiles(GetSetupFolder(), "Setup_*.exe");
  return files.Select(f => new {
    File = f,
    Version = GetVersionFromFileName(Path.GetFileName(f))
  })
  .Where(f => f.Version != null)
  .OrderByDescending(f => f.Version)
  .First()
  .Version;
}