Binary Thoughts

The Magpie

Sat, 14 Apr 2018 00:00:00 +0000

Note: I split this post into two to make it more digestible. For background and context, see my post Shiny things.

TL; DR

Background and definition
The Magpie is misunderstood and provides a good model for learning
The V model of learning (contrasted to the model of T-shaped people) attempts to maximise intersection points in knowledge to enable fast learning
It’s vital to build a solid core to enable the transfer of skills from one domain to another
Haphazard learning achieves little - intentional learning is key
Practice learning, unlearning, and connecting ideas. Look for intersection points and know what you don’t know

Back in 2008 Jeff Atwood wrote a popular blog post speaking about the fast pace of change in technologies, and a specific class of developers attracted to new technologies, sometimes to their detriment. If you have not read it yet do so now . It contains some gems.

If you read the comments on that post it seems like some people took away the idea that sticking with what is familiar is the right thing to do. The Magpie developer got a bad rap and the analogy is incomplete. It ignores the benefits this pattern has for learning if used responsibly. In this post we will take a deeper look at that definition. I will attempt to describe my (personal) mental model of approaching continuous learning. We all have models on how the world (and our brains) work. Knowing that all models are wrong but some are useful I will proceed with caution on this topic. This is but one of my mental models but might be useful in shaping your thinking.

What is a Magpie?

Someone who hangs at the train station or bus stop or anywhere public all day long, usually asking for a cigarette, spare change or pocket lint. Usually unemployed and unemployable, using nothing but slang, and usually impossible to get rid of, until threatened with violence.

Called magpies because if they have gotten it once, they’ll expect it again, and again, regardless of who it is.

Oh wait. Not that kind of Magpie .

Let us have a look at Jeff’s definition :

I’ve often thought that software developers were akin to Magpies , birds notorious for stealing shiny items to decorate their complex nests. Like Magpies, software developers are unusually smart and curious creatures, almost by definition. But we are too easily distracted by shiny new toys and playthings.

…

I became a programmer because I love computers, and to love computers, you must love change. And I do. But I think the magpie developer sometimes loves change to the detriment of his own craft.

Jeff closes off with the following advice:

That’s the beauty of new things: there’s always a new one coming along. Don’t let the pursuit of new, shiny things accidentally become your goal. Avoid becoming a magpie developer. Be selective in your pursuit of the shiny and new, and you may find yourself a better developer for it.

I cannot agree more with the conclusion. Never let the pursuit of shiny things become a goal in itself to your detriment.

An old wives’ tale

Magpies scrambling to collect shiny objects is a myth .

Researchers placed mounds of edible nuts just 30cm away from each of the collected objects. In 64 tests during feeding, magpies picked up a shiny object only twice - and discarded it immediately.

The birds essentially ignored or avoided both shiny and blue objects, and often fed less when they were present.

Lead author Dr Toni Shephard said: “We did not find evidence of an unconditional attraction to shiny objects in magpies. Instead, all objects prompted responses indicating neophobia – fear of new things.”

While it’s interesting in itself that debunking this myth warranted a research topic, the conclusion appeals to me. Magpies seem to have an innate fear of new things and treat them with suspicion.

People often see the world in black and white, in polar opposites, in extremes. I have learned over time that nothing is black and white. Whenever you find an opinion on the extreme end of a spectrum of options the truth is probably somewhere in the middle.

Let us reign back these absolute statements:

With a more plausible statement:

That is the advice I give my children - don’t be scared, be careful. Then make sure they know the safety rules to avoid death. Fear has lost its usefulness in modern society where we are not hunted down by lions in every day life.

My philosophy is that worrying means you suffer twice.

Newt Scamander (J.K. Rowling, Fantastic Beasts and Where to Find Them )

The V model of learning

Agile proponents talk about “T-shaped” people. The general idea is that effective people have broad (but shallow) knowledge in various topics, but deep knowledge in specific ones. It espouses the idea that a person should be able to work in different disciplines and that is indicative of a persons adaptability in a work environment.

While this model is simple to understand it fails to model the interconnectedness of concepts, that concepts build on each other. It represents a persons skillset as poorly as the skills matrix that recruiters keep asking me for and I refuse to provide.

Knowledge models are better expressed through graphs than trees. Although this T-shaped person can do A and B reasonably well it’s safe to assume that they will struggle with other tasks without a significant time investment in learning. What it means to perform a job “reasonably well” is a subjective and religious topic and inconsequential to this discussion.

What we need are V-shaped people.

Let’s assume a person knows technology A and now needs to learn technology B:

The deeper you go down, the more knowledge you have of how concepts work under the covers. You understand the underlying concepts and are able to transfer those learnings to other technologies or disciplines. The point of competence is the theoretical point that you need to reach to be able to produce work with reasonable competence using technology B. The point of commonality is where the knowledge base for these two branches intersect and where you can reason about them on the same level.

If your understanding is where I’ve plotted it in technology A, you have two options on how to proceed with learning B. You could jump the cliff between A and B with the potential risk of falling to your death:

The jump is difficult, and the end result is that large gaps remain in your knowledge as you have not reached the point of commonality - you have no point of reference. It will be difficult to achieve full competence in any of these technologies since the underlying concepts are still not clear and no skill transfer between A and B has taken place. Don’t get me wrong - there is a time to skim a subject and get the job done - but if you are looking at longevity and productivity in a skill then this approach is sub-optimal. Even worse, the chance of failure to make the leap increases because the fundamentals are missing.

Compare a different approach - starting from the point of commonality and working your way up.

Note that I’m not suggesting a specific learning direction - having a point of commonality available accelerates the learning process. Whether you start with the details and work your way up to higher level abstractions or the other way around is a matter of personal preference.

In reality there are V’s with intersection points:

The more intersection points there are the easier the learning process becomes. You can increase the intersection points available to you by increasing the number of V’s or deepening the learning of existing ones.

The Magpie tries to maximize the potential number of intersection points they have at their disposal through continuous learning. It’s an investment that pays off over the long term.

Generalists are a useful team compliment where tech/product is new, since the core skill is learning and applying new tech fast leveraging fundamentals and the web. I joke and call them full stackoverflow devs
— Jacques de Vos (@jacdevos) March 17, 2018

Build a solid core

Back in 2006 I knew my trade or at least thought I did. .NET 2.0 was the flavour of the day and I made it my personal mission to learn every nook and cranny of the BCL off by heart.

Then, .NET 3.0 got released. With it came Windows Presentation Foundation (WPF) , Windows Workflow Foundation (WWF) , and Windows Communication Foundation (WCF) . These were massive frameworks and it dawned on me that my current approach to learning was not sustainable. That was a hard blow for someone who hoards knowledge and whose basic fear in life is being incompetent .

In response, I pivoted . I learned principles and techniques rather than focusing on the implementation details. I started building a solid core .

I learned how to achieve quality through testing and how to architect applications to accommodate change and scale. I learned how to communicate my intent better by changing the way I write code. I learned how to work and communicate in teams which led me to the principles of agile methods and the psychology behind it .

I became proficient at learning and unlearning.

I became fascinated by people and how they see the world. I built up my leadership skills to lead and manage from both a technical perspective and with a human centric approach. I delved into the principals of Lean manufacturing and opened up my eyes to the economics and methods of software development, product and business development .

I became a more well-rounded technologist which allowed me to perform well in management and strategic positions and still keep in touch with my technical side. And I’m still learning.

The trick behind learning and unlearning is to have a solid core in place. Your solid core represents the skills that are transferable across technologies and paradigms. As a developer you need to carry a toolbox with a decent collection of tools so that you can use the appropriate one when the situation presents itself.

Even if you don’t get to use a technology on a day to day basis, learning it might still be of benefit as you can apply what you’ve learned elsewhere. Having a solid core deepens your V’s and increases the amount of intersection points you have available. Building on practices like TDD allows you to do this with confidence.

Programmers who program “in” a language limit their thoughts to constructs that the language directly supports. If the language tools are primitive, the programmer’s thoughts will also be primitive.

Programmers who program “into” a language first decide what thoughts they want to express, and then they determine how to express those thoughts using the tools provided by their specific language.

My favourite quote from Steve McConnell’s Code Complete book

Intentional learning

The responsible Magpie does not latch on to every shiny object. Life is too short to accumulate unused knowledge. Instead, the responsible Magpie follows a structured process.

graph LR A[Shiny thing] -->|Evaluate..| B{Useful now?} B -->|No.| C{In the future?} C -->|No.| D(Drop it) C -->|Yes.| E(Defer) B -->|Yes.| F{Effort vs Value} F -->|"Bad."| H(Defer) F -->|"Good."| G(Backlog) classDef plain fill:#fff,stroke:#000,stroke-width:2px; classDef question fill:#fff,stroke:#000,stroke-width:2px; classDef conclusion fill:#FDFDF4,stroke:#000,stroke-width:3px; class A plain; class C,B,F question; class D,E,G,H conclusion;

The core evaluation criteria is usefulness (now or in the future) and the effort to learn versus the potential returns.

Haphazard learning isn’t useful. Identify a goal and a plan to achieve that goal. Learning has to be intentional and deliberate. I would recommend keeping a prioritised backlog.

Once the goal is in place set aside some time for learning. Using your personal time for this can be difficult if you have children. But even a 2 hour slot once a week will yield good results.

Set aside time for learning at work. Not making the time at work or your employer denying you that time will be detrimental to both of you.

Programming is in ways like playing a music instrument.
- You have your preferred instrument (language)
- It's an advantage to know another instrument,but not a necessity
- Sometimes it's boring,like practising scales
- Little compares to when you find your groove.
— Pieter Louw (@pieterlouw) March 18, 2018

Also, there’s a bunch of people running around tooting horns they’ve never practised on. And another bunch of people, who’ve never even picked up a instrument, trying to direct them on when to play and complaining if they stop to try tune their instruments.
— Kevin Trethewey (@KevinTrethewey) March 20, 2018

Make peace with the fact that you can’t remember everything. I’m at a point where it feels like as soon as I learn something new something gets discarded. My memory is full and I hope it operates in a LRU fashion. Knowledge gets shipped off to long term storage if frequently used - which, in a way, is a good thing as unused knowledge is pretty worthless.

My brain seems to respond pretty well to saving pointers instead of the actual information. I can’t remember the full content of that article I read but I can sure conjure up the search terms to Google it again.

Do what works best for you.

In conclusion

Practice learning, unlearning, and connecting ideas. It’s the most important skill you will ever gain. Plan ahead on what to learn and look for intersection points as a guide. Try to know what you don’t know. It will be worth your while in the long run.

I’m a Magpie, but a responsible one. And that’s ok.

Photo by Marina Salles on Unsplash

Shiny things

Tue, 27 Mar 2018 00:00:00 +0000

Rabbit holes

I disappear from time to time. No, not in the physical sense. Rabbit holes are everywhere. I follow them and emerge a couple of days (or weeks) later, hopefully wiser.

For illustration, I wandered through Docker land for a whole 6 months. That particular rabbit hole went deep. The weird and wonderful container landscape is still maturing and changing fast. Below the surface of a seemingly simple virtualisation technology lies myriad tools, concepts and philosophies. We have built out a single, auto-scaling CI grid that runs in containers, using other containers to build new container images, spinning up containers from those newly built images and testing them before publishing them. It’s turtles all the way down .

Other than the standard unit and integration testing, we have to add static code analysis , application vulnerability analysis and image vulnerability analysis before an image even makes it to a private registry . Then there are production considerations - orchestration with Rancher and Kubernetes (amongst other flavours), vulnerability analysis of production images and network security scans , scaling across geographic locations with redundancy, logging infrastructure, monitoring and reporting. Inside environments you can have containers that do nothing but provide services for other containers - whether that be providing data volumes by being a data container or providing signals for routing and scheduling through labels. Yes, it’s possible to build a container that is less than 1kb for those use cases. It’s mind blowing. “The Docker way” is an enabler for micro-services and has changed the way I architect applications. It’s difficult to explain until you’ve chosen the red pill.

You take the blue pill - the story ends, you wake up in your bed and believe whatever you want to believe. You take the red pill - you stay in Wonderland, and I show you how deep the rabbit hole goes. Remember: all I’m offering is the truth. Nothing more.

Morpheus, The Matrix

Sometimes following rabbit holes lead to procrastination. I haven’t posted to this blog in a long time because I wanted to:

change my blog from Octopress to pure Jekyll
convert all my old posts from HTML to Markdown
migrate my blog to run on Docker in AWS
recreate my automated deployment workflow
set up SSL using Let’s Encrypt

It took me a while to get around to it. I also refused to post until I could do my writing on a mobile device:

The novelty of being able to do this on my phone never wears off for me. // @termux pic.twitter.com/S6x1AbuCnp
— Riaan Hanekom (@rhanekom) February 22, 2018

Soon after, I became frustrated with the limitations of Jekyll. I found the excuse I was looking for to try out Gatsby.js . After all, by now I have stopped treating JavaScript like a toy language and delved deep into it. I got my hands dirty with React and had GraphQL on my todo list. Now, on top of linting my JavaScript code my workflow incorporates a linter for Markdown , spelling and prose . No reason exists not to treat your blog like you would any other production application.

In preparation for coming posts I wrote a Gatsby plugin called gatsby-remark-draw to render static diagrams using SvgBob . I proceeded down the rabbit hole and added Graphviz and Mermaid.js support as well. Combine it with React hot loader and DrawIt and you can see your ASCII drawings come to life as you hit save. It’s a beautiful thing.

Faster and faster

Technology is moving at an incredible pace. This is evident in the plethora of new languages, frameworks and paradigms we are coming up with. We are working with new tools and need to adapt to entire new ways of thinking and reasoning about solutions. The pace of innovation is accelerating and there is no sign of it dissipating.

We’ve seen an entire transformation in web development over the last decade. JavaScript flourished as browsers matured aided by the increase of computing power in the hands of developers and consumers of software. Transpiling is now commonplace where we have SASS , LESS , CoffeeScript , TypeScript , and others. Tools like Babel allows us to use new JavaScript features by transpiling to older versions of the language. We can tie everything together with webpack to hide some of the inherit complexities in polyglot programming and large scale application development. We are challenging conventional wisdom with Atomic CSS , CSS in JavaScript and Component Styles .

And then, the uber frameworks. Angular.js and React have transformed the way we build user interfaces and handle data and user interaction. Node.js brought JavaScript to the server side and the desktop. Running a proper command line utility on my machine that was pure JavaScript was a humbling experience back in the day.

When npm was first released in 2010, the release cycle for typical nodeJS package was 4 months, and npm restore took 15-30 seconds on an average project. By early 2018, the average release cycle for a JS package was 11 days, and the average npm restore step took 3-4 minutes. 1/11
— Dylan Beattie 🇪🇺 (@dylanbeattie) March 22, 2018

The thread is worth reading.

The “big data” revolution has spawned an industry by itself where open source solutions became ubiquitous. Data collection, analysis and prediction tools became accessible to the masses. General use databases are now commodity. PostgreSQL , MySql , MongoDB , Riak , Redis and Cassandra (to mention some) gained widespread adoption.

I have been around long enough to witness the server revolution first-hand:

These are some examples. Don’t get me started on languages .

The list goes on and on. Novelty only lasts so long before it becomes generalised. There are so… …many… …shiny… …things. It’s exhausting and exhilarating at the same time.

I am not surprised that developers feel overwhelmed. Building a modern web application requires knowledge of a thousand frameworks, tools and languages. Take a closer look at the dependencies of the create-react-app application to see what I mean. Staying on top of it all is hard for anyone.

Nobody knows everything

Let us deal with a common theme in technology community - imposter syndrome . Scott’s post from 2011 still resonates with me. In particular, Scott quotes Dave Ward :

I think the more you know, the more you realize just how much you don’t know. So paradoxically, the deeper down the rabbit hole you go, the more you might tend to fixate on the growing collection of unlearned peripheral concepts that you become conscious of along the way.

That can manifest itself as feelings of fraudulence when people are calling you a “guru” or “expert” while you’re internally overwhelmed by the ever-expanding volumes of things you’re learning that you don’t know.

However, I think it’s important to tamp those insecurities down and continue on with confidence enough to continue learning. After all, you’ve got the advantage of having this long list of things you know you don’t know, whereas most people haven’t even taken the time to uncover that treasure map yet. What’s more, no one else has it all figured out either. We’re all just fumbling around in the adjacent possible, grasping at whatever good ideas and understanding we can manage to wrap our heads around.

The first part is an adaption from a quote widely attributed to Albert Einstein but Dave adds beautiful context to it. That statement made a profound impact on my life, my career, and my self-image.

The more I learn, the more I realize how much I don’t know.

Albert Einstein

I cannot stress this point enough. Nobody knows everything. People who boast about their knowledge and abilities are unaware of what they don’t know. It’s healthy to keep your skills up to date and increase your knowledge, but comparing yourself to others leads to increased anxiety. Attempting to know more than everyone else is an exercise in futility.

Here is what is more important. Have confidence in what you do know while keeping to the spirit of “strong opinions, weakly held” . Make a decent attempt at knowing what you don’t know.

In a world with too much information and too little time, focus on your ability to digest and process what matters. Let go of what is not useful any more.

In this industry, if you are not moving forward you are moving backwards. Become good at learning and unlearning.

See how to do that in the follow up post - The Magpie.

Photo by Glenn Carstens-Peters on Unsplash

Setting up Jekyll - navigation with a modern site-map (and R.I.P)

Sun, 18 Mar 2018 00:00:00 +0000

Preface

In previous posts in this series I’ve used categories rather than tags. I have now switched these around because naming matters. If you need to do this, a quick sed execution will do the trick:

Bash

1

sed -i 's/categories:/tags:/g' *.md

Having changed the variable of tags we also need to change the include for displaying tags to replace categories with tags.

Autopages

The jekyll-paginate-v2 gem has built-in support for generating pages based on the tags and categories in posts. First, we need enable Autopages through our site configuration:

YAML


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


autopages:
 enabled: true

 categories:
 enabled: true
 layouts:
 - 'home.html'
 title: 'Posts in category :cat'
 permalink: '/category/:cat'

 collections:
 enabled: false

 tags:
 enabled: true
 layouts:
 - 'home.html'
 title: 'Posts tagged with :tag'
 permalink: '/tag/:tag'

I don’t use collections on this blog right now so I’ve left that as disabled. I’ve updated the home.html file from the Minima theme to display excerpts when Jekyll recognises a page as auto-generated and to add the title specified in the configuration file. This means that there is a single page to maintain for the front page as well as the generated content:

HTML


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28


<h1 class="page-heading">
 {% if page.title and page.title != site.title %}
 {{ page.title | escape }}
 {% else %}
 Posts
 {% endif %}
 </h1>

 {{ content }}

 <ul class="post-list">
 {% for post in paginator.posts %}
 <li>
 <span class="post-meta">{{ post.date | date: "%b %-d, %Y" }}</span>

 <h2>
 <a class="post-link" href="{{ post.url | relative_url }}">{{ post.title | escape }}</a>
 </h2>
 {% if page.autopages %}
 <p>{{ post.excerpt }}</p>
 <p><a href="{{ post.url }}">READ MORE &gt;&gt;</a></p>
 {% else %}
 <p>{{ post.content }}</p>
 {% include page_tags.html tags=post.tags %}
 {% endif %}
 </li>
 {% endfor %}
 </ul>

We can then add a link to the tag page in the permalink format we specified in the config file which will take us to a page with the tag title and excerpts (with paging) from all the blog posts with the specified tag.

HTML

1

<li><a href="/tag/{{tag}}">{{ tag }}</a></li>

Note: The tags on this blog are simplistic. If you have some special characters in your tags the link will require some more filtering as the paginate gem performs transformations on the tag name in the URL.

Improving performance for local build

Enabling autopages can have a noticeable impact on the site build time since it needs to generate many pages. I’ve added a separate configuration file that disables expensive features so that I can get faster feedback on changes made, _config_dev.yml:

YAML


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36


###################################################
## Disable features to speed up the build locally #
###################################################

# Disable autopages
autopages:
 enabled: false

 categories:
 enabled: false

 collections:
 enabled: false

 tags:
 enabled: false

# Disable noscript feature for gists as this makes an API call to Github for every gist
gist:
 noscript: false

# Disable LSI because this can be dog slow
lsi: false

#########################
## Development features #
#########################

# Enable showing drafts
show_drafts: true

# Enable showing future posts
future: true

# Enable incremental builds
incremental: true

With this config in hand we need to direct jekyll to the configuration files to use. In this example, any values specified in _config_dev.yml will override the values in _config.yml.

Bash

1

jekyll serve --config _config.yml,_config_dev.yml

See the configuration documentation for a list of options available.

A site map

Let’s start building a site map to add some extra options for navigation. Yes, I’m old school like that.

Create a page in the root of the source content ( map.md in my case) so that Jekyll can process it. With the Minima theme this will automatically show up in the main menu.

To start off we’ll generate a list of tags and size the elements according to their usage frequency using inline styles. Following the advice on this post and borrowing some code from it,

HTML


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


---
layout: page
title: Map
permalink: /map/
---

## All tags

{% assign tags = site.tags | sort %}
<div class="post-tags">
 <ul>
 {% for tag in tags %}
 <li>
 <a href="/tag/{{ tag | first | slugify }}/" style="font-size: {{ tag | last | size | times: 6 | plus: 80 }}%">
 {{ tag[0] | replace:'-', ' ' }} ({{ tag | last | size }})
 </a>
 </li>
 {% endfor %}
 </ul>
</div>

To avoid overflows of the tag text to the next line and cropped borders, we add some margins to the top and bottom, set white-space to nowrap and set the list item style to inline-block.

CSS


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


.post-tags {
 ul {
 list-style: none;

 li {
 display: inline-block;
 margin-top: 10px;
 margin-bottom: 10px;

 a {
 padding: 5px 10px 5px 10px;
 border-radius: 20px;
 background: #EEEEFF;
 white-space: nowrap;
 }
 }
 }
}

The results look promising, though a bit unpolished:

I have some cleaning up to do.

Most recent

We can display an index to the most recent posts. We’ll do this by starting off with a generic component to display a list of posts given to it.

HTML


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


{% if include.items %}
 <div class="post-mini-list">

 {% if include.heading %}
 <h3>{{ include.heading }}</h3>
 {% endif %}

 <ul>
 {% for post in include.items limit: 5 %}
 <li>
 <a href="{{ post.url }}">{{ post.date | date: site.dateformat }} - {{ post.title }}</a>
 </li>
 {% endfor %}
 </ul>
 </div>
{% endif %}

This partial expects page to provide two variables - include.items which contain some posts, and include.heading which could contain an optional heading. Note the limit on 5 posts when iterating over the posts so that we don’t end up with a massive list of posts. Also note the variable usage of site.dateformat, which I’ve set in the _config.yml file. You can find a good reference to specifying date formats to use in Liquid tags here .

We can include this into the about page:

HTML

1

{% include post-list.html items = site.posts heading = "Latest" %}

R.I.P. Jekyll

This would have been part of this article if I did not rage quit:

List of series
Most popular
Most recent
By date buckets

It’s hard to build advanced features in Jekyll. The API exposed by the framework impose limitations to the point where any sufficiently advanced plugin becomes a mess of spaghetti code in time. Back in the day when static site generators were a novelty, Jekyll was the solution. Times have moved on and there are hoards of good alternatives available.

I’ve moved this blog over to Gatsby.js . Gatsby is super fast, fully customisable and spits out a Progressive Web App built on top of React. The migration process was seamless other than having to fix some posts’ frontmatter. Customisation takes a while longer as you find your way around the starter pack. On that note, I tried hard not to break any permalinks but one or two mistakes might have slipped through. I’ll leave advanced Gatsby.js topics for another post and move on.

Photo by Jeremy Bishop on Unsplash

Setting up HTTPS with Let's Encrypt

Mon, 05 Mar 2018 00:00:00 +0000

Chrome 68, due in July 2018, will mark all non-HTTPS sites as insecure . Instead of buying a certificate it seemed like an apt opportunity to use Let’s Encrypt , a free and automated Certificate Authority. It’s easy enough to get started with Certbot supporting most platforms .

Getting started

By far the easiest way of getting started is if you have access to the web content directory. Certbot will generate a file in a well known path and handle the handshaking process between Let’s Encrypt and the website to verify the websites identity. Google Analytics has a similar approach to prove site ownership and the ACME protocol documents the process.

I’m using an Alpine Linux image for my blog container and a package for Certbot exists in the standard repositories. This installs other dependencies (notably Python) which bloats the image somewhat but it’s a small price to pay in this instance. Here’s the amended Dockerfile:

Dockerfile


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


FROM nginx:alpine

# Copy content
COPY ./_site /usr/share/nginx/html

# Copy config
COPY nginx.conf /etc/nginx/nginx.conf
COPY default.conf /etc/nginx/conf.d/default.conf

VOLUME /etc/letsencrypt
RUN apk add --no-cache bash certbot

EXPOSE 80
EXPOSE 443

First we create a volume for /etc/letsencrypt where Certbot will save the certificates. This allows other containers to use the same certificate by mounting the same volume. On the server I’ve manually created a volume with the following command:

Bash

1

docker volume create --name certificates

We add the certbot package using apk for later use and of course expose port 443 for SSL. When we run the container we need to mount the newly created volume at the location specified in the Dockerfile:

Bash

1
2

docker run -d --name blog -p 80:80 -p 443:443 \
 -v certificates:/etc/letsencrypt blog:latest

With this container built and running, we need to get the initial certificates manually. Inside the container we ask Certbot to generate a certificate for us. I’ve included both yourdomain.com and <www.yourdomain.com> as I have a redirect from www to non-www. The --webroot directive makes Certbot use the file system for certificate generation and validation and not integrate directly with the web server through a plugin - I prefer to edit the website configuration by hand.

The --agree-tos and --email flags are necessary to avoid interactive prompts for this information on first registration.

Bash


1
2
3


certbot --webroot -d yourdomain.com -d www.yourdomain.com \
 --webroot-path /usr/share/nginx/html \
 --email youemail@domain.com --agree-tos

You should see the output shown below. Take note of the renewal instructions - we’ll get back to that.

Bash


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


Congratulations! Your certificate and chain have been saved at:
 /etc/letsencrypt/live/yourdomain.com/fullchain.pem
 Your key file has been saved at:
 /etc/letsencrypt/live/yourdomain.com/privkey.pem
 Your cert will expire on xxxx-xx-xx. To obtain a new or tweaked
 version of this certificate in the future, simply run certbot
 again. To non-interactively renew *all* of your certificates, run
 "certbot renew"
 - Your account credentials have been saved in your Certbot
 configuration directory at /etc/letsencrypt. You should make a
 secure backup of this folder now. This configuration directory will
 also contain certificates and private keys obtained by Certbot so
 making regular backups of this folder is ideal.
 - If you like Certbot, please consider supporting our work by:

 Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
 Donating to EFF: https://eff.org/donate-le

You can find the generated certificates inside the /etc/letsencrypt/live folder.

NGINX setup

Now that we have the certificates we can set up the web server (NGINX in my case) to use them. First, let’s redirect all traffic from port 80 to port 443:

Nginx


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


server {
 listen 80;
 server_name www.yourdomain.com;
 return 301 https://yourdomain.com$request_uri;
}

server {
 listen 80;
 server_name yourdomain.com;
 return 301 https://yourdomain.com$request_uri;
}

This will return a 301 moved status code and keep the original sub-URI for redirection. Now to set up HTTPS :

Nginx


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


server {
 listen 443 ssl default deferred;
 server_name yourdomain.com www.yourdomain.com;

 ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
 ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;

 # Session resumption
 ssl_session_cache shared:SSL:10m;
 ssl_session_timeout 5m;

 # Specify types of TLS, specifically avoiding SSL3
 ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

 # Disable insecure ciphers
 ssl_prefer_server_ciphers on;
 ssl_ciphers "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS";


 # HTTP Strict Transport Security
 add_header Strict-Transport-Security "max-age=63072000; includeSubdomains";

 # ....
}

It’s important to disable SSLv3 as the protocol is vulnerable to attacks so we specify TLSv1, TLSv1.1, and TLSv1.2 explicitly as the allowed versions.

Furthermore we add HSTS as another layer of defence to avoid access of the website over HTTP by default :

The HTTP Strict Transport Security header informs the browser that it should never load a site using HTTP and should automatically convert all attempts to access the site using HTTP to HTTPS requests instead.

In other words, once the browser spots this header and once it confirms that the site has HTTPS enabled it will always use HTTPS for future requests. We can set the max-age parameter as high as it can go as I’m not planning on rolling this back to HTTP any time soon.

We also need to disable insecure ciphers to avoid downgrading of the connection to a less secure cipher. SSLlabs have a great article that they frequently update with best practices in this regard .

We enable session resumption for performance reasons. The server will store a session id of the client so that future connections with the same session id (in a short time frame) the client can resume the session instead of going through the whole negotiation process again.

Pro tip: The SSLlabs TLS testing tool is the best tool to test your SSL implementation and will highlight any potential issues. While using it read the notes about compatibility - it’s essential to get the balance right between security and compatibility.

Renewing certificates

Let’s Encrypt certificates are valid for three months so we need to run the renew process frequently to avoid sitting with an expired certificate. Certbot has this built in with the certbot renew command, mentioned in the initial output after first creation of the certificates. There are a couple of different ways to do this with containers:

Running a sidekick container to renew the certificates
Installing the dependencies on the host and running the command directly on the volume
Running crontab on the host machine and exec’ing into the container

(2) is way too dirty and spoils all the isolation benefits we have with containerisation and (1) is the way I would do it with serious production applications abiding to “the Docker way”. In the spirit of doing the simplest thing possible I opted for (3). On the host machine you can edit the crontab file via the command

Bash

1

sudo crontab -e

We can create a simple entry to run the renew command in the container on a schedule:

Bash

1

43 6 * * * docker exec blog certbot renew --post-hook "nginx -s reload"

This will run the renew process and if the script renewed the certificate execute the post hook, which restarts the web server and let the change take effect.

Note

Let’s Encrypt applies rate limiting on requests to their services. While you are testing you can use their staging area to avoid hitting the request ceiling.

Photo by James Sutton on Unsplash

Setting up Jekyll - related posts and series

Thu, 01 Mar 2018 00:00:00 +0000

By default, the related_posts feature of Jekyll returns the newest 10 posts rather than related posts. This is to speed up generation of the site by not using it’s Latent Semantic Indexing (LSI) capabilities implemented by classifier-reborn since that takes a while to run.

To use LSI, you need to add the --lsi flag. Out of the box, it takes an incredibly long time to run. On my small blog, it took over an hour to complete the build process with LSI included. We’re in luck though - installing GSL and the rb-gsl gem speeds up the process (down to less than a minute for my blog). Advice from the rb-gsl gem after installation is that you could install the nmatrix and narray gems as well, but do this before rb-gsl installation.

I’ve found that adding the nmatrix and narray gems to your Gemfile causes runtime dependency issues (specifically, duplicate definitions of classes). To automate the process, I’ve added a separate Gemfile.preinstall with these gems included to aid in the automated deployment on my web server. You can do this by taking advantage of the --gemfile parameter to bundler:

Bash

1

bundle install --gemfile Gemfile.preinstall

To install rb-gsl, install gsl as well as the development libraries:

Bash

1

sudo apt-get install gsl-bin libgsl2 libgsl-dev

And then add the rb-gsl to your Gemfile and install it.

Note: You can also directly install from source:

Bash


1
2
3
4
5
6


wget ftp://ftp.gnu.org/gnu/gsl/gsl-latest.tar.gz
tar -xvf gsl-latest.tar.gz
cd gsl*
./configure
make
sudo make install

To link to the library, add the following to your .bashrc (or .zshrc, whichever shell you are using):

Bash

1

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/

Now that we have installation completed we can use the related_posts property with confidence. Note that this property is not supported by GitHub pages, a common hosting platform, and by default contains the x most recent posts if the --lsi flag is not passed to the generator.

We’ll start off with a simple list of related posts in the main post page, limiting it to three recommendations:

HTML


 1
 2
 3
 4
 5
 6
 7
 8
 9
10


{% if site.related_posts.size > 0 %}
<aside>
 <h3>Related:</h3>
 <ul>
 {% for related in site.related_posts limit:3 %}
 <li><a href="{{related.url}}">{{ related.title }}</a></li>
 {% endfor %}
 </ul>
</aside>
{% endif %}

The final result looks pretty accurate:

Post series

Another thing that is missing is to automatically link a series of posts together. We can achieve this by adding a custom series variable in the front matter of a post. This variable specifies the name of the series which we can find posts with and group them together. For lack of a better name we will call the variable series:

Markdown


1
2
3
4
5


---
title: "Setting up Jekyll - related posts and series"
tags: jekyll ruby blog
series: building-a-blog
---

The include for this looks something like the following:

HTML


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


{% if include.series %}
 <aside class="series">
 <h3>Choose your own adventure! This post is part of the series <strong>&quot;{{ include.series | replace: '-', ' '| capitalize }}&quot;</strong></h3>
 <ol>
 {% for post in site.posts reversed %}
 {% if post.series == include.series %}
 <li>
 {% if post.url == include.url %}
 {{ post.title }} &lt;-- <strong>this post</strong>
 {% else %}
 <a href="{{ post.url }}">{{ post.title }}</a>
 {% endif %}
 </li>
 {% endif %}
 {% endfor %}
 </ol>
 </aside>
{% endif %}

SCSS


1
2
3
4
5
6
7


aside.series {
 border-color: #BABDB6;
 border-style: solid;
 border-width: 1px;
 padding: 5px 10px 5px 10px;
 margin: 5px 0px 5px 0px;
}

And the final result:

Photo by rawpixel.com on Unsplash

Setting up Jekyll - prettier paging and categories

Wed, 28 Feb 2018 00:00:00 +0000

Following up from the previous post on setting up Jekyll, we take out our tools and customize it some more. When reading any of my posts you should realise I’m after utility rather than beauty. That said, we’ll try to improve the appearance to an acceptable level as we go along.

Prettier paging

The next/previous paging is functional, but rather ugly. The jekyll-pagination-v2 gem has what we need with support for pagination trails. We need to update our config for this:

YAML


1
2
3
4


pagination:
 trail:
 before: 2
 after: 2

And replace our previous paging with a more functional one, unceremoniously borrowed from the example page .

HTML


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35


{% if paginator.total_pages > 1 %}
<ul class="pager">
 {% if paginator.first_page %}
 <li class="previous">
 <a href="{{ paginator.first_page_path | prepend: site.baseurl | replace: '//', '/' }}">First</a>
 </li>
 {% endif %}

 {% if paginator.previous_page %}
 <li class="previous">
 <a href="{{ paginator.previous_page_path | prepend: site.baseurl | replace: '//', '/' }}">&larr; Newer Posts</a>
 </li>
 {% endif %}

 {% if paginator.page_trail %}
 {% for trail in paginator.page_trail %}
 <li {% if page.url == trail.path %}class="selected"{% endif %}>
 <a href="{{ trail.path | prepend: site.baseurl | replace: '//', '/' }}" title="{{trail.title}}">{{ trail.num }}</a>
 </li>
 {% endfor %}
 {% endif %}

 {% if paginator.next_page %}
 <li class="next">
 <a href="{{ paginator.next_page_path | prepend: site.baseurl | replace: '//', '/' }}">Older Posts &rarr;</a>
 </li>
 {% endif %}

 {% if paginator.last_page %}
 <li class="previous">
 <a href="{{ paginator.last_page_path | prepend: site.baseurl | replace: '//', '/' }}">Last</a>
 </li>
 {% endif %}
</ul>
{% endif %}

We’ll add some styling to make it look slightly prettier:

SCSS


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


ul.pager {
 list-style: none;

 li {
 display: inline;
 padding: 5px 5px 5px 10px;
 border-color: #BABDB6;
 border-style: solid;
 border-width: 1px;

 }

 li.selected {
 background-color: #EEEEFF;
 }
}

And voilà!

Displaying categories

Displaying categories on pages are a useful way of navigating a website to find related content in collections curated by the author. These are also known as tags in blog nomenclature but I’ve opted to use the tags variable (pre-defined in Jekyll) for internal organization of pages (more on this later in a later post). With categories already a native concept in Jekyll we can iterate over them on a page level (_layouts/post.html in the Minima theme):

HTML


1
2
3
4
5
6
7
8


<div class="post-categories">
 <h3>Posted in:</h3>
 <ul>
 {% for category in page.categories %}
 <li>{{ category }}</li>
 {% endfor %}
 </ul>
</div>

And then add some rounded corners and pretty colours via CSS:

SCSS


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


.post-categories {
 ul {
 list-style: none;

 li {
 display: inline;
 padding: 5px 10px 5px 10px;
 border-radius: 20px;
 background: #EEEEFF;
 }
 }
}

I changed this home page to display full post content and not the excerpts so that one can read in peace without having to click through to an article. To display the categories on the home page as well we need to edit the home layout (_layouts/home.html in the Minima theme). Instead of copying and pasting, let’s move the categories markup to an include file , _includes/page_categories.html in my case:

HTML


 1
 2
 3
 4
 5
 6
 7
 8
 9
10


{% if include.categories %}
<div class="post-categories">
 <h3>Posted in:</h3>
 <ul>
 {% for category in include.categories %}
 <li>{{ category }}</li>
 {% endfor %}
 </ul>
</div>
{% endif %}

Notice that the variable we loop over is now include.categories. The different pages/layouts pass this as an include variable as such:

HTML

1

{% include page_categories.html categories=post.categories %}

In a future post we’ll explore category pages and tag clouds.

Photo by Goran Ivos on Unsplash

Converting HTML blog posts to Markdown

Sun, 25 Feb 2018 00:00:00 +0000

I am reviving some projects - this blog as well as some open source work. They desperately need some TLC…

Since this blog has travelled through different hosting options and technologies I still had some legacy posts formatted in HTML. I’ve taken the plunge to refactor them into much cleaner Markdown syntax.

This turned out easier than expected. With a couple of good libraries to lean on I wrote a quick Node application to do the dirty work. This aspect of the Node.js community is the same reason why I fell in love with Ruby in the first place - the wealth of small, well-crafted libraries/gems/packages that focus on solving specific problems eloquently.

We’ll design the app with stream input and output, taking HTML input from stdin and outputting the resulting Markdown on stdout. In this way we can use it in conjunction with other tools, true to the Unix tools philosophy . First, let’s start with package.json file. We can create one with npm init and fill in the blanks:

JSON


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


{
 "name": "htmlconvert",
 "version": "1.0.0",
 "description": "",
 "main": "index.js",
 "scripts": {
 "test": "echo \"Error: no test specified\" && exit 1"
 },
 "dependencies": {
 "turndown": "^4.0.1",
 "get-stdin": "^5.0.1",
 "turndown-plugin-gfm": "^1.0.1",
 "gray-matter": "^3.1.1"
 },
 "bin": {
 "htmlconvert": "index.js"
 },
 "author": "Riaan Hanekom",
 "license": "ISC"
}

We make use of these great packages:

get-stdin : gets stdin as a string or buffer
turndown : convert HTML to Markdown using JavaScript
turndown-plugin-gfm : a plugin for turndown to enable GitHub Flavoured Markdown
gray-matter : a library that parses different types of front-matter

Note the bin section in the package.json file. This allows us to run npm link and have the convenience of not having to type in node index.js every time we want to run the app. If we decided to publish this application as an official npm package we would have to set the preferGlobal flag to true as well so that a user gets warned if the package is not installed with the --global flag [further reading ].

Here is the index.js file marked for execution in all its (quick and dirty) glory:

JavaScript


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


#! /usr/bin/env node
const getStdin = require('get-stdin');
const TurndownService = require('turndown');
const turndownPluginGfm = require('turndown-plugin-gfm');
const matter = require('gray-matter');

getStdin().then(str => {

 let turndown = new TurndownService( {
 "codeBlockStyle": "fenced",
 "linkStyle": "referenced"
 });

 let gfm = turndownPluginGfm.gfm;
 turndown.use(gfm);

 let parsed = matter(str);
 let content = parsed.content;
 let data = parsed.data;

 let markdown = turndown.turndown(content);
 console.log(matter.stringify(markdown, data));
});

First we retrieve all the input from stdin. The turndown package is nicely customizable and I’ve set the output format styles to what I prefer where it differs from the defaults .

We ask the turndown library to use the gfm plugin to support GitHub Flavoured Markdown. The turndown library strips out newlines from the Jekyll front-matter at the moment. This issue provides a simple workaround - use the gray-matter library to parse the front matter.

Now that we have the input and output mechanisms ready, we can write

Bash

1

echo "<b>ICanHaz Bold!</b>" | htmlconvert

The output will be:

Markdown

1

**ICanHazBold!**

Let’s pipe a sample blog post with front matter in:

Bash

1

STR=$'---\n title: stufff \n---\n<b>ICanHazBold\!</b>' && echo "$STR" | htmlconvert

Output:

Markdown


1
2
3
4


---
title: stufff
---
**ICanHazBold!**

Beautiful!

We can now convert our posts in bulk by iterating over all the HTML posts:

Bash


1
2
3


for file in *.html; do
 cat "$file" | htmlconvert > "$(basename "$file" .html).md"
done

And then we dutifully proceed with QA on each converted post before deleting the original. This code will not handle some of the edge cases that I was coding up in HTML back in 2005. To be honest, I’m not sure whether this was intentionally bad markup or signs of scars received while fighting with Wordpress, but inline styles for italics and bold text and auto-closing paragraph tags (<p/>) are some examples. Those cases are rare so I chose to handle them manually rather than diving into the insanity which is HTML parsing .

Photo by Pankaj Patel on Unsplash

Setting up Jekyll - the basics

Thu, 22 Feb 2018 00:00:00 +0000

Introduction

Since Octopress has been deprecated for pure use of Jekyll, I switched this blog to use Jekyll. See this post as a walkthrough on creating a functional blog via Jekyll (part 1). This is yet another collation of snippets I have found useful, as a reference for my future self - and perhaps others.

Starting off

You can find the installation instructions for Jekyll in the official documentation .

After you install the Jekyll gem, scaffolding a new site is as easy as

Bash

1

jekyll new blog_name

This will create a new folder with a starter site. To serve your site to preview it, run the

Bash

1

jekyll --serve

command and navigate to http://localhost:4000 (the default port) with your favourite browser.

Defaults, excerpts and URL structures

By default the home page will display the full list of posts with all it’s glorious content. This might be a bit too much for longer posts. To enable excerpts with links to the full post you need to configure the excerpt_seperator. You can specify this in the front matter of each post. We can also specify the format of the permalink to the posts, with a sane URL structure including the date and title of the post.

Instead of repeating ourselves in the frontmatter of each post we can DRY it up a bit by specifying defaults in the config.yml file:

YAML


 1
 2
 3
 4
 5
 6
 7
 8
 9
10


defaults:
-
 scope:
 path: "" # an empty string here means all files in the project
 type: "posts" # previously `post` in Jekyll 2.2.
 values:
 layout: "post"
 excerpt_separator: "<!`--` more `--`>"
 comments: true
 permalink: "/blog/:year/:month/:day/:title"

The format YYmmDD is locale specific - but one advantage with using this format is that your posts can be chronologically ordered even on the file system. The final URL for a post would look something like this:

HTML

1

http://riaanhanekom.com/2018/01/21/travis-with-sonarcloud

With the configuration above applied, content displays on the home page up to where the content processor finds the  marker:

Override the look and feel

We can create a main stylesheet in the assets directory in your source folder. main.scss will be the Sass file included after everything else so you can override any styles that you want. Since we’ve used the minima theme it’s important that we import it at the top. Let’s put in a separator between the posts:

SCSS


1
2
3
4
5
6
7


@import "minima";

.post-list>li {
 border-bottom: 1px solid #b6b6b6;
 padding-bottom: 30px;
 margin: 10px 0px 30px 0px;
}

I’m not a big fan of the 800px fixed-width style - by adding the following CSS into assets/main.scss we can take up more real-estate on the screen:

CSS


1
2
3
4
5
6
7


.wrapper {
 max-width: none;
 margin-right: auto;
 margin-left: auto;
 padding-right: 10%;
 padding-left: 10%;
}

To override the markup provided by the default theme you can “eject”, copy the files from the gem installation directory into your own source directory and change them at will.

The command

Bash

1

bundle show minima

will show you the source directory to copy the files from.

Productionise it

As with any software development it’s important to get the production aspects out of the way. You can always change this later, but doing this early in the process avoids a bunch of “whoops, didn’t think about that” moments. Deploying to GitHub pages is a popular method of serving a blog made with Jekyll, but since I’ll be deploying this on a private server we’ll focus on building and running a docker container for this purpose. Let’s create a simple NGINX server to host our application with this Dockerfile:

Dockerfile


1
2
3
4


FROM nginx
COPY ./_site /usr/share/nginx/html
COPY nginx.conf /etc/nginx/nginx.conf
COPY default.conf /etc/nginx/conf.d/default.conf

Simple enough. Jekyll generates the site static content under the _site directory which can we can copy into the default serving directory of NGINX. I’ve added a configuration file to add cache control headers for static content (images, js, css). HTML pages are still set for the browser to check back to server if any content has changed. We also enable gzip and set up NGINX to scan for the index.html file in directories to have nice file-less URLs:

Nginx


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43


user nginx;
worker_processes 1;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
 worker_connections 1024;
}

http {
 server {
 location / {
 try_files $uri $uri/index.html $uri.html =404;
 }

 location /assets {
 expires 1M;
 add_header Cache-Control "public, max-age=691200, s-maxage=691200";
 }

 location /images {
 expires 1M;
 add_header Cache-Control "public, max-age=691200, s-maxage=691200";
 }
 rewrite ^/(.*)/$ /$1 permanent;
 expires $expires;
 }

 include /etc/nginx/mime.types;
 default_type application/octet-stream;

 log_format main '$remote_addr - $remote_user [$time_local] "$request" '
 '$status $body_bytes_sent "$http_referer" '
 '"$http_user_agent" "$http_x_forwarded_for"';

 access_log /var/log/nginx/access.log main;
 sendfile on;
 keepalive_timeout 65;
 gzip on;

 include /etc/nginx/conf.d/*.conf;
}

At this point don’t forget to update the header include template to add a cache buster for the stylesheet:

HTML

1

<link rel="stylesheet" href="{{ "/assets/main.css" | relative_url }}?{{site.time | date: '%s%N'}}">

The generated link to the stylesheet would look something like /assets/main.css?1519308247603964426. The time value at the point of generation gets appended to the stylesheet URL allowing us to cache the asset on clients until it changes.

Working with drafts

To work on draft posts without publishing them, create a ‘drafts’ folder in your source folder per the documentation . When running jekyll serve, pass in the –draft parameter to show drafts. Depending on your needs it might also be useful to add the –future parameter to display posts that have a date in the future.

Paging

No good blog should be without paging. By default my installation came with the jekyll-paginate gem installed. I’ve found that the next iteration of the gem, jekyll-paginate-v2 works better and is simpler to set up.

To activate, add the jekyll-paginate-v2 to the plugins section of the config and your Gemfile. There are tons of configuration options for pagination, but here were the ones I found important for now:

YAML


1
2
3
4
5


pagination:
 enabled: true
 collection: 'posts'
 per_page: 8
 permalink: '/page/:num/'

Where you display your posts you will need to iterate over the paginator.posts instead of posts:

Ruby

1

for post in paginator.posts

If you are using the minima theme, it’s time to copy over the home.html file from the layouts folder and update it. We also shouldn’t forget to add the pagination links at the bottom:

HTML


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


{% if paginator.total_pages > 1 %}
<ul>
 {% if paginator.previous_page %}
 <li>
 <a href="{{ paginator.previous_page_path | prepend: site.baseurl }}">Newer</a>
 </li>
 {% endif %}
 {% if paginator.next_page %}
 <li>
 <a href="{{ paginator.next_page_path | prepend: site.baseurl }}">Older</a>
 </li>
 {% endif %}
</ul>
{% endif %}

This gives us an easy way to navigate between pages:

Hmmm, functional, but not pretty. We’ll improve on this in future posts.

Photo by rawpixel.com on Unsplash

Using Sonar Cloud on .NET Core with Travis

Sun, 21 Jan 2018 00:00:00 +0000

Note to self since I spent a frustratingly long time on this. To analyse your .NET Core project with Sonar on Linux (using Travis in my case since it’s an open source project), you need the following:

Have a machine with Mono and .NET Core installed
Download the MSBuild Sonar Analyzer and ensure that the scripts are executable
Execute it with the right amount of parameters

For reference, some build steps out of a Travis YAML file I’ve been working for NGenerics :

Bash


1
2
3
4
5


- mono ../tools/sonar/SonarQube.Scanner.MSBuild.exe begin /n:NGenerics /k:ngenerics-github /d:sonar.login=${SONAR_TOKEN} /d:sonar.host.url="https://sonarcloud.io" /d:sonar.cs.vstest.reportsPaths="*/TestResults/.trx" /v:"2.0"
- dotnet build
- dotnet test NGenerics.Tests --logger:trx
- dotnet test NGenerics.Examples --logger:trx
- mono ../tools/sonar/SonarQube.Scanner.MSBuild.exe end /d:sonar.login=${SONAR_TOKEN}

Notes:

The CLI scanner does not work for C# code at the time of writing. The MSBuild scanner works.
The MSBuild scanner requires Mono as the SonarQube team is still in the progress of migrating some dependencies.
${SONAR_TOKEN} is an environment variable injected by Travis that contains an authentication token. The VSTest reports come from NUnit tests via the “–logger:trx” parameter passed to dotnet run.
You need to pass the property sonar.login to the “end” step of the MSBuild scanner since that information is not persisted to disk when running “begin”.

Hope this saves someone some time - the feedback cycle is incredibly long as the Sonar scan tends to break on the last step.

Photo by Shane Colella on Unsplash

Still alive

Sun, 25 Dec 2016 00:00:00 +0000

I’m still alive. And so will this blog be once again.

It’s an interesting feeling to notice that your last post dates back to 4 years (4 YEARS!) ago.

I’ve done the migration from Octopress to vanilla Jekyll now and in the spirit of doing the smallest thing possible, this is the MVP.

Since my last post I have made the leap to Software Development Manager, and then to General Manager - Technology and Development at DStv Digital Media. It’s been an interesting journey and I have some scars to show and some stories to tell.

For now, let’s consider this is a kickstart of a habit before new years resolutions kick in.

Hold thumbs. Let’s see how this goes.

Photo by Josh Withers on Unsplash

Acceptance testing with SpecFlow and Selenium

Sun, 27 May 2012 00:00:00 +0000

I’m an avid believer in testing - TDD helps drive design, and having a test suite available to verify behaviour while maintaining an application is worth a million bucks. Even the most complete unit test suite still doesn’t guarantee that the integration between different components is correct, nor does it test the value a system delivers from a user perspective. Does a feature do what a user expects it to do? Does your application fulfil it’s functional and non-functional requirements? Does it even run?

Acceptance testing tests features as if a user was interacting with them. This means testing the system through the user interface, a system driver layer, or a combination of the two. This could be useful in a couple of ways:

As a smoke test for a deployed application.
As part of your continuous integration build.
As part of your deployment pipeline strategy.

Acceptance testing can be implemented as a form of BDD - in a user story we should be able to express the requirements in a format that we can use to write an executable specification .

I liked the following simple distinction between what BDD and TDD aims for:

TDD implements and verifies details - did I build the thing right? BDD specifies features and scenarios - did I build the right thing? Armando Fox and David Patterson, Engineering Long-Lasting Software

To enable BDD, RSpec does a beautiful job in setting up context and Cucumber guides you down the given when then path. Both (as well as other testing frameworks) can be used to write acceptance tests, but Cucumber makes sense if you have a business facing person looking at your specs. If not, writing your tests in English is a complete waste of time.

The .NET world does not have the same amount of tooling support for this kind of thing. This Stack Overflow question gives a nice summary on the current state of BDD libraries for .NET. For the xSpec (context / specification) flavour, we have NSpec , Machine.Specifications , and NSpecify . Because of the constraints of a statically typed language, the syntax of these libraries sucks. We could try and use RSpec with IronRuby , but it will add an extra, unneeded paradigm in a project.

For the xBehave family we are in luck - SpecFlow , is a BSD licensed library that allows us to write specifications like Cucumber in the popular Gherkin language on .NET. SpecFlow integrates well with Visual Studio, providing support for auto completion in feature files. Features compile down to NUnit tests, which can be run with your stock standard testing tool set.

I’ve written a small web application that allows you to search through an in-memory list of fruit, to test out the tool set:

Sample application

This is what my feature looks like:

Cucumber


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41


Feature: Search
 In order to avoid scanning through a long list
 As a user
 I want to be able to search for fruit by it's name


Background:

 Given I have the following fruit available:

 | Name |
 | Banana |
 | Pear |
 | Kiwi |
 | Watermelon |
 | Orange |
 | Naartjie |
 | Lemon |
 | Apple |

 Given I am on the "Home Page"


Scenario: Browse search page
 Then I should see "Search Fruit"

@search
Scenario: Search for a specific fruit
 When I search for "apple"
 Then I should see "Apple"

@search
Scenario: Search for a partial fruit name
 When I search for "on"
 Then I should see "Watermelon"
 And I should see "Lemon"

@search
Scenario: No search results
 When I search for "nonexistentfruit"
 Then I should see "No Results"

Behind the scenes, SpecFlow matches the Gherkin sentences to methods using regular expressions, which then drives the browser using Selenium backed by ChromeDriver for speed. For example, the following code drives navigation:

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


private static readonly Dictionary<string, string> paths = new Dictionary<string, string> { { "Home Page", "/" } };

private const string appUrl = "http://localhost:51553";

[Given(@"I am on the ""(.*)""")]
public void GivenIAmOnThe(string name)
{
 this.WhenIGoTo(name);
}

[When(@"I go to the ""(.*)""")]
public void WhenIGoTo(string name)
{
 string path;

 if (!paths.TryGetValue(name, out path))
 {
 ScenarioContext.Current.Pending();
 }
 else
 {
 SeleniumDriver.NavigateTo(appUrl + path);
 }
}

Note that if the path does not exist, yet the test is marked as pending and not failing - this allows us to write the specifications ahead of the actual code. We can of course also interact with the page:

C#


1
2
3
4
5


[When(@"I set ""(.*)"" to ""(.*)""")]
public void WhenISetField(string field, string text)
{
 SeleniumDriver.SetTextBoxValue(field, text);
}

Following Capybara , the code backing this will attempt to find the element by its id or the text of the label referencing the input element. This style of locating elements shows how we can write our tests from a user point of view. Another such example is finding text on a page:

C#


1
2
3
4
5
6


[Then(@"I should see ""(.*)""")]
public void ThenIShouldSee(string text)
{
 int textIndex = SeleniumDriver.PageSource.IndexOf(text, StringComparison.InvariantCulture);
 Assert.IsTrue(textIndex >= 0, "Could not find text \"" + text + "\" in the document.");
}

With these tests, keep them DRY by composing bigger steps out of little steps. This also helps us to stay in sync with the ubiquitous language :

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


[When(@"I search for ""(.*)""")]
public void WhenISearchFor(string what)
{
 steps.WhenISetField("Search For", what);
 this.WhenIClickSearch();
}

[When(@"I click ""Search""")]
public void WhenIClickSearch()
{
 SeleniumDriver.ClickButton("Search", driver => driver.FindSingleElement("searchHeading").IsVisible());
}

For any kind of test to be repeatable, we need to have full control over the environment. This might entail setting up data, or even mocking out certain components in the application. Notice the setting up of the dataset for all of the tests:

Cucumber


 1
 2
 3
 4
 5
 6
 7
 8
 9
10


Given I have the following fruit available:
 | Name |
 | Banana |
 | Pear |
 | Kiwi |
 | Watermelon |
 | Orange |
 | Naartjie |
 | Lemon |
 | Apple |

In this example I’ve used the fantastic Deleporter library to reach into the running application and change its’ internals. This trick comes in useful when mocking dependencies that are difficult to control, like external web services.

C#


1
2
3
4
5
6


[Given(@"I have the following fruit available:")]
public void GivenIHaveTheFollowingFruitAvailable(Table table)
{
 Fruit[] fruit = table.Rows.Select(x => new Fruit { Name = x["Name"] }).ToArray();
 Deleporter.Run(() => Fruit.SetFruit(fruit));
}

My initial impression of SpecFlow has been great. It’s also nice to bring a little of the Ruby tooling into my day to day .NET work . I’m now trying it out on a much bigger project - stay tuned for lessons learned.

Saving the environment with Vagrant

Wed, 23 May 2012 00:00:00 +0000

I’ve been playing with Vagrant , an easy way to set up virtual machines for development purposes. Vagrant provides a simple command line interface paired with a setup script (VagrantFile) in Ruby to provision virtual machines , share folders between the host and the virtual machine and to provide port forwarding . I’ve been using it to set up some Linux development environments on Windows, as well as for keeping my development machine clean, and running different versions of otherwise incompatible packages/gems.

To illustrate how easy Vagrant makes this process, let us set up a Solr development environment. After downloading and installing Vagrant , we can initialise a project with the init command :

Bash

1

vagrant init

This command creates a VagrantFile with some Ruby code in it that acts as configuration for your virtual machine. Vagrant builds a virtual machine from a base box . You can find a list of publicly available boxes here . For this machine, we can base it on the Ubuntu Lucid 32 bit image:

Ruby


1
2
3
4
5
6
7


# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant::Config.run do |config|
 config.vm.box = "lucid32"
 config.vm.box_url = "http://files.vagrantup.com/lucid32.box"
end

That’s all you need for a complete virtual machine. Running the up command will create and initialise it:

Bash


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


$ vagrant up

[default] Importing base box 'lucid32'...
[default] Matching MAC address for NAT networking...
[default] Clearing any previously set forwarded ports...
[default] Forwarding ports...
[default] -- 22 => 2222 (adapter 1)
[default] Creating shared folders metadata...
[default] Clearing any previously set network interfaces...
[default] Booting VM...
[default] Waiting for VM to boot. This can take a few minutes.
[default] VM booted and ready for use!
[default] Mounting shared folders...
[default] -- v-root: /vagrant

$

Vagrant imported the hard drives from an existing box in my profile - if you have not downloaded the lucid32 box before, it will happen on initialisation.

Note the port forwarding bit in the snippet above. Vagrant makes it easy to forward network traffic from a port on the host to one on the virtual machine - by default it will create a forwarding port 2222 to forward traffic to SSH port 22. On operating systems that come with SSH, Vagrant can open an SSH session to the box by using the ssh command:

Bash

1

vagrant ssh

Since SSH is not shipped with Windows out of the box, the ssh command is not available on that platform:

Bash


1
2
3
4
5
6
7
8


`vagrant ssh` isn't available on the Windows platform. You are still able
to SSH into the virtual machine if you get a Windows SSH client (such as
PuTTY). The authentication information is shown below:

Host: 127.0.0.1
Port: 2222
Username: vagrant
Private key: C:/Users/[user]/.vagrant.d/insecure_private_key

Windows users can use Putty (or any other SSH client) to connect to the virtual machine with the private key provided. Vagrant automatically shares the current folder between the host and the virtual machine:

Bash


1
2
3
4


vagrant@lucid32:~$ cd /vagrant/
vagrant@lucid32:/vagrant$ ls
Vagrantfile
vagrant@lucid32:/vagrant$

We now have a functional Ubuntu machine with sudo access to it, but we still need to customize it for our development purposes. Vagrant integrates well with Puppet and Chef to install software and to manage it. In the enterprise you could integrate this with your existing Puppet or Chef infrastructure, enabling you to create a clone of your production environment. In this example we’ll use Puppet to set up Solr running in a Tomcat container. We’ll create a directory called “modules” to hold pre-developed puppet modules, and a directory called “manifests” to hold our manifest. We need to point Vagrant to the manifest and the modules directory.

Ruby


1
2
3
4
5


config.vm.provision :puppet do |puppet|
 puppet.manifests_path = "manifests"
 puppet.manifest_file = "solr_tomcat.pp"
 puppet.module_path = "modules"
end

We need to ensure a package refresh (aka apt-get update) to install the solr-tomcat package - there’s a good example of how to do that here . After we’ve added the package dependency, our manifest file looks like this:

Puppet


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


class apt {
 exec { "apt-update":
 command => "/usr/bin/apt-get update"
 }

 # Ensure apt is setup before running apt-get update
 Apt::Key <| |> -> Exec["apt-update"]
 Apt::Source <| |> -> Exec["apt-update"]

 # Ensure apt-get update has been run before installing any packages
 Exec["apt-update"] -> Package <| |>
}

package { 'solr-tomcat':
 ensure => present,
}

vagrant reload will reboot our machine and apply the configuration, and we should have Tomcat installed with Solr running on top of it. By adding the following line to our VagrantFile, we can get Vagrant to forward all communication from our local port 8181 on the host to the VM port 8080 where Tomcat is listening:

Ruby

1

config.vm.forward_port 8080, 8181

When you go to http://localhost:8181/solr/ in your local browser, you should now see Solr up and running, congratulations! Now all that’s left is to share some files between the guest and host operating systems, specifically the configuration files in /etc/solr/conf. Vagrant shares the project folder with the guest OS by default under the directory /vagrant. Let’s start by copying the files we want to the host machine.

Bash


1
2
3


cd /etc/solr/conf/
mkdir /vagrant/solr
cp -R * /vagrant/solr

Once we have the files locally, we can mount this local directory over the one in the VM by adding this line to the VagrantFile :

Ruby

1

config.vm.share_folder "solr", "/etc/solr/conf", "solr"

You can reload the virtual machine by running vagrant reload. You are now able to do some Solr development while maintaining a clean running environment for your indexes. Best of all, you should now be able to reproduce the environment reliably, so you can feel free to experiment.

Photo by Katy Belcher on Unsplash

Now blogging on Octopress

Sat, 05 May 2012 00:00:00 +0000

A week ago, I wrote :

Sigh, I can’t seem to make up my mind about the platform to host my blog on. You can find my new blog over at <www.riaanhanekom.com>.

Will post the details soon over there. This blog will self-destruct in a week or so.

My first blog was at Blogger , back when Google just bought it. What pained me then, was the lack of control - I couldn’t customize it to do what I wanted it too. After that, I tried Wordpress , where I was happy for some time but the workflow of editing posts in html or via the rich text editor didn’t work out - I realised that I was spending more time formatting posts (images, code, etc.) then writing them.

This drove me to Posterous , where I could write in Markdown , have gist support, and post by email . This helped me to blog more regularly since it lowered the friction in posting considerably. There were some bugs here and there, but nothing too dramatic. Over time, I was more and more convinced that I should move again due to:

Death by a thousand paper cuts - little features that worked in other blogging platforms were not functional (or plain non-existent) in Posterous.
The acquisition of Posterous by Twitter puts the platform at risk of stagnating.
Lack of control (again).

I’ve decided to host my own instance of a blogging platform that I can control. This blog is now running on Octopress , the “blogging framework for hackers”, and I’ve never been happier.

It features :

Built on Jekyll , the blog aware static site generator. There is nothing dynamic about the content served - these pages have been pre-rendered as static html files. This makes it fast and easy to scale.
An awesome starter template that looks well on mobile and desktop. Easy to customize, override, and replace.
Supports Markdown and Textile . Personally I prefer writing in Markdown, but Textile is also a viable alternative to HTML.
A bunch of plugins , and Liquid templating support.
A collection of custom Rake tasks to help with previewing, generating and creating content.

Revision control support comes out of the box with git once you clone the Octopress repository . This is useful in versioning and restoring posts (and the entire site, including styles and plugins), but also serves as a backup in case of emergency .

For writing posts, I now have a much more minimalistic toolset. I use Vim , with the Janus pack to provide markdown support and much more. To test rendering of Markdown, I use vim-markdown-preview - a simple [:Mm] command brings up a browser with the rendered html.

When pushing to my remote git repository a git hook clones the repository, generates the site, and copies the static files to where it gets served from - rudimentary CI for blogging.

Octopress is truly the blogging platform for hackers. I think I’ll stick with it for a while.

Photo by Thomas Lefebvre on Unsplash

Moving to Git

Sun, 25 Mar 2012 00:00:00 +0000

Git , the distributed source control system is fast becoming the de facto standard, at least in the open source arena. Although it comes with a bit of a learning curve, the advantages of using Git over more traditional SCMs like Subversion and TFS far outweighs the investment of time necessary to learn it.

TL;DR

A quick introduction to Git, with a basic command reference. If you are familiar with Git, you probably won’t learn anything new. If not, please continue reading.

The Pros and cons

Why switch to Git?

It’s fast. Repository history is stored locally, so we can diff and traverse history at a whim without having to involve the server.
Cheap local branches. Ever wanted to branch per feature? Git is your friend.
Merges are almost magical. Git does a much better job of merging than any other SCM I’ve seen.
Since commits are done locally, you can commit as often as you like. Because history for your local repository can be rewritten, commits become more of a checkpoint than something permanent.
Awesome command line flow. Although Git has plenty of commands available, you need a couple to get the job done on a daily basis. Git integrates well with various command line and GUI tools, and you can jump to the GUI for more complex operations like merging.

The downside to Git is with binary assets - if you store and change binary assets in your repository, it will grow to an unwieldy size. An alternative way to manage binary files with git is using git-annex . See this Stack Overflow question for details.

Getting started

On Linux operating systems, use your favourite package manager to install the git client. On Windows, the easiest way to get Git is to install GitExtensions , which installs msysgit , kdiff and Putty or openssh . GitExtensions also includes Visual Studio integration which is useful every now and then.

Git on Windows runs in Git bash, which is bash ported to Windows. For something that feels more native to the Windows platform, PoshGit allows for functional usage of Git inside a PowerShell shell. TortoiseGit (based on the popular TortoiseSvn ) integrates Git functions into Windows Explorer, allowing you to drive Git in a more graphical way. For browsing the history of a repository, gitk on Windows (should come installed with Git Extensions), and tig on Debian based systems can provide a graphical way of performing this task.

Avoiding the command line and relying on the GUI for driving Git makes you miss out on the cross-platform experience, and means that you will need to drop down to an uncomfortable place for some of the more advanced operations, so I highly recommend starting out at the command line and using the GUI pieces for convenience where it makes sense.

Where to get help

Git has an awesome built-in help system, accessed by calling

git –help

You can get help on specific commands by adding the –help parameter after the command :

git add –help

Basic commands

At its most basic Git works like more traditional SCMs:

git pull

Pulls down the latest version of the source code from the remote repository, and merges it into the current working set.

git push

Pushes (commits) the locally performed commits to the remote repository. After it’s been pushed, other team members can pull your changes.

git add -A git commit -m “My detailed commit message.”

Git has the concept of a staging area , which is a holding area for the changes that you would like to commit. The add command adds either a specific file to the staging area, or all detected changes with the -A parameter.

The commit command performs a local commit, while the -m parameter specifies a message for the commit. If the -m parameter is not provided, Git will open up a text editor for you to edit the commit message.

Should a pull or a push result in merge conflicts, you can start a manual merge process by running the following command :

git mergetool

To view the history of the repository, you can query the log via

git log

The commands above is all you need to get started with Git - once you get those under your belt, the true power of this distributed source control provider can kick in with some more advanced commands.

More advanced usage

One of the features I like most about Git is its cheap, local branches. With these, there is no reason not to branch for even the simplest change. To create a new branch and switch to it, run the following :

git checkout -b “newbranch”

The -b parameter lets Git know to create a new branch, with name newbranch. Change and commit as normal, and when you are ready to merge back into the master (trunk in traditional terms), you can do the following to merge the changes in :

git checkout master git merge newbranch

You can then delete the branch if not needed any more:

git branch -d newbranch

Because the repository is local, switching branches is fast and painless - unlike more traditional SCM’s that need to talk to the server to achieve this.

Git does not like having uncommitted changes lying around between operations like branching and merging. Git provides a “stashing” mechanism where it temporarily stores away changes that you can reapply later:

git stash save “stashname”

Saves the current changes into a stash with the specified name, and check outs the most recent commit on that branch. You can run

git stash apply

To apply these changes to the file set again.

Rebase is worthy of its own post - there is so much power in being able to rewrite history.

Using Git With Other Source Control Systems

If for some reason, you can’t use Git on the server, you might still be able to use it on the client. Git works well with other source control systems like TFS , and Subversion through the use of plugins. Git integrates with these SCM’s by “cloning” the remote repository locally, and then pushing local changes back to the remote servers when needed.

For TFS support, after installing git-tfs , you’ll have access to the following commands :

git tfs clone http://tfs:8080/tfs/collection $/project_name

Clones the project with the specified name contained in that collection.

git tfs checkintool

Brings up the check in tool with the changes available to commit to TFS.

git tfs shelve name

Creates a shelve set with the changes since the last commit to TFS.

Integrating with Subversion works in a similar way - with the git-svn package you can perform the following operations :

git svn clone [repositoryurl]

Creates a “cloned” local Git repository based of the remote Subversion repository. In the local repository, Subversion commits become Git commits.

git svn rebase

Performs an update, and rebases your change sets on top of it.

git svn dcommit

Takes your Git commits and streams your changes to the Subversion server.

Tips and tricks

When I started out with Git, it took me a while to realize that some of the situations I ran into resulted from me not adhering to the git workflow . I recommend reading up on it before you do any serious work with Git.

If you have a complex release strategy, it will be worth your time to familiarize yourself with git-flow , and the successful branching model behind it.

Also learn what git pull does, and read up about why git fetch works better .

Making learning fun

Git, being a power tool, is feature rich. For encouragement on exploring all it’s features, and to make the learning process fun I recommend these two tools:

git-achievements tracks your usage of features, and unlocks “achievements”, much like Stack Overflow badges.
githug (Linux and macOS) provides a game like environment where you perform specific operations to get to the next level. Levels vary in difficulty, so this is a great way of exercising your Git chops.

How Small is Small Enough

Mon, 27 Feb 2012 00:00:00 +0000

As developers, we are good at breaking up components into sub-components - sometimes excessively so. When confronted with a larger than average chunk of work, we make a choice: either consider the work as indivisible to be delivered in its entirety, or break it up smaller pieces. Most of us already experience a sense of foreboding when confronted with a piece of work that is too large. Whether you are estimating in hours or in story points the question is: when breaking up work items into smaller pieces - how small is small enough?

Delivering software, one piece at a time

By now, we know that the best way to write software is to evolve it bit by bit. Doing work in small chunks allows us to

free our minds of details that are not relevant to the here and now
make small, certain steps towards a goal
measure progress as soon as it happens
get feedback as soon as possible

Too much of a good thing can also be bad for you - too few small tasks could cause the overhead of managing these tasks to become significant in itself, and measuring progress could become a nightmare. Also, when estimating on fine-grained (low-level) tasks, local safety becomes a problem.

How small is small enough?

One way of determining the appropriate size for a piece of work could be to link it to the notion of testability - the smallest piece of functionality that could be finished and pushed through the delivery pipeline. This approach produces too fine-grained work items, linked to their technical implementation.

Rather, a story should represent a single piece of deliverable business value :

The rule here is to make the story as small as possible, yet that it retains some measurable amount of intrinsic value - something that, when demonstrated to the end users, they’ll agree is an improvement to the system.

We can test work item size by looking at the value delivered:

Conversely, if you break a story into two pieces, but there is no value to be had until both pieces are completed, then that is too small. The stories should be recombined - or split another way.

Optimizing the flow

Consider also the idea of single piece flow. In manufacturing terms :

Single piece flow is the ideal state where parts are manufactured one at a time, and flow throughout the manufacturing and supply chain as a single unit, transferred as customers order.

Manufacturing large batches of parts simultaneously, or accumulating parts in a bin for shipping or transferring 2 or more parts at the same time is opposite or contrasted to the definition of Single Piece Flow.

The idea is that large batches of work (the waterfall way ) equate to rework (waste), inventory (also considered waste), and delayed delivery time (loss of profit). There are times when work items should not be split up into smaller pieces. Since the existence of software is solely linked to profit (businesses do want to make money, don’t they?), it makes sense to optimize for flow and profitability by

sharing setup costs, that is, group items where setup cost can be shared, and
maximizing flow by minimizing batch size.

When breaking work into chunks, you must strike a careful balance between the two.

Photo by Céline Haeberly on Unsplash

The Humble Story Point

Mon, 13 Feb 2012 00:00:00 +0000

I’ve had some interesting discussions on the management of work through user stories. A lot of teams, those starting to use agile techniques, seem to have some uncertainty around common topics:

The theory behind story points and why they are preferred over estimations in hours
Why story points and velocity are self-correcting measures
The sizing process and appropriate sizes for stories

The problems with sizing in units of time

I’ve had the opportunity of working on some waterfall projects. I feel blessed to have been in these teams because I now recognize that:

Most estimations in hours are thumb-sucked. Given that software development is a complex beast, how can we possibly forecast time to be spent on a specific task in an accurate fashion?
Work break down structures are frequently used to aid in estimations, even though they exasperate inaccuracies of estimation at lower levels.
As teams typically contain members at different skill levels, how long a task takes depends on which members of the team actually do the work. An estimate based on time does not make sense in this context.
Traditional project management focuses on getting projects on time, sometimes sending teams on a death march and severely compromising on quality.
Waterfall based software teams are held to inaccurate estimations. The Gantt chart proves to be inflexible as the focus of the traditional project manager is on staying true to the original plan.

It should be obvious that I’m not a big believer in time-based estimations. Apart from the above, one should also watch out for the problem of local safety :

If asked how long a small task will take, a developer will naturally not want to deliver it late. After all, this is a small task, it should be on time or the developer wouldn’t be professional. Hence, the estimate will include a buffer to absorb the developer’s uncertainty about the task.

The estimate given by the developer will depend on how confident that developer is that he has understood the task and knows how to perform it. If there is uncertainty in the understanding of the task, then the estimated time to completion is likely to be significantly longer. No one wants to be late with such a small task.

The Agile Management for Software Engineering book also describes the need for a “relative” estimation method :

Accurate project planning techniques must avoid asking developers to estimate small tasks based on effort. The tasks to be performed must be broken out based on some form of (relative) complexity and risk. To work best, this requires a codification scheme for assessing the complexity and risk. An analysis method that enables small pieces of client-valued functionality to be codified for complexity or risk is required. This enables the inventory in the system of software production to be estimated without the psychological effect of local safety.

We have an alternate estimation method available in the form of story points. Jeff Sutherland explains why story points are better than hourly estimates :

The way we do story point estimation gives better estimates than hourly estimates as they are more accurate and have less variation. A CMMI Level 5 company determined that story point estimation cuts estimation time by 80% allowing teams to do more estimation and tracking than a typical waterfall team. A telecoms company noticed that estimated story points with planning poker was 48 times faster than waterfall estimation practices in the company and gave as good or better estimates.

Story points are therefore faster, better, and cheaper than hours and the highest performing teams completely abandon any hourly estimation as they view it as waste that just slows them down.

Measures that work for the team

Given that story points are better than time-based measures, what is the unit of a story point? The most common sizing measures are T-Shirt sizing (S, M, L, XL), and the Fibonacci series (1, 2, 3, 5, 8, 13, 21, 34).

I prefer having less numbers to choose from since we can complete sizing faster by eliminating debate on the exact size of a story. My vote goes to the limited Fibonacci sequence (1, 2, 3, 5, 8), also in use by one of my favourite online tracking tools . I’m not impartial to the other estimation units though - even a simple scale of (1, 2, 3) could be of use to a team.

It’s all about averages

Points are about effort, not complexity . A team’s velocity is the amount of points completed in a sprint. Based on the velocity of the team in a previous sprint, we can forecast the amount of work the team should be able to do in a following sprint with reasonable certainty.

Velocity and story points are based on the environment and members of the team. A new addition to the team, for example, might change the definition of a story point, and will have a direct effect on the team’s velocity. An awesome property of velocity and story points though, is that they are self-correcting measures. Since the team’s workload is derived from them, it will also adjust accordingly to any change in the team or environment. Wrong estimates also tend to average out - while we underestimate some stories, we also overestimate others.

Estimating in story points

With story points being an arbitrary measurement, how do we estimate in a consistent manner? There is no hard answer to this, except that you get a “feel” for the effort represented by a story point. The definition of a story point is unique per team, and it will take a while before a team synchronizes on the magnitude of a story point.

For teams new to story points, Chris Sims details an approach to sizing by sorting stories according to size that will make it easy to get started.

One can also try creating a personalised relative scale for points, with reference to previous/familiar work. For example, the following scale might be used to size stories :

Adding a button to a page
A form with a single text box, required validation.
A form with various inputs, required validation, saved to the database.

And so forth. Craig Heckrath introduced me to this technique, and it seems to work well in teams new to agile.

More experienced teams tend to play planning poker since it

limits discussion and gets stories sized faster
keeps everyone in the team contributing to the discussion
is fun!

Complexity in Software

Sun, 29 Jan 2012 00:00:00 +0000

In a discussion with a former colleague of mine on the organization of components and on system boundaries, we focused on the complexity inherent to software building. It hit me that we can learn from physics here.

The law

The first law of thermodynamics states that

energy can be transformed, i.e. changed from one form to another, but cannot be created or destroyed.

This law, in my mind, can be applied to software development broadly:

Complexity in software can be transformed, i.e. changed from one form to another, but cannot be created or destroyed.

I was about to make this law my own, but on writing this I found that another named this back in 2003 . Matt’s first law of software complexity states :

The underlying complexity of a given problem is constant. It can be hidden, but it does not go away.

Or:

Complexity is conserved by abstractions. In fact, apparent complexity can be increased by abstractions, but the underlying complexity can never be reduced.

Why manage complexity?

The way we construct software is influenced on all levels by complexity. Yes, user requirements, even functional requirements drive us towards a certain design when building software - but any requirement introduces further complexity that we need to manage. A software developer’s primary role is to manage complexity to provide anything useful.

In short, complexity

makes software more difficult to write
makes software more difficult to maintain
makes it almost impossible to communicate intent to a fellow programmer (or the next)
makes it progressively more difficult to construct larger components out of smaller ones.

What is complexity in software development?

When looking at the complexity problem, it might be useful to identify categories of complexity when looking at components. I’m veering away from the academic classifications here to make this discussion more practical.

For any working component in a system, that is, a subsection of a software application that is useful, we might look at

Internal complexity : complexity contained by the component, either caused by the implementation of the component itself or inherited by its integration with subcomponents.
External complexity : complexity leaked outside the component’s interface. Other components will inherit this complexity when interfacing with or containing this component.

When looking at a system (a component in itself), we can also identify

Local complexity : the complexity in understanding a specific component/piece of functionality
Global complexity : the complexity in understanding the system in its entirety - all the interactions between the different components within.

Techniques for managing complexity

Since software development has a lot to do with managing complexity, various techniques have sprung up over the years to help manage it. Too many techniques exist to list here as we could argue that even the move to higher-level languages had complexity as its primary motivation.

A helper in managing complexity in recent years has been Object Orientation (OO) , with the concept of encapsulation stopping the spread of complexity outside a component. Internally and externally, OO has taught us to reduce coupling , and maximize cohesion . These two principles are the underpinning of most (all?) complexity management techniques in this paradigm.

Related to encapsulation is the idea of service boundaries and a bounded context . Service boundaries cut off complexity at a specific point, and provide a simple, well-thought out contract to interact with a module, thereby reducing the surface area of interactions with it and hiding the complexity within a module. A bounded context helps identify models to keep separate to avoid complexity.

A more technical technique to reduce complexity in a codebase is to apply the Inversion of Control / Dependency Injection pattern/technique. It promotes simplicity in classes, and reduces coupling between components. The former pushes down local complexity - we can zoom in and understand a single component, change it, or replace it - but the latter causes an increase in global complexity since the system is more difficult to understand in it’s entirety, and it becomes more difficult to identify all the execution paths in a system.

If this technique pushes up one category of complexity, why use it? My theory is that global complexity favours the maintenance phase over the development phase. In the development phase the focus is on the creation of code - major components are designed and created, and we are expected to have a clear vision of the system in its whole (we have to deliver this thing, don’t we?). In the maintenance phase, we generally make smaller changes, and the focus is on “zooming in” to an appropriate level to fix, refurbish, or replace components to serve a new purpose. DI makes sense since the maintenance phase accounts for most of the effort put into a software product.

Other techniques for managing complexity

In Matt’s post he points out an underused strategy - if the problem is complex, it might be possible to redefine the problem and sidestep that complexity entirely.

Do the simplest thing possible . Your customers will love you for it.

Greener pastures

Fri, 13 Jan 2012 00:00:00 +0000

This post is more than two months late, but I’ve been at DStv Online since November 2011.

Intervate has been great to me, and is still on my recommended list of employers. I experienced significant growth there, and I thank them for that - but with consideration of some quality of life issues I’ve been having, and a need to stretch my skill set, I’ve decided to move on.

So far, I’ve found my new environment stimulating and challenging. I’ve spent most of my career in finance (aka banks) - the media industry is new to me so I’m learning a lot there. The people are eager to experiment and learn on both technical and process levels, and I can say that I have never felt at home so fast at any of my previous jobs. This video shows the kind of environment they have.

Some perks :

Exciting, pioneering work. Not (as a friend put it) writing Excel over and over again.
Public facing sites. You know that “What do you do for a living?” question? I have an answer for people who are not complete geeks. I can show them.
I work close to my house. No highway travelling. Period. No sitting in traffic. I might even downgrade to one of these soon.

Nothing like some change.

Photo by Fré Sonneveld on Unsplash

Blog moved

Sat, 17 Dec 2011 00:00:00 +0000

In preparation for my recurring new years resolution of updating this blog, I’ve followed the advice of Simon Cropp and moved my old blog to Posterous .

If you subscribe to my FeedBurner feed , there should be no changes necessary from your side to receive updates from me. Please update any other bookmarks to my site.

Wordpress has served me well, but has become a bit of an antiquity when compared to some of the modern blogging platforms. The features Posterous provides are encouraging.

Here’s to a new year of fresh content!

Photo by Nathan Anderson on Unsplash

Feature request for SQL Server Management Studio

Thu, 30 Sep 2010 00:00:00 +0000

A “You sure you want to do this?” confirmation when you attempt to run a query that contains a destructive update (delete/update) without a where clause. With the hoards of database tables getting wiped in the world, why hasn’t this been implemented yet? This feature would remove a lot of fear of pressing F5.

Those who are not afraid running a delete query on a production database haven’t lived long enough yet. On a similar note, those who trust their code still need to experience taking down a couple of servers with it.

Photo by Caspar Rubin on Unsplash

Just Finished Reading (Lean Edition)

Sun, 11 Jul 2010 00:00:00 +0000

Lean Software Development : An Agile Toolkit, Mary Poppendieck and Tom Poppendieck . A great resource on how the lean principles from product manufacturing can be applied unto software development. You can find a good overview in their 2002 paper here (pdf) and the Wikipedia article on Lean .

Overview

Each chapter in the book describes “thinking tools” for the 7 Lean Principles .

Eliminate waste describes how to identity and remove waste to maximize the flow of the value stream.
- Seeing Waste
- Value Stream Mapping
Amplify learning emphasizes how short feedback cycles can improve the resulting product and development process.
- Feedback
- Iterations
- Synchronization
- Set-based development
Decide as late as possible to deal with uncertainty, and make better decisions.
- Options thinking
- The last responsible moment
- Decision making
Deliver as fast as possible to enable late decision making, and produce value as fast as we can.
- Pull systems
- Queuing theory
- Cost of delay
Empower the team to make use of the expertise that lies with the people that do the work.
- Self-determination
- Motivation
- Leadership
- Expertise
Build integrity in to accommodate change and increase the period of usefulness of a product.
- Perceived integrity
- Conceptual integrity
- Refactoring
- Testing
See the whole warns against sub-optimization, and provides tools for keeping the bigger picture in mind.
- Measurements
- Contracts

Books on process take me longer to get through than those dealing with technical topics – with the density of information in some of the chapters, this one was no exception. It’s a real eye-opener though and comes highly recommended for those wishing to improve their software development process, even for those in non-agile environments.

What to read if you are in a hurry

The part on Queuing Theory is informative and captures the substance of the book well. More on this later.

Photo by Aliis Sinisalu on Unsplash

Dependency injection - the what and the how

Sun, 04 Jul 2010 00:00:00 +0000

Background: We’ve started a weekly patterns & practices meeting at work with some of our senior developers where our discussions and actions will hopefully bring some improvement to the current development environment. Once a week one of us has an opportunity to showcase a new topic – much akin to knowledge transfer session but more fine-grained and at a higher level than the technology. Gareth Stephenson suggested we blog about the content for the benefit of others, which I think is a good idea. Furthermore, I’ve never been able to find a good, down-to-earth resource that explains the benefits of DI and how to get started with it. This post is a recording of a short presentation I performed on DI and the patterns that emerge in its usage – I hope it proves useful.

.NET Developers don’t do patterns

A surprising number of .NET developers are not familiar with Dependency Injection (DI) , and patterns in general. In the Java world, File –> New Project would give you a template with Spring baked in – when working with .NET that kind of guidance is missing from the toolset. Perhaps some exposure to MEF with .NET 4.0 will entice .NET developers to look further than the Microsoft toolset, in the same way that Entity Framework exposed most developers to ORM ’s with NHibernate getting some adoption as a result. DI and Inversion of Control go hand-in-hand, but be careful not to confuse the two – DI is a tool, whereas Inversion of Control is a pattern.

A World filled with legacy code

We’ll get to why we would use a container soon, but consider the following code in the meanwhile:

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


public class OrderPlacementService {
 public void Process(Order order) {
 // For a big order, notify somebody important
 if (order.Amount > 500) {
 SmsSender.SendSms("0113334444", "Pretty big order received.");
 }

 // Actually process the order
 Console.WriteLine("Order processed.");
 }
}

public static class SmsSender
{
 public static void SendSms(string number, string text)
 {
 Console.WriteLine("Sent sms \"{0}\" to number \"{1}\".", text, number);
 }
}

The important parts that denote processing represent themselves as writes to standard output. This code is typical of legacy systems, and even some systems currently under development. It suffers from the following problems related to coupling and cohesion :

Static classes and calls to static members increase the coupling between classes and makes the code harder to change, and reduces testability .
Violation of the Open/Closed Principle (pdf ) forces us to change the service when we want to change the notification functionality.
Violation of the Single Responsibility Principle (pdf ) by handling order processing, business rules and notifications makes the service more fragile by introducing reasons for change.
Calls to concrete classes and violation of the Dependency Inversion principle (pdf ) maximizes coupling between completely unrelated pieces of code.
Resembles procedural programming making it difficult to plug code in and out without affecting anything else.
The above makes the code inherently untestable – we cannot test the code without sending an SMS. Keeping in mind that we would wish to run our tests as frequently as possible, a test that costs money is not the best idea.

Using a DI framework changes the way you code by solving the problem of constructing objects and the provision of dependencies – it makes it possible to decouple your implementations from one another, thereby improving your code by making it more amendable to testing and more amendable to change. Before we introduce a DI framework to the sample code above, we need to make some improvements to it.

Improving the existing code

We start off by removing any static calls by making SendSms an instance member. This will allow us to replace SmsSender with a test object (test double, mock, or a stub ) to test it.

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


public interface ISmsSender
{
 void SendSms(string number, string text);
}

public class SmsSender : ISmsSender
{
 public void SendSms(string number, string text)
 {
 Console.WriteLine("Sent sms \"{0}\" to number \"{1}\".", text, number);
 }
}

We can then write the OrderService class in a cleaner way to reduce the dependency on a concrete implementation of an SMS sender:

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


public class OrderPlacementService {
 private readonly ISmsSender smsSender;

 public OrderPlacementService(ISmsSender smsSender)
 {
 this.smsSender = smsSender;
 }

 public void Process(Order order) {
 // For a big order, notify somebody important
 if (order.Amount > 500)
 {
 smsSender.SendSms("0113334444", "Pretty big order received.");
 }

 // Actually process the order
 Console.WriteLine("Order processed.");
 }
}

Notice the effects this simple change has:

We’re using inversion of control to “push” the dependency to the caller, thus relieving ourselves of responsibility of it.
Our OrderPlacementService now explicitly states the dependencies that it needs to function properly by requiring an ISmsSender as a constructor parameter.
Since we’re not interested in the concrete implementation of an SmsSender, we’ve replaced the concrete dependency with an interface. We care about the What (contract), not the Who (concrete type) or the How (actual implementation details).

A Sample implementation

The calling code must now provide the dependency to the OrderService. For example :

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


static void Main(string[] args)
{
 var orderPlacementService = new OrderPlacementService(new SmsSender());

 Console.WriteLine("Placing small order...");
 orderPlacementService.Process(new Order { Amount = 200 });

 Console.WriteLine("Placing big order...");
 orderPlacementService.Process(new Order { Amount = 700 });

 Console.WriteLine();
 Console.WriteLine("Done.");
 Console.ReadLine();
}

Running the code above provides the following output:

Providing extension points

We can improve the code even more by abstracting at a slightly higher level, and removing the knowledge of the order amount condition (greater than 500) and the resulting action (sending an SMS) from the service by introducing an extension point and moving the logic to a custom extension :

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31


public interface IInterceptor<T>
{
 void Intercept(T item);
}

public interface IOrderInterceptor : IInterceptor<Order>
{
}

public class LargeOrderInterceptor : IOrderInterceptor
{
 private readonly ISmsSender smsSender;

 public LargeOrderInterceptor()
 : this(new SmsSender())
 {
 }

 public LargeOrderInterceptor(ISmsSender smsSender)
 {
 this.smsSender = smsSender;
 }

 public void Intercept(Order order)
 {
 if (order.Amount > 500)
 {
 smsSender.SendSms("0113334444", "Pretty big order received.");
 }
 }
}

With the interceptor interface in place, our code for our OrderService is greatly simplified :

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


public class OrderPlacementService : IOrderPlacementService {
 private readonly IOrderInterceptor orderInterceptor;

 public OrderPlacementService(IOrderInterceptor orderInterceptor)
 {
 this.orderInterceptor = orderInterceptor;
 }

 public void Process(Order order)
 {
 orderInterceptor.Intercept(order);

 // Actually process the order
 Console.WriteLine("Order processed.");
 }
}

Enough dependencies to go around

Our calling program needs to provide the appropriate dependencies :

C#

1

var orderPlacementService = new OrderPlacementService(new LargeOrderInterceptor(new SmsSender()));

and now another problem emerges. Firstly, it’s a real pain to have to provide the dependencies for OrderPlacementService (and it’s dependencies) by hand. The bigger problem here is that we’ll be violating DRY if the service gets used in more than one place, which it’s bound to be. Coupling increases since every object that has a dependency on OrderPlacementService would have a direct dependency on the LargeOrderInterceptor and SmsSender classes, which denotes a violation of the Single Responsibility Principle. The classic solution for this problem is to make use of the Factory pattern in order for this knowledge to be in one place (and thus changed with ease if the need arises):

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


public interface IServiceFactory
{
 IOrderPlacementService GetOrderPlacementService();
}

public class ServiceFactory : IServiceFactory
{
 public IOrderPlacementService GetOrderPlacementService()
 {
 return new OrderPlacementService(new LargeOrderInterceptor(new SmsSender()));
 }
}

##Factories Are Not The Solution

Using the Factory pattern in this case has it’s own sets of problems :

Dependency Replacement : While we got rid of the dependency on LargeOrderInterceptor and SmsSender, we’ve replaced it with a dependency on ServiceFactory. ServiceFactory then becomes difficult to maintain as any changes to it affects a big chunk of the codebase.
Proliferation of helper classes : Splitting the ServiceFactory into smaller factories, each responsible for constructing a different object removes the hotspot, but brings to life a multitude of redundant helper classes (classes that have no real identity or well-defined purpose).

Another technique called Poor Man’s Dependency Injection can reduce the spread of dependency knowledge by one level by creating a default constructor that injects the dependencies into a more specialized constructor. Applying this technique to the LargeOrderInterceptor would yield the following constructors :

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


private readonly ISmsSender smsSender;

public LargeOrderInterceptor() : this(new SmsSender())
{

}

public LargeOrderInterceptor(ISmsSender smsSender)
{
 this.smsSender = smsSender;
}

Poor Man’s Dependency Injection also isn’t helpful in this regard, but it may be useful in constrained environments to increase the testability of code.

Dependency injection to the rescue

This is the situation where DI is helpful – it takes on the knowledge of construction and stores it in a central place, removing duplication and making it easy to change. It provides the glue that holds an application together and allows us to create small classes isolated from each other, but working together to achieve a common goal. You can think of a container as a simple hash table of types with interfaces mapping to concrete types. If we ask the container for a type implementing IFoo, it can construct and return a Foo concrete instance. If Foo needs any dependencies, the container will construct and inject those as well. It will construct an entire object graph without your code having any knowledge on the concrete types or the dependencies they take.

We’ll use StructureMap (one of many containers available for .NET) to illustrate. First, we need to add some code to initialize the container. Typically, this bootstrapping code gets run whenever your application starts up in some executable bootstrapper. Initialization of the container should add set up the container for construction of any objects that we might need during the execution of our program. StructureMap’s fluent API makes this an easy task :

C#


1
2
3
4
5
6


StructureMap.ObjectFactory.Initialize( x =>
{
 x.For<ISmsSender>().Use<SmsSender>();
 x.For<IOrderInterceptor>().Use<LargeOrderInterceptor>();
 x.For<IOrderPlacementService>().Use<OrderPlacementService>();
});

The code above is registering a type with the container. Now, whenever we need an OrderPlacementService, we ask the container to construct it for us using the Service Locator pattern :

C#

1

StructureMap.ObjectFactory.GetInstance<IOrderPlacementService>();

And magic! We have an usable IOrderPlacementService:.

In the background StructureMap matched the requested interface to a concrete type, found the greediest constructor (the one that has the most parameters), and injected the IOrderInterceptor dependency into it. The same happened to the ISmsSender dependency for the interceptor. Our code just ran without any compile time knowledge on concrete implementations for its dependencies. We achieved both coding to interface (ignorance on concrete implementations) that reduces coupling, and solved the problem of object construction without being inappropriately intimate with them.

Configuring all your dependencies can cause blindness

Coding using all the practices discussed earlier can result in a lot of types (and registrations) in big projects - having to update the container configuration every time you add a class can be a real pain. To counter this, most container implementations provide a way of Auto-Registration - scanning types in assemblies and registering the m automatically using certain conventions.

With StructureMap, we can instruct the container to scan on initialization :

C#


1
2
3
4
5
6


ObjectFactory.Initialize( x => x.Scan(scanner =>
{
 scanner.AssembliesFromPath(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location));
 scanner.WithDefaultConventions();
 scanner.LookForRegistries();
}));

This little snippet above has the following effects :

Let StructureMap scan all the assemblies in the application’s home directory.
Register types that match the default conventions – that is, if StructureMap finds an interface IFoo and a concrete implementation called Foo, it will automatically register the types. Note that we can be customize these conventions.
LookForRegistries finds and adds the information contained in registries . Registries allow us to split the registration configuration in scope (for example on a namespace or assembly level).

Note that scanning, in our case, removed the need for us to explicitly register ISmsSender and IOrderService since the naming of these classes match the default conventions. Our interceptor is not so lucky, and has to be manually registered using a registry :

C#


1
2
3
4
5
6
7


public class DomainRegistry : Registry
{
 public DomainRegistry()
 {
 For<IOrderInterceptor>().Use<LargeOrderInterceptor>();
 }
}

## Containers Accommodate Change

We might realise that we have the need for more interceptors to run whenever an order comes in. We can solve this problem by using the Composite pattern , which allows us to treat a collection of items as a single item in the following way :

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


public interface ICompositeOrderInterceptor : IInterceptor<order>
{
 IList<IOrderInterceptor> Items { get; }
}

public class CompositeOrderInterceptor : ICompositeOrderInterceptor
{
 public CompositeOrderInterceptor(params IOrderInterceptor[] items)
 {
 Items = items;
 }

 public void Intercept(Order order)
 {
 foreach (var interceptor in Items)
 {
 interceptor.Intercept(order);
 }
 }

 public IList<IOrderInterceptor> Items { get; private set; }
}

With this in place, we can construct our container to add all instances of IOrderInterceptor and inject the composite wherever it’s used. Their are two distinct ways of registering types in an container. Which one to use depends on the usage and number of items of the individual types:

We can request the default implementation for an interface using the Service Locator pattern. Asking for an IFoo implementation will always result in a Foo instance with no further information.
We can request a list of implementations when multiple concrete implementations back an interface. A request for IFoo implementations will result in a list of instances of every type that implements IFoo.

Adding

C#

1

scanner.AddAllTypesOf<IOrderInterceptor>();

to our scanning and removing the default registration of LargeOrderInterceptor will register all instances implementing IOrderInterceptor. As a result we’ll be able to ask the container for all instances IOrderInterceptor and get back a list of instances of all the concrete implementations found. If we replace the dependency of IOrderInterceptor with ICompositeOrderInterceptor in our order service, we can effectively add functionality to our order service without changing a line of code in it or touching the container configuration again.

C#


1
2
3
4
5
6


private readonly ICompositeOrderInterceptor orderInterceptor;

public OrderPlacementService(ICompositeOrderInterceptor orderInterceptor)
{
 this.orderInterceptor = orderInterceptor;
}

Our OrderPlacementService will not receive a composite interceptor with the known interceptors injected into it :

We’re taking advantage of a specific feature of StructureMap here : if you require a list or an array of items in your constructor, the container will provide all instances registered in it. If we add another interceptor to trigger on small orders,

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10


public class SmallOrderInterceptor : IOrderInterceptor
{
 public void Intercept(Order order)
 {
 if (order.Amount > 300)
 {
 Console.WriteLine("Small order!");
 }
 }
}

the container will automatically inject an instance of the new interceptor :

Which allows us to change the behaviour of our application by adding code and not modifying anything. See Jimmy Bogard’s post on connecting implementations to open generics types for a more advanced example on how containers can accommodate change in an application.

Different types of injection

Two distinct styles of injection are Constructor Injection and Setter Injection .

Constructor injection takes dependencies using the constructor like the code above – it involves setting global variables (if necessary) to the constructor parameters to make these dependencies available for the entirety of the class. The constructor advertises dependencies as required – you can not instantiate an object without providing it’s dependencies.

Setter Injection provides properties for setting dependencies. No explicit advertising of dependencies is necessary and you can instantiate a class without the required dependencies – so a set of reasonable defaults should be assumed. This does make a class easier to instantiate, but adds the onus on the class to manage it’s default dependencies.

Constructor injection is clearly preferred over Setter Injection in most cases. Where you can’t control the instantiation of objects (as in Web Forms), you can apply Setter Injection to the already existing object .

You can do DI wrong

There are several anti-patterns you can get yourself into : Relying on the Service Locator pattern effectively replaces any dependencies with a dependency on the container , defeating the purpose somewhat. Not only could the choice of container percolate throughout our codebase, but now any tests need to mock out the container. Even worse is a direct dependency on a static container like ObjectFactory since we can’t run tests without some shared state between them. In some cases this querying the container direct is a necessary evil, particularly when using the container as a plugin framework. This is the most common anti-pattern I see with people getting started in DI.

Containers are best called only on application initialization and resolving root components with the rest of the codebase not having any explicit dependency on it. Too many dependencies tied to a class can cripple a code base, making it fragile and prone to change . While the problem is not tied directly to DI, DI can help hide the pain of having too many dependencies (not a good thing). Using Setter injection enables objects to be instantiated in an invalid state. It also manages to tie classes directly to their dependencies through the need for reasonable defaults.

But wait! There’s more

Dependency Injection containers have grown into much more than simple type resolvers. Once you’re comfortable with the basics you can start investigating:

Containers typically provide some kind of lifecycle management for constructed objects. For example, it’s dead easy to configure a dependency as a singleton, or to tie it to the HttpContext .
Conditional object construction .
Custom conventions for dependency registration .
Connecting implementations to open generic types .
Nested containers .
Interception and substitution .

A container implementation for everyone

There are several open source/freely available containers for use on .NET. These are the most popular ones :

StructureMap : the oldest one, and still my favourite. Combines power with speed and an elegant Fluent API.
Castle Project : container is part of a suite of tools. Powerful, but I’ve found the syntax rather hard to get started with.
Spring.NET : not just a container, but an application framework. Very heavy on XML configuration, but the documentation is good and enterprise support is always welcome.
Ninject : lightweight container that embraces lambdas to it’s fullest.
AutoFac

Some people even write their own .

Conclusion

Probably the most challenging aspect of explaining DI is the overwhelming amount of information that comes along with it. In the last couple of years I’ve come to the conclusion that this is the way of building applications using statically typed languages – of course, most of the principles apply to dynamic languages, but IOC just isn’t that big in, for example, Ruby .

Photo by Ilya Pavlov on Unsplash

Just finished reading...

Fri, 25 Jun 2010 00:00:00 +0000

The Passionate Programmer: Creating a Remarkable Career in Software Development (Chad Fowler) and The Pragmatic Programmer (Andrew Hunt and David Thomas).

I wish I had read The Pragmatic Programmer earlier in my career as it provides a thorough overview of good practices in software development. Reading this a couple of years ago would have shortened some learning cycles.

The Passionate Programmer is a truly inspiring book, and comes highly recommended if you

feel that your career is in a lull, or
need guidance in evolving your career as a software developer.

Photo by Roman Trifonov on Unsplash

NGenerics - new team member

Wed, 21 Oct 2009 00:00:00 +0000

A hearty welcome to David Schmitt who joined the NGenerics Team. David’s bio : David is a Debian Admin and .Net Developer living and working in Vienna, Austria. He’s partner in a small tech company startup at http://dasz.at/ . His current interests with NGenerics is the implementation of the Immutable namespace. As stated in his bio, David will be working on implementing a selection of immutable data structures.

Photo by rawpixel.com on Unsplash

NGenerics 1.4.1 Beta released

Tue, 25 Aug 2009 00:00:00 +0000

You can find it on our new project site . This build includes the following (straight from the release notes) :

New Platforms

NGenerics now has a Silverlight version in this release targeting Silverlight 3.

New data structures

Single-Valued (i.e) not KeyValuePair version of the RedBlackTree.
A generic version of the priority queue that can have a key implementing IComparable, or any object with an IComparer instance.

New Patterns

Added the Converter pattern under NGenerics.Patterns.Conversion.
Added the Specification pattern under NGenerics.Patterns.Specification.

New Extension Methods

ConvertTo, Serialize, Deserialize, and DeepCopy extension on Object
ForEach extension on IEnumerable
IsSimiliar extensions on double
Missing list extensions in Silverlight

Changes to existing data structures

Added construction mechanism to Singleton - the Singleton value can now be lazy-instantiated.
Removed the IVisitable interface and added extension methods for applying a visitor over any instance of IEnumerable. (*breaking change*)
Moved visitor-related classes to NGenerics.Patterns.Visitor (*breaking change*)
Added FindNode method to GeneralTree.
The previous version of the priority queue that accepts integers as keys is now named ClassicPriorityQueue. (* breaking change *)

Bug Fixes

Collection Add raised in ObservableHashList instead of Remove on RemoveItem

Misc

Lots of internal housekeeping and refactoring to up the quality of the code base.
More documentation.
Upgrade to Sandcastle Help File Builder 1.8.0.2 Beta.
Code Coverage is now at 96%.

Please log any issues on the issues list , and feature requests on UserVoice . For discussions, please use the ngenerics-users google group . Enjoy!

Edit: NGenerics is now hosted on GitHub .

Photo by Markus Spiske on Unsplash

NGenerics has moved

Sun, 09 Aug 2009 00:00:00 +0000

The home of NGenerics has moved to Google Code from it’s original project site on CodePlex . We’re still in the process of migrating some of the content like the downloads, but we’re mostly up and running. If you have any issues or feature suggestions, please submit them to the Google Code project site .

Edit:

NGenerics is now hosted on GitHub here .

Photo by Joey Kyber on Unsplash

NGenerics project move update

Sun, 09 Aug 2009 00:00:00 +0000

Content migration for NGenerics went well - but please note the following changes:

All downloads for releases moved to the new project site .
Due to Google not being willing to add the MS-PL license to Google Code , we’ve had to re-license NGenerics under the LGPL . LGPL is practically the same license but more wordy.
A User Voice site is available for feature requests - please log (and vote for) any features you would like to see in NGenerics. We’ll get to these in order of the number of votes attached to them.
Discussions are now hosted on the ngenerics-users Google group .

If you experience any problems caused by the move, let us know by sending a message to the ngenerics-users group or leaving a comment to this post.

Edit: NGenerics is now hosted on GitHub .

Photo by Denys Argyriou on Unsplash

NGenerics overview - the Priority Queue

Tue, 04 Aug 2009 00:00:00 +0000

Previous instalments

General Data Structures

Trees

The Priority Queue data structure has the same basic behaviour and operations as the classic queue found in .NET . A queue applies a first-in, first-out (FIFO ) approach to a list of objects. This data structure is typically used in message processing to process messages in order of arrival. Queue implementations tend to have the following operations available :

Enqueue - Adds the object at the back of the queue.
Dequeue - Fetches and removes the object at the front of the queue.
Peek - Fetches the object at the front of the queue for inspection without removing it.
IsEmpty - Provides information on whether the queue contains any items.
Count - The number of items in the queue.

A priority queue still retains queue semantics, but orders items with relation to their priority - that is, in a Min Priority Queue, the item with the lowest priority will always be at the head of the queue, while items in a Max Priority Queue will sort items in such a way that the item with the highest priority will be at the head. To illustrate, take the medical Triage system for applying priority to the treatment of patients (copied directly from Wikipedia ) :

Immediate: The casualty requires immediate medical attention and will not survive if not seen soon. Any compromise to the casualty’s respiration, haemorrhage control, or shock control could be fatal.

Delayed: The casualty requires medical attention within 6 hours. Injuries are potentially life-threatening, but can wait until the Immediate casualties are stabilized and evacuated.

Minimal: “Walking wounded,” the casualty requires medical attention when all higher priority patients have been evacuated, and may not require stabilization or monitoring.

Expectant: The casualty is expected not to reach higher medical support alive without compromising the treatment of higher priority patients. Care should not be abandoned, spare any remaining time and resources after Immediate and Delayed patients have been treated.

We can define these classes of priority as a Java style enumeration :

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


public class Triage : IComparable<Triage> {

 // "Constant" values
 public static readonly Triage Green = new Triage(1);
 public static readonly Triage Yellow = new Triage(2);
 public static readonly Triage Red = new Triage(3);

 // Triage Implementation
 public Triage(int value)
 {
 Value = value;
 }

 public int Value { get; private set; }

 // IComparable Members
 public int CompareTo(Triage other) {
 if (other == null)
 {
 throw new ArgumentNullException("other");
 }

 return Value.CompareTo(other.Value);
 }
}

Using a priority queue we can trivially ordering of patients using the Triage value:

C#


1
2
3
4
5


var queue = new PriorityQueue(PriorityQueueType.Maximum);
queue.Enqueue(new Patient("P1"), Triage.Red);
queue.Enqueue(new Patient("P2"), Triage.Green);
queue.Enqueue(new Patient("P3"), Triage.Green);
queue.Enqueue(new Patient("P4"), Triage.Yellow);

Running the code above will result in a priority queue that looks like this :

The priority queue will insert any new patients in their appropriate spot, much like the behaviour of a Sorted List . Inserting another critical patient (Red), will result in the following state :

Note that items follow chronological order when their priorities are equal. Retrieving the next patient is as simple as calling the Dequeue method on the queue. The Dequeue operation removes the item at the head of the queue:

C#


1
2
3
4


Triage priority;
Patient criticalPatient = queue.Dequeue(out priority);
Console.WriteLine(criticalPatient.Name); // P1
Console.WriteLine(priority.Value); // 3 (Red)

Typically, priority queues are typically implemented using a heap or a self-balancing tree . The implementation in NGenerics uses a Red-black tree which makes the operations perform at O(log n).

Photo by Charles Deluvio 🇵🇭🇨🇦 on Unsplash

Playing with types - non-unique property and field names

Thu, 30 Jul 2009 00:00:00 +0000

Cool, I didn’t know you could do that:

Of course, this doesn’t play all that well with C#:

I should complete my reading of that CLI specification …

Photo by Avi Richards on Unsplash

Emitting a Type that inherits from a base type referencing the built Type

Sun, 26 Jul 2009 00:00:00 +0000

Simple in hindsight, yet it took me a while to craft the correct search terms to come up with this solution. I can’t even find that forum post now, so hopefully this entry will save someone else some time.

In a lot of solutions there exists a base class that would take the inheritor as a type reference to be able to do some work on it - you’ll typically see this pattern in fluent interfaces and classes that use reflection. In our current solution we generate NHibernate entities on the fly, which all need to inherit from a base class that look something like this :

C#


1
2
3
4


public abstract class DynamicEntity : IDynamicEntity where T:DynamicEntity
{
 public virtual long Id { get; set; }
}

Utilizing TypeBuilder , we can build new classes as follows :

C#


1
2
3
4


TypeBuilder typeBuilder = moduleBuilder.DefineType(className, TypeAttributes.Public);
Type baseType = typeof(DynamicEntity);
typeBuilder.SetParent(baseType.MakeGenericType(?)); // Add methods, properties, etc.
Type newType = typeBuilder.CreateType();

Note the question mark (?) in the call to MakeGenericType. What do we set the base type to if the base type has to include the unconstructed type? Turns out that the solution is simple (and I’m embarrassed I didn’t see it sooner) : TypeBuilder inherits from System.Type , so we can reference it as a type placeholder:

C#

1

typeBuilder.SetParent(baseType.MakeGenericType(typeBuilder));

Photo by Markus Spiske on Unsplash

NGenerics Overview - The ObjectMatrix

Thu, 09 Jul 2009 00:00:00 +0000

Previous instalments

General Data Structures

HashList

Trees

Looking at tree structures will make your eyes bleed, so we’ll take a break to discuss one of the simpler data structures NGenerics offers - the ObjectMatrix . The ObjectMatrix is a representation of a 2-dimensional array of objects, like, for example, a game board :

Why not use a two-dimensional array in the first place? The ObjectMatrix offers the following useful operations at the time of writing:

Rows and Columns - formal definition of rows and columns - much better than having .Length scattered all over your code.
GetSubMatrix - enables the retrieval of a subset of the matrix.
InterchangeRows and InterchangeColumns - swap individual rows and columns.
GetRow and GetColumn - provides an easy way of retrieving individual rows and columns.
AddRow and AddColumn - add new rows and columns when you need more space.
DeleteRow and DeleteColumn - delete individual rows and columns.
Resize - resize the matrix to have any arbitrary column/row count.

Use it like you would use any two dimensional array :

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


var matrix = new ObjectMatrix(3, 3);
matrix[0, 0] = new Cross();
matrix[0, 1] = new Nought();
matrix[0, 2] = new Nought();
matrix[1, 0] = new Nought();
matrix[1, 1] = new Cross();
matrix[2, 2] = new Cross();

var enumerator = matrix.GetEnumerator();

while (enumerator.MoveNext())
{
 Console.WriteLine(enumerator.Current);
}

The ObjectMatrix is also the base class for one of the biggest classes in NGenerics - the mathematical Matrix , which we’ll get to in a later post.

Photo by Markus Spiske on Unsplash

Tarjan's Algorithm

Mon, 06 Jul 2009 00:00:00 +0000

Thanks to a contribution from Andre van der Merwe (blog , twitter ), NGenerics now features Tarjan’s strongly connected components algorithm on it’s Graph implementation. By using Tarjan’s algorithm, we’re able to find the strongly connected components (read : cycles) in a directed graph. You can invoke it via the FindCycles method on the Graph<T> class, available from build #1.0.0.55764 (30 Jun 09 13:40) .

Photo by Simon Abrams on Unsplash

NGenerics Overview - Binary Trees and Binary Search Trees

Tue, 30 Jun 2009 00:00:00 +0000

Previous instalments

General Data Structures

HashList

Trees

GeneralTree and the Visitor Pattern

If you haven’t read my previous post on General Trees, it might be a good idea to do so before reading further.

A Binary Tree is a refined version of the General Tree that limits the number of children each node can have - two to be exact. Child nodes are thus referred to as the Left and Right child nodes. The BinaryTree<T> class in NGenerics provides an implementation of a Binary Tree, which, when simplified, looks something like this :

C#


1
2
3
4
5
6
7


public class BinaryTree
{
 BinaryTree Left { get; set; }
 BinaryTree Right { get; set; }

 T Data { get; set; }
}

We can represent a simple binary tree visually like this:

Binary Trees support pre-order and post-order traversal as discussed in my previous post, and also support in-order traversal. An in-order traversal visits the left node, the root node and lastly the node on the right. When supplied with an InOrderVisitor instance, the visitor will visit tree nodes in the order (7, 4, 8, 2, 5, 1, 3, 9, 6, 10) :

As with General Trees, the BinaryTree<T> class supports breadth first traversal and has ICollection<T> semantics.

Binary Search Trees

Search trees attempt to address the problem of reducing the number of comparisons in finding items. While not as efficient as Hash tables with their O(1) performance, they do cut down search times over larger data sets. Search trees are able to do this by adding constraints to tree data structures. A Binary Search Tree is the simplest of search trees with the following rules in place:

Each value in the tree is distinct from any other (unique).
Each left child node has a value smaller than the value of its parent.
Each right child node has a value larger than the value of its parent.

The BinarySearchTree<TKey, TValue> class in NGenerics implements the IDictionary<TKey, TValue> interface for simple lookups based on keys, and supports the usual traversal algorithms. Reusing our example tree and changing it to comply to the rules could result in the following tree:

It’s easy to see how a Binary Tree can cut down search times - when using a sequential data structure (array, list), or when manually searching an unordered tree like the plain BinaryTree, we potentially need to search through every item in the collection before we find it with a worst-case performance of O(n). With Binary Search Trees we can cut down that time by comparing node values with the search value and deciding on which branch to continue searching on - enough to make searching a Binary Trees on average an O(log n) operation. To illustrate, searching for the value “3” in our sample tree will result in 4 comparisons :

Although Binary Trees are not nearly as efficient as hash tables with regards to searching (assuming that you’ve implemented a proper hash function), Binary Search Trees (and most other search trees) provide an enormous benefit over hash tables in that we can traverse items in sorted order without any preprocessing. We can use an in-order traversal to yield items in sorted order:

Binary Trees do suffer from a couple of worst-case performance scenarios where there performance could degrade to O(n) in searches. Adding items to the tree in ascending order will result in the following worst-case scenario:

Adding items in descending order will result in a tree that grows in the opposite direction, with the same devastating effect. Self-balancing trees address this problem but that’s a topic for a future post. If the data you’re pumping into the tree is random you’ll probably never hit this situation - but if you have data with some order in it a self-balancing tree might be a better fit.

Photo by Andrew Haimerl on Unsplash

Developer Podcasts

Mon, 29 Jun 2009 00:00:00 +0000

I’ve found podcasts a good way to spend your time when you’re stuck in traffic - here’s a list of my favourite developer podcasts in alphabetical order :

.NET Rocks - still one of the best podcasts out there. The banter is fun to listen too, and they have a wide variety of guests and topics on .NET. Also presents a large (but fun) challenge in catching up on previous episodes with more than 400 episodes recorded.
43 Folders - a productivity podcast, recommended.
Alt.NET Podcast - infrequent releases, but some interesting topics.
Deep Fried Bytes - good topics and friendly banter in a Southern flavour.
Elegant Code - good topics, nice variety of guests.
Hanselminutes - possibly my top podcast. Scott Hanselman provides a technical (and sometime hilarious) view on a variety of developer topics.
Get Scripting - useful if you’re into PowerShell scripting. The audio quality is not that great though.
Herding Code - falls into the same class as Deep Fried Bytes and Elegant Code.
Linux Outlaws - News cast for everything Linux.
On Open Source - variety of topics on different platforms/languages, infrequent releases in audio format.
Polymorphic Podcast - infrequent releases, but some good content in most of the episodes.
Pragmatic Podcast - I love the format of the Pragmatic Podcast - it plays like a news reel.
Rookie Designer - originally downloaded it for my wife, but now I’m listening to it. All about the life of a designer (web / graphics).
Ruby On Rails Podcast - ruby on rails.
Software Engineering Podcast - more formal discussions on computer science topics. The accent takes some getting used to, but the content is worthwhile.
Sparkling Client - short and sweet Silverlight podcasts.
The ASP.NET Podcast - short podcasts on ASP.NET and web development in general - sound quality gets in the way sometimes.
The Java Posse - newscast on everything Java.
The Startup Success Podcast - podcast on what it takes to build an IT startup - highly recommended.
The Thirsty Developer - wide variety of topics, good content.

And the winners are…

If you are a .NET developer starting out with podcasts try these first:

Also see this Stack Overflow question and Scott Hanselman’s post for other lists of podcasts.

Photo by NeONBRAND on Unsplash

NGenerics overview - GeneralTree and the Visitor pattern

Sun, 21 Jun 2009 00:00:00 +0000

Previous instalments

General Data Structures

HashList

[ Note : This post feels like Computer Science 101 - but I felt it necessary to discuss the basic tree concepts before we move on to some of the more specialized trees like search trees. ]

The GeneralTree<T> class in NGenerics provides a way of defining tree structures in a simple, recursive way. It has a simple, recursive definition :

C#


1
2
3
4
5


public class GeneralTree
{
 T nodeData;
 List childNodes;
}

Visually, we can represent it, well, as a tree :

In contrast to some of the more specialized tree structures (like binary trees and search trees), the GeneralTree enforces no constraints on the structure of the tree. Each tree node have no constraints on the number of children or order of those child nodes. General trees are useful for expressing some relationship between objects without having these relationships encoded in the object itself. Once we have these relationships set up, we can start traversing the tree in all kinds of interesting ways.

The Visitor Pattern

The visitor pattern provides a way of separating the acting logic from the traversal logic of the structure. Almost all data structures in NGenerics can accept a visitor instance for traversing the structure, either via the Accept extension method on IEnumerable<T> , or through specialized traversal methods on the data structures themselves. A visitor needs to adhere to the IVisitor<T> interface :

C#


1
2
3
4
5


public interface IVisitor
{
 bool HasCompleted { get; }
 void Visit(T obj);
}

When traversing a data structure the Visit member gets called on the visitor with each visited instance. A contrived example of a visitor is a SummingVisitor:

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


public class SumVisitor : IVisitor
{
 public void Visit(int obj)
 {
 Sum += obj;
 }

 public bool HasCompleted
 {
 get { return false; }
 }

 public int Sum
 {
 get; private set;
 }
}

The HasCompleted property provides a way for the visitor to stop traversing the structure. This might be useful in visitors that search for specific items, and would like to quit after they’ve found what they’re looking for. Tree Traversals There are two ways of traversing trees - breadth-first traversal, and depth-first traversal. Both these implementations in NGenerics make use of the visitor pattern. Breadth-first traversal visits each node on a level before moving on to the next level. In our example tree above, breadth first traversal would result in the following order of visitations:

Depth-first traversal, will result in visiting nodes from top to bottom in the tree. There are three types of depth-first traversal routines that visit the nodes in different orders:

Pre-order traversals: root -> left child -> right child
In-order traversals: used in binary trees - left child -> root node -> right child.
Post-order traversals: left child -> right child -> root

For these traversal types, NGenerics defines the InOrderVisitor , PostOrderVisitor , and PreOrderVisitor classes that custom visitors can inherit from. Traversing our sample tree in pre-order fashion will result in the following order of visitation:

An ubiquitous use for pre-order traversals is in the evaluation of expression trees .

A post-order visitor will visit the tree in the following order:

More GeneralTree<T> Goodness

Apart from having ICollection<T> semantics and enabling traversals of the tree structure, the GeneralTree class offers other useful methods:

Ancestors : Retrieve all ancestors of the current node as a path through the tree.
Descendants : Retrieve all descendants of the current node as a path through the tree.
Each node keeps track of both the parent and the child nodes and enables easy detaching and re-attaching of nodes to any parent node.
We can sort TreeNodes on a child level (i.e. all children of a specific node) or recursively through the tree.

Photo by Bart Zimny on Unsplash

Avoid the home page in Internet Explorer

Wed, 17 Jun 2009 00:00:00 +0000

More for my own records than anything else… I’m a big fan of the about:blank home page, but sometimes corporate policies dictate (and enforce) the company intranet site as IE’s home page. To start IE without going to the set home page, add the nohome command line parameter as such :

Bash

1

"C:\Program Files\Internet Explorer\iexplore.exe" -nohome

Photo by Colin Rex on Unsplash

NGenerics for Silverlight

Wed, 17 Jun 2009 00:00:00 +0000

The Code Better TeamCity guys were kind enough to install the Silverlight runtime on the build server a while ago, which means that NGenerics now has an automated Silverlight build.

If you’re interested in trying out NGenerics on the Silverlight platform, you can find the assemblies under NGenerics Trunk/Artifacts. Be sure to let us know if you find any bugs/pain points/features that can need improvement on this particular platform.

Photo by Markus Spiske on Unsplash

NGenerics on Ohloh

Wed, 17 Jun 2009 00:00:00 +0000

I share the same sentiment as Ayende on the visibility of open source project usage - page views and downloads are not useful in determining usage. If you do use NGenerics in your projects (whether it be public or private), you can let us know on the Ohloh project page . If you’re using it in an public/open source project, drop us a line so that we can add a link to you in our Hall Of Fame .

Photo by Kevin on Unsplash

100% Code Coverage

Mon, 15 Jun 2009 00:00:00 +0000

I’ll say it once more - it’s not worth it.

Let it go.

Your aim should be at about 70% code coverage - as long as you know in your hearts heart that you have tested that you should have.

What’s vitally important (and which is the whole reason why I run coverage tools in the first place), is not the warm, fuzzy feeling I get in my tummy when I see a high number - it’s the easy identification of untested code and pathways that yield value in this regard.

Photo by Ben White on Unsplash

NGenerics overview - the HashList

Mon, 15 Jun 2009 00:00:00 +0000

Something that I find a use for in almost every project I work on, is the HashList (also known as a MultiMap in the Java world) in NGenerics 1.2. A HashList is a multi-valued dictionary that uses a Dictionary<TKey, IList> under the covers. It still retains dictionary semantics but handles the creation and destruction of the key/list pairs itself.

For example, adding a couple of items to the HashList will have the following result:

C#


1
2
3
4
5


var legsHashList = new HashList();
legsHashList.Add(4, "Cow");
legsHashList.Add(4, "Horse");
legsHashList.Add(2, "Chimpanzee");
legsHashList.Add(8, "Spider");

Key	Values
4	Cow	Horse
2	Chimpanzee
8	Spider

Removing the cow and the horse will result in removing the list (and the key) from the dictionary, affecting both the key count and the value count. The HashList implements IEnumerable<KeyValuePair<TKey, IList<TValue»> so you can traverse items in grouped order. Sure beats the heck out of having all that list management code sitting in your code base.

Photo by Pankaj Patel on Unsplash

Public CI for NGenerics

Mon, 30 Mar 2009 00:00:00 +0000

Thanks to the Code Better guys , JetBrains , Ideavine , and Devlicio.us , NGenerics is now hosted on a public CI server .

The build script packages the main NGenerics assemblies in a zip file, so you always have access to the latest build from source. You can find those under “Artifacts”.

We’re still waiting for a response from the Code Better team on supporting Silverlight on the server - if we get that, the latest Silverlight version of NGenerics will be available as well. If you own an open source project and want CI for your project, find James Kovacs’ post here .

Photo by Igor Ovsyannykov on Unsplash

Visual SVN - Subversion at its best

Tue, 10 Mar 2009 00:00:00 +0000

About a year ago, the VisualSVN guys gave the NGenerics team a complimentary license for their product. After a year with it, I would not using Subversion as a VCS without it!

Get it here, and have a look at their VisualSVN Server product while you’re there - also a great product.

Photo by Tyler Nix on Unsplash

Specification pattern

Sun, 05 Oct 2008 00:00:00 +0000

The Specification pattern has been added to NGenerics . In my previous post on the Specification Pattern, we explored creating specification functionality using extension methods. It’s been implemented with the operator methods (And, Or, Xor) on the actual interface, with an abstract class forming the base of all specifications. The deal-breaker for this approach was the need to add operators |, & and ^ to trim down the syntax some. With the Specification Pattern in NGenerics, you can now do this:

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35


public class Customer
{
 public string Name { get; set; }
 public string Surname { get; set; }
 public CustomerStatus Status { get; set; }
 public BalanceStatus BalanceStatus { get; set; }
}

 public enum BalanceStatus
 {
 Settled,
 Owing
 }

public enum CustomerStatus
{
 Bronze,
 Silver,
 Gold
}

public bool ValidateGoldCustomer()
{
 var specification = new PredicateSpecification(x => x.Status == CustomerStatus.Gold)
 .And(x => x.Name == "CustomerName")
 .And(x => x.BalanceStatus == BalanceStatus.Settled).Not();

 var testCustomer = new Customer
 {
 Name = "CustomerName",
 Surname = "CustomerSurname",
 Status = CustomerStatus-.Gold
 };
 return specification.IsSatisfiedBy(testCustomer);
}

Or you can use the operators defined in AbstractSpecification<T>:

C#


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


public bool ValidateGoldCustomerWithOperators()
{
 var specification = new PredicateSpecification(x => x.Status == CustomerStatus.Gold) &
 new Predicate(x => x.Name == "CustomerName") &
 !new PredicateSpecification(x => x.BalanceStatus == BalanceStatus.Settled);

 var testCustomer = new Customer
 {
 Name = "CustomerName",
 Surname = "CustomerSurname",
 Status = CustomerStatus.Gold
 };

 return specification.IsSatisfiedBy(testCustomer);
}

The part I’m looking into now is subsumption and partially satisfied specifications. From Martin Fowler and Eric Evans’ paper :

Subsumption allows you to compare specifications to see if satisfying one implies satisfaction of the other. It is also sometimes possible to use subsumption to implement satisfaction. If a candidate object can produce a specification that characterizes it, the testing with a specification then becomes a comparison of similar specifications — removing the coupling of specification mechanism entirely from the domain.

Photo by William Felker on Unsplash

MEF now MS-PL

Sat, 04 Oct 2008 00:00:00 +0000

I take back every word I said. Now it feels worth looking into .

Photo by Olu Eletu on Unsplash

NGenerics 1.3 Release

Thu, 25 Sep 2008 00:00:00 +0000

NGenerics 1.3 has finally reached production status - you can download it here .

With it released, we can start working on some exciting new features for 1.4. Watch this space!

Photo by Lewis Ngugi on Unsplash

LiveWriter Plugins

Sat, 13 Sep 2008 00:00:00 +0000

Installed a bunch of them - ahh, it’s good to have them back.

13 September 2008 - Update : We’ve found the little girl in the picture’s first tooth!

Photo by Neven Krcmarek on Unsplash

Extension Method Patterns

Wed, 10 Sep 2008 00:00:00 +0000

One of the walls I hit the most in C# when designing classes is the lack of support for multiple inheritance, which makes that one spot for inheriting from a very valuable spot indeed. For the purposes of this discussion, we’ll start with a simple implementation of the specification pattern :

C#


1
2
3


public interface ISpecification<T> {
 bool IsSatisfiedBy(T item);
}

The evaluation is all set, but now we feel the need to add the boolean operators And, Or, and Not. Most developers follow the route of changing the interface to add And, Or and Not, create a base (abstract) class (let call it SpecificationBase) implementing the operators and leaving the IsSatisfiedBy method abstract. The downside of this approach is that concrete implementations of specifications are forced to give up their one empty slot for inheritance. An alternative is to leave the interface as is, and add extension methods that operate on the interface - no inheritance needed.

C#


1
2
3


public static ISpecification<T> And<T> (this ISpecification<T> left, ISpecification<T> right) {
 return new AndSpecification<T>(left, right);
}

The downside with this implementation is that you need to import the namespace in order use the methods. Although most people will probably do this automatically anyway, it might not be clear what operations the class has to offer.

My question is this : In the context of a reusable class library like NGenerics , is using extension methods in this way evil?

Photo by Kyle Glenn on Unsplash

MS-LPL

Wed, 10 Sep 2008 00:00:00 +0000

As Miguel pointed out , this just seems … well, evil.

Notice the last clause in Section 2 :

(F) Platform Limitation- The licenses granted in sections 2(A) & 2(B) extend only to the software or derivative works that you create that run on a Microsoft Windows operating system product.

Wow. Can’t believe that you can still get away with open-sourcing code and then choosing a license like that.

There’s a great need for .NET application developers and consumers for them to be able to be cross-platform compatible - limiting yourself to a Windows environment limits your audience and adoption.

Hats off to Miguel and the Mono team for keeping Mono alive and well.

Photo by rawpixel.com on Unsplash

Vimperator

Tue, 09 Sep 2008 00:00:00 +0000

The best Firefox add-on ever.

Photo by Dan Gold on Unsplash

NGenerics 1.3 Beta Release

Mon, 07 Jul 2008 00:00:00 +0000

We’ve just released NGenerics 1.3 Beta - grab it while it’s hot! Also, don’t forget to give feedback on new features / anything that really bugs you.

Quite a bit of time passed between this release and the Alpha release. We’re currently working on achieving a quicker release schedule, about one release every month or so (alpha, beta, release version).

In doing this, we hope to:

Get features out there quicker.
Decrease the time to get feedback on what we’ve done.
Keep the momentum.
Generate less bugs per release as the scope of releases are reduced.

NGenerics 1.3 is more or less fixed in scope now. All the release version will add is some more unit tests, bug fixes, and updates to documentation. As always, you can request new features for version 1.4 here .

Photo by Markus Spiske on Unsplash

Cayla Rose

Thu, 26 Jun 2008 00:00:00 +0000

I’m a little bit late with this announcement - a month or so.

Welcome to the world, little one!

Cayla Rose Hanekom was born on the 4th of June 2008 in Johannesburg, South Africa, weighing a whopping 4.1 kg.

Photo by Jörg Angeli on Unsplash

Team Foundation Server

Thu, 05 Jul 2007 00:00:00 +0000

You would think that the integration between Visual Studio (using the TFS client) and TFS would be better than the Visual SourceSafe integration. Nope, the same annoying bugs that were present in the Visual SourceSafe integration has rocked up in TFS, so I’m assuming someone extended the old codebase.

After some unsuccessful check-ins (read, breaking the build) when I’ve worked disconnected on NGenerics, I’ve given up on TFS. Well, not quite, but as soon as the SVN bridge brings out a stable version, I’m switching.

Congratulations to the CodePlex team for trying to fill the gap - although it took Subsonic abandoning TFS to get to this point.

Photo by Tanguy Sauvin on Unsplash

Unit testing and mocks

Sat, 12 May 2007 00:00:00 +0000

I’m one of the converted when it comes to test-driven development. One of the main “aha” moments for me is not to write unit tests - it’s writing testable code. Any code that’s testable is sure to be of a higher quality and easier to maintain than typical spaghetti code.

That brings me to the second part of the title - mocks. The art of mocking objects have taken off over the last couple of years. I feel that there is a danger of abusing them to test code, when the code is not written in a testable way. I’ve seen examples where developers use mocking so much that what you’re testing is the mock object’s functionality - and not your code. In cases like these you’d be much better off refactoring than mocking dependencies.

So here’s my recommendation: focus on writing testable code, and try to reduce the use of mocks. Use it when you have no other method available to test your code.

Photo by Leio McLaren on Unsplash

New Company, new job, NGenerics

Fri, 04 May 2007 00:00:00 +0000

I’ve left Avision to join Intervate . Although I miss Avision and its people tremendously, I felt that I’ve reached my growth ceiling and that I needed some new challenges.

I’ve been an employee at Intervate for two months now - and it’s a great company : Microsoft Gold Partner, intelligent and competent people, exciting projects and learning opportunities.

At the moment I’m somewhat of a contractor - but that should pass pretty soon.

Other than that, I’m working hard to get the 1.3 release of NGenerics out of the door. We are focusing on the quality of the library - thus we’ve introduced a couple of breaking changes (like the changing of namespaces). After this release, NGenerics will remain stable (and backwards compatible). As we go along and change it, we’ll start marking methods as deprecated whenever we break compatibility with previous releases.

Photo by Fatos Bytyqi on Unsplash

Sealed classes

Fri, 04 May 2007 00:00:00 +0000

As part of version 1.3 of NGenerics , I’ve removed most of the sealed keywords from the classes (where it made sense). It took me a while, but I’ve accepted the following rule (and Microsoft is of the same opinion, it seems) :

Never seal a class unless there’s a very good reason to do so (like it being internal, security reasons, etc). Programmers using your library will use it many creative ways you can’t even imagine, and sealing a class limits that creativity.

Also see the post here .

Photo by Adam Wilson on Unsplash

NGenerics 1.2 released

Wed, 28 Mar 2007 00:00:00 +0000

Yeah, NGenerics 1.2 has been released!

If you haven’t checked it out yet, do so now - no decent programmer should go without a toolbox of data structures and algorithms.

Some new features in NGenerics 1.2 includes :

Red Black Tree
Hash List
Object Matrix
~~ReadOnlyPropertyList~~
Prim’s Minimal Spanning Tree Algorithm .

And if you try it, be sure to give some feedback - we’re always looking for new ideas / improvements .

Photo by Chris Ried on Unsplash

NGenerics Born

Thu, 28 Dec 2006 00:00:00 +0000

My data structures article on CodeProject has grown up and became a project on ~~CodePlex~~ GitHub .

To do this, I had to make a couple of major changes - like changing all the namespaces to NGenerics.*.

The new version includes a Binary Search Tree, Euclid’s Algorithm, and a couple of changes to the BinaryTree, GeneralTree, and Matrix classes.

Photo by Markus Spiske on Unsplash

Generic Data Structures and Algorithms in .NET 2.0

Tue, 12 Dec 2006 00:00:00 +0000

I’ve posted an article on generic data structures on CodeProject.

At the moment, it provides the following data structures :

Deque
GeneralTree
BinaryTree
SorterList
Heap
Bag
Set
Graph
Association
Matrix

I’ve also implemented a couple of sorting algorithms.

If you get a change, go check it out , and give me some feedback.

Photo by Clint Adair on Unsplash

I see the sea shore

Wed, 04 Oct 2006 00:00:00 +0000

I’m back from a holiday of doing nothing on Ballito’s beautiful coast line (KwaZulu Natal, South Africa)…

Back to reality…

Photo by rawpixel.com on Unsplash

XLANGS.BTENGINE.BTXTIMERMESSAGE

Tue, 26 Sep 2006 00:00:00 +0000

Wow. BizTalk is always full of surprises.

Stephen W. Thomas has an explanation here .

Photo by Marcelo Leal on Unsplash

A very interesting hobby

Fri, 08 Sep 2006 00:00:00 +0000

I’ve grown interested in Chess programming this year, and started building my own chess engine that will understand the WinBoard protocol. Chess programming is fascinating. Initial development is difficult but I’m starting off with the basics :

A simple bitboard representation.
A simple implementation of the Alpha-Beta algorithm.
Implementation of the WinBoard protocol.

When that’s done there’s going to be ample room for improvement (twenty or so algorithms left to try out and then you can still play around with different representations of the board).

Chess engines can compete against each other on sites like these - I’d love to do this soon.

Photo by rawpixel.com on Unsplash

Time is of the essence (or so it seems)

Fri, 08 Sep 2006 00:00:00 +0000

There’s been some long gaps between blog entries due to the amount of time I can afford to write.

What am I up to these days?

Still working my full time day job as a systems developer (I actually think it qualifies as a day-night job).
Writing (and fixing) articles for CodeProject.
Doing lots and lots of R&D (new frameworks, applications, technologies are emerging at an incredible rate these days). I blame this one on Microsoft with their pre-release software.
Spending lots of time on my studies (B.Sc. Hon in Information Systems). Didn’t think it would take up this much time when I started it, but I’m almost there…
Working on the next big thing, which will hopefully provide an extra revenue stream.
Spending time with the family ;)
Trying to get some actual sleep.

I’m taking a break soon to spent all my time on a beach for a couple of days. Live Writer is coming with, I’ll post a pic of South Africa’s beautiful coastline.

Photo by rawpixel.com on Unsplash

Test - Please Ignore

Mon, 21 Aug 2006 00:00:00 +0000

Wow, this actually works! (Testing with Live Writer).

Photo by ShareGrid on Unsplash

First CodeProject Article

Thu, 17 Aug 2006 00:00:00 +0000

Got my first CodeProject article submitted!

Yes I know, it’s small and still a work in progress but it’s a start!

The URL’s going to change soon, I’ll update this post with the new link.

Photo by NeONBRAND on Unsplash

Creating or Input Multiple Messages from or to Biztalk Maps

Wed, 19 Jul 2006 00:00:00 +0000

Sure, we all know how to do that by know, right?

Funny thing is, all the blog entries I’ve seen like the ones here and here , end up creating a new map, usually situated in the root of your assembly (or event worse, create temporary orchestrations to do that). In order to move it to another folder in your project you’ll have to edit the XML contained inside the map since it contains relative paths to the schemas mapped (and watch out for those namespaces).

Here’s an easier way to do it :

Create your empty map exactly where you want it.
Add a transforms shape and open up the properties.
Instead of choosing new map, choose existing map.
Select your newly created (empty) map.

The BizTalk Editor will modify the map to take the new message types - multiple if you selected them.

Photo by Chris Lawton on Unsplash

Sharepoint Object Model - Good Practices

Tue, 18 Jul 2006 00:00:00 +0000

Dispose those objects!

If you’re using the SharePoint object model anywhere, make sure that you dispose any objects that you create (not those that the SharePoint object model creates for you, like calling GetContextWeb()).

These objects have a small memory footprint visible to the Garbage Collector, but a large amount of unmanaged resources (effectively not visible to the Garbage Collector).

If you don’t dispose of them, the GC will not be in any rush to free up the memory used by these objects since it can’t (or won’t) see the bigger picture. The result? OutofMemoryExceptions when doing any operation that creates a lot of these objects and not disposing them.

(Side track : this is what happens to the SqlCeCommand objects ).

See the full details on “Best Coding Practices” for SharePoint in this Microsoft article .

One thing’s for certain - we can’t ignore the unmanaged world yet…

Photo by Giu Vicente on Unsplash

.NET CF - Part 1

Sat, 15 Jul 2006 00:00:00 +0000

I’ve been porting an application of ours to .NET CF 2.0, Mobile 5.0 and SQL Mobile the past couple of weeks.

.NET CF 2.0 is impressive. The .NET Compact Framework 1 was a bit lacking in functionality - it felt like Microsoft shipped an unfinished product. .NET CF 2 provides a lot of new functionality : Pocket Outlook Access, Phone and messaging capabilities, new controls, etc. Basically everything the guys at OpenNet CF had in their previous version, is now in the new version of the framework. They’ve released a new version of their library that adds even more functionality - download it and use it if you work with the Compact Framework.

Working with generics is an absolute godsend - can’t tell you how much the incessant casting of objects chewed on my nerves…

I’m a bit disappointed in the SQL Mobile zone - all the old features (read “bugs) are still there like the failure of cleaning up SqlCeCommand objects. That said, being able to make more than one connection to the database and remove my Singleton connection manager class is great! Side track : Note, Making the connection to the database is a real bottleneck. For batch operations such as inserts (Microsoft, when can we get batch transactions on Mobile?), it’s best to keep a connection open and reuse the same command - you’ll notice the speed difference when you don’t.

Microsoft has added support in the Visual Studio IDE for designing SQL Mobile databases (avoiding the pain of typing in SQL queries using a stencil on the device). Works great, gets deployed onto your device and you can even query it! What I’m missing there (and it’s driving me nuts) is support for ordering columns (I’m adding a new column and I want it to be next to the primary key), renaming tables (the way I’ve found to do this is to recreate the entire table), and stored procedures.

OK, the stored procedures is going a bit too far, but it would be nice, wouldn’t it?

Photo by John-Paul Joseph Henry on Unsplash

SSIS Woes

Fri, 02 Jun 2006 00:00:00 +0000

I’ve been dabbling with SQL 2005 a lot these days and two features impress me: SSIS and Reporting Services.

You might find it difficult to actually deploy a SSIS package.

I’m missing something with calling a web service in SSIS… Even when you change the connection manager setting for the HTTP Connection manager to a new value (say we deploy to a live environment), it still reads the WSDL file you provided and calls the wrong web service (Ethereal is the coolest network tracing tool ever btw.).

And deployment!

There doesn’t seem to be a way (enlighten me if you know a way to do this) to specify a username and a password to your HTTP connection manager at any time but design time - i.e. it’s saved with your SSIS package.

Now comes the problem : by default SSIS encrypts sensitive data like usernames and passwords in the XML file. The encryption type defaults to “Encrypt With User Key” which means that SSIS encrypts the file with your username token - no other user can run your package.

Set this to “Encrypt Sensitive With Password”, deploy and run it - it works! Now try to schedule the SSIS package with a SQL Agent Job - it fails. Try to run it manually with dtsexec: it fails, stating that it can’t decrypt the data even though you’ve specified the password for the package (Side track : no useful information comes out of running a SSIS package manually. If you want to know what actually went wrong, use dtsexec).

One final try : “SQL Storage” for the package, which leaves in unencrypted and depends on the database security to secure sensitive information. You cannot at present deploy a package to SQL Storage (unencrypted) from Visual Studio (again, enlighten me if you know how). Instead, open SQL Server Management Studio and find the “Save a copy” option under the file menu - there you can select a SQL server to deploy to.

Otherwise than the deployment issues, great work done by Microsoft on SSIS and Reporting Services!

Photo by Dominik Scythe on Unsplash

Windows Mobile 5.0

Sat, 20 May 2006 00:00:00 +0000

Yeah! Got my new toy - an IMATE JAMIN with (important!) Windows Mobile 5.0. I’ll be playing around with the new SQL Mobile and .NET Compact Framework 2.0 on this device - expect a storm of new posts.

Photo by NASA on Unsplash

Biztalk Listen Shape

Fri, 19 May 2006 00:00:00 +0000

I’m no Scott Woodgate, but I think I know a fair amount about BizTalk. However, it seems that I never recognized what the Listen shape actually does (you don’t see this mentioned very often btw.). Turns out that this little shape can solve many of the problems we’re currently having, mainly correlation issues, like situations where the message you are waiting for might never arrive, keeping your orchestration in a dehydrated state, waiting forever.

This shape works like a parallel branch, but with one major difference : Only one leg of the shape needs to complete for the orchestration to continue, while in a parallel branch all branches need to complete. Add it to your orchestration, add a receive shape, add a delay shape with a terminate on the other branch. If the orchestration does not receive a message in one day or whatever your delay is configured for, it will terminate. However, if it receives a message, the orchestration will continue as normal and the termination won’t occur.

Photo by rawpixel.com on Unsplash

Now this is jewel in my Biztalk Toolbox

Fri, 19 May 2006 00:00:00 +0000

If your BizTalk orchestration is published as a web service and something goes awry in your orchestration a specific exception might not be thrown to your web service, but rather it will throw ApplicationTimeOutException.

I was prepared to live with it, until I found the following jewel here.

Turns out you can have custom SOAP faults returned in your orchestration. Who would’ve known?

Photo by jesse orrico on Unsplash

The question about life, the universe, and everything.

Fri, 19 May 2006 00:00:00 +0000

Amazing how little time you get as a developer for doing the things you really want to do (and here I am sitting in front of a computer again ;)). This blog has been a bit stale, and I’ve made a quarter-year resolution to put new energy into it.

Projects have been keeping me extremely busy these last couple of weeks - specifically an integration project (using BizTalk of course). That’s one of those projects where everything goes wrong and all unforeseen circumstances come to the party. Hell, by this time I’m an expert at crisis management. Luckily, none of these problems are unsolvable (but that’s where the time issue come in).

How does a systems developer end up making money, moving forward in life? Not that money is my main goal in life, but hey, it would be nice to retire at 35 wouldn’t it? Probably wouldn’t stop doing what I do though…

Anyway, enough with the ramblings, let’s get some real content on this blog!

Photo by Yong Chuan on Unsplash

First Post

Thu, 11 Aug 2005 00:00:00 +0000

Here we go - my first post…

I am working as a Systems Developer for a company in Johannesburg, specializing in Microsoft technologies (C#, SharePoint, BizTalk, SQL, the whole stack).

This blog is a place where I can get my thoughts into the open and get opinions from the whole wide world (www). My intention is for this to be a learning experience for both the reader and myself.

This blog will cover BizTalk, Mobile development, SharePoint, SQL and general C# .NET topics with a little C++, AI and theoretical computing sprinkled on top.

I’m breaking a bottle of champagne against this blog and naming it “Binary Thoughts”.

Photo by Kelly Sikkema on Unsplash

About

Mon, 01 Jan 0001 00:00:00 +0000

Riaan is a Chief Technology Officer at Ozow , a leading fintech company in South Africa. He is a solution-focused technology leader with deep expertise spanning executive-level strategy and hands-on technical depth.

With a wide range of experience in software development, project delivery, and technology strategy, Riaan has led teams building some of the most prominent internet applications in Africa. He is a polyglot technologist who prides himself on his ability to learn and adapt to new challenges.

Beyond technology, Riaan has a keen interest in people, organisational dynamics, and what drives individuals and teams to succeed.

Riaan also consults through Pragcraft , providing consulting services for software engineering, architecture, and fractional CTO engagements.

Search

Mon, 01 Jan 0001 00:00:00 +0000

Binary Thoughts

The Magpie

TL; DR

What is a Magpie?

An old wives’ tale

The V model of learning

Build a solid core

Intentional learning

In conclusion

Shiny things

Rabbit holes

Faster and faster

Nobody knows everything

Setting up Jekyll - navigation with a modern site-map (and R.I.P)

Preface

Autopages

Improving performance for local build

A site map

Most recent

R.I.P. Jekyll

Setting up HTTPS with Let's Encrypt

Getting started

NGINX setup

Renewing certificates

Note

Setting up Jekyll - related posts and series

Post series

Setting up Jekyll - prettier paging and categories

Prettier paging

Displaying categories

Converting HTML blog posts to Markdown

Setting up Jekyll - the basics

Introduction

Starting off

Defaults, excerpts and URL structures

Override the look and feel

Productionise it

Working with drafts

Paging

Using Sonar Cloud on .NET Core with Travis

Still alive

Acceptance testing with SpecFlow and Selenium

Further Reading

Saving the environment with Vagrant

Now blogging on Octopress

Moving to Git

TL;DR

The Pros and cons

Getting started

Where to get help

Basic commands

More advanced usage

Using Git With Other Source Control Systems

Tips and tricks

Making learning fun

Further reading

How Small is Small Enough

Delivering software, one piece at a time

How small is small enough?

Optimizing the flow

The Humble Story Point

Complexity in Software

The law

Why manage complexity?

What is complexity in software development?

Techniques for managing complexity

Other techniques for managing complexity

Further reading

Greener pastures

Blog moved

Feature request for SQL Server Management Studio

Just Finished Reading (Lean Edition)

Overview

What to read if you are in a hurry

Dependency injection - the what and the how

.NET Developers don’t do patterns

A World filled with legacy code

Improving the existing code

A Sample implementation

Providing extension points