I recently moved my virtualization setup at home to the newly released Minisforum MS-A2. The experience so far has been solid, besides it being slightly noisier than I expected.

As part of the move, I migrated all my VMs over to the new machine. When I ordered the MS-A2, I only got a single Lexar NM790 4TB drive initially as I still had the other 4TB SSD still built into the old home server. The idea was that I could tear it off the old server and plug it into the MS-A2 (as it has 3 NVMe slots, albeit with different speeds) after the migration.

But after moving all the VMs I faced a dilemma: I was lacking the space to hibernate them for machine maintenance. Shutting them down instead of hibernating them would cause me a great deal of pain - restarting the Valar VM instances is sadly non-trivial.

Thus, I needed to look for space somewhere else. One of the largest logical disks I have mounted to VMs was a 2.44TiB volume used for storing RAW files for my photography collection. Only 1.3TiB was unused, so I could easily reduce the disk size temporarily.

So I came up with a plan.

# This happens inside the VM that is normally mounting that volume.
# First, make sure /dev/sdb1 is good and happy.
e2fsck -f /dev/sdb1 
# Next, resize the partition to 1.7TiB.
resize2fs /dev/sdb1 1.7T

For the record, here is the resulting layout as reported by fdisk.

Disk /dev/pve/vm-104-disk-1: 2.44 TiB, 2684354560000 bytes, 5242880000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 65536 bytes
Disklabel type: gpt
Disk identifier: 3637D381-0EFA-44FE-89E3-016A3BA2C60F

Device                   Start        End    Sectors  Size Type
/dev/pve/vm-104-disk-1p1    40 3565158439 3565158400  1.7T Linux filesystem

Now, I could get to work with lvreduce. As I already knew we need some extra space around that single partition, I reduced the disk to 1.75TiB.

$ lvchange -an /dev/pve/vm-104-disk-1
$ lvreduce -L 1.75T /dev/pve/vm-104-disk-1
  Size of logical volume pve/vm-104-disk-1 changed from 2.44 TiB (640000 extents) to 1.75 TiB (458752 extents).
  Logical volume pve/vm-104-disk-1 successfully resized.
$ lvchange -ay /dev/pve/vm-104-disk-1

Should be good right? I started the VM using the disk and was shocked to see the following lsblk output.

$ lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
sda      8:0    0  600G  0 disk
└─sda1   8:1    0  600G  0 part /
sdb      8:16   0  2.4T  0 disk
sr0     11:0    1 1024M  0 rom

You might not notice it immediately, but my /dev/sdb1 partition was suddenly missing. My first thought was to investigate using fdisk -l /dev/sdb, only being left more confused afterwards.

$ fdisk -l /dev/sdb
Disk /dev/sdb: 1.75 TiB, 1924145348608 bytes, 3758096384 sectors
Disk model: QEMU HARDDISK
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Device     Boot Start        End    Sectors Size Id Type
/dev/sdb1           1 4294967295 4294967295   2T ee GPT

Where did my GUID partition table go? And why is there this weird GPT partition entry at sector 1?

This is when I learned about GUID partition tables primary and backup locations: A standard GPT layout includes a primary GPT header and partition array at the beginning of the disk and a secondary (backup) GPT header and partition array at the very end of the disk.

So when I shrunk the disk, I correctly protected the data inside the EXT4 partition – but my backup GPT header still went to the dogs. And fdisk gets very confused when that backup GPT header is missing as it tries to look for that backup header and finds garbage data, falling back to the protective MBR.

Using gdisk revealed the problem in a more descriptive fashion.

$ gdisk /dev/pve/vm-104-disk-1
GPT fdisk (gdisk) version 1.0.9

Warning! Disk size is smaller than the main header indicates! Loading
secondary header from the last sector of the disk! You should use 'v' to
verify disk integrity, and perhaps options on the experts' menu to repair
the disk.
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

Warning! One or more CRCs don't match. You should repair the disk!
Main header: OK
Backup header: ERROR
Main partition table: OK
Backup partition table: ERROR

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: damaged

****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************

So how to fix it? Just write the GUID partition table using gdisk again. That’s it.

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): Y
OK; writing new GUID partition table (GPT) to /dev/pve/vm-104-disk-1.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.

Now, even fdisk got the note.

$ fdisk -l /dev/pve/vm-104-disk-1
Disk /dev/pve/vm-104-disk-1: 1.75 TiB, 1924145348608 bytes, 3758096384 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 65536 bytes
Disklabel type: gpt
Disk identifier: 3637D381-0EFA-44FE-89E3-016A3BA2C60F

Device                   Start        End    Sectors  Size Type
/dev/pve/vm-104-disk-1p1    40 3565158439 3565158400  1.7T Linux filesystem

Afterwards, I was able to set the discard flag on the mounted disk in my VM and use fstrim on the /dev/sdb1 partition to reclaim the unused space in my volume group.

The Initial Offer and Team Matching

Let me rewind a bit. I signed the offer surprisingly early, back in March 2022. This was even before team matching, so I had no idea which team I’d end up on.

Fortunately, the team matching process worked out well in the end. The first potential match was with an Android team. My recruiter reached out to ask if I was interested, and I was quick to say no. I’m not really into Android development, and mobile development isn’t my primary interest. While I enjoy dabbling with mobile projects sometimes, I didn’t want it to be my full-time focus.

A couple of weeks later, she contacted me again about a team working on vulnerability management. At that point, I didn’t have a concrete picture of what that would entail. Ultimately, I said yes – why not? It sounded more like backend software engineering and distributed systems work, which was far more appealing to me than mobile development.

I then met the hiring manager (who became my manager for the first few months). That meeting went great. A few days later, I also had a chat with the technical lead. They were both incredibly fast talkers, but the work sounded fun, so I decided to go for it.

I accepted the offer, and just like that, I had my team. That was pretty much the prelude. I officially started working in October and joined the team in Munich. Though, initially, Google flew me out to Sunnyvale for the first few weeks, which was a fantastic start.

The “Big Tech” Feeling in the Bay Area

I’d never been to the US before, so visiting and experiencing the Bay Area tech scene was fascinating. The presence of the tech world there is vastly different compared to Munich. Here, even a large office like Google feels… well, just like another company office. It’s nice, you get free food, all the perks you’d expect – but it doesn’t feel particularly special or like you’re part of something massive.

You definitely get that sense in the US, especially in the Bay Area. When you catch a company shuttle, you can choose between dozens of different buildings. That felt wild. Meeting the US-based colleagues I now work with in person was incredibly valuable.

I’m not against remote work at all, but I think meeting regularly, even just occasionally, to talk face-to-face and understand what others are working on is hugely important. Communication feels different in person. Chat is fine, but resolving complex problems often takes more effort via text.

Diving In: Rapid Onboarding Strategy

My approach during those first weeks was to dive in headfirst as much as possible. I believe getting a head start is always crucial. It builds goodwill with your manager and colleagues, allowing you to tackle problems quickly and absorb information rapidly. You can always find a more sustainable pace later.

This is a strategy I’ll definitely repeat whenever I switch teams, projects, or companies down the line. Just intensely focusing for a few weeks at the start. After that initial sprint, you can figure out the right balance: how much work do you want to do, how much are you doing, and how much is expected? Finding that equilibrium takes time. Even with my intensive approach, onboarding still took a while, maybe about a month.

I immersed myself in coding for the first month or two, and productivity really started to ramp up after that. Getting familiar with the code, getting to know people, and navigating code reviews became the focus.

The Critical Role of People Skills

One thing I perhaps underestimated, even after internships, is just how crucial it is to get to know the people who will be reviewing your code. It’s absolutely vital for the code review process itself.

Why? Because everyone has a different style and different priorities when writing code. Some emphasize strict “Don’t Repeat Yourself” (DRY) principles, others prioritize simplicity, and there are many other preferences. You honestly can’t satisfy all of them simultaneously.

Sometimes, with a specific piece of code, you have to make a choice – you can lean into one principle or another, but you can’t perfectly achieve everything without spending excessive time on it.

Knowing who your reviewers are allows you to anticipate their preferences and subtly adapt your coding style accordingly. I guess it’s similar to “code-switching” in language, where you vary how you speak depending on who you’re talking to.

Applying this to code reviews means you vary the “dialect” of your code based on your audience. This helps get your changes submitted much quicker, which is essential for team velocity. You spend less time debating why you made a certain choice because you’ve already aligned closer to their expectations.

So, yes, that’s a key takeaway: understanding your reviewers is incredibly important.

Beyond code reviews, I was assigned my first project collaborating with a senior engineer who was also relatively new to the team. We were both learning a lot. Another team member had been there longer and helped guide us. It was a fantastic experience. The senior engineer gave me plenty of space to lead the coding work, mostly handling external communication and coordination, while I focused on writing the code and ensuring everything ran smoothly.

For the second project, he was still assigned with me but was much more hands-off. I essentially had full ownership – writing the design docs and executing the implementation. It was incredibly fun figuring out the problems and finding solutions.

Throughout all this, I regularly checked in with my manager about my progress. At one point, they simply suggested, “You’re writing design docs, you seem ready for the next level. You should apply for promotion.”

I did, and it went through! I got promoted within my first year and have been at the next level for about six months now. Honestly, it doesn’t feel drastically different. The main change is that I’m no longer in the onboarding phase; I know the systems and, more importantly, the people.

Regarding people, it’s probably the topic whose impact on the work I most underestimated. How much difference do diverse personalities and approaches make within a team? A lot. Different people handle problems differently, and communicating effectively with each person often requires a distinct approach.

Some colleagues prefer you to cut straight to the chase: “Here’s the problem, here’s the solution.” Others really want to understand the problem deeply first and explore potential paths together before hearing a proposed solution.

Honestly, it can sometimes be challenging to fully grasp how certain people think or “tick.” But I’m working on getting better at it. Reading people is an extremely valuable skill. I didn’t arrive with the expectation that it would be so central to the job, but it truly is.

This one is reasonably short, let’s start with having a look at the device diagram:

There are a number of important changes present: First and foremost, I got rid of the EdgeRouter-X. Ubiquiti stopped updating it and it couldn’t handle 700 Mbps routing anyway. Now I’m back to using the ISP-provided modem box. It works reasonably well, at least it can handle my upstream/downstream speeds. My building uses G.fast to provide Gigabit access through old phones lines up until the signal reaches the ONT in the cellar. I was quite sceptical of the technology initially, but so far I’m really quite happy with the connectivity and reliability of the uplink.

The Pi and separate NAS box are gone and have been replaced by a custom-built 8L PC (nicknamed Takobia). The Pi ended up being too slow for some tasks (as it was the only “capable” device that was always on). Phasing it out would only make sense.

The NAS box was fine, but always waiting for it to turn on for a backup (and automatically shutting it down afterwards) proved to be a hassle. I also noticed that my storage footprint is pretty small and I don’t really care too much about data redundancy. Having an off-site backup for important stuff is unavoidable anyway. Also, idle power consumption was a nightmare due to all the HDD (~50-60W even after powertop tuning, which translates to ~12.50€ a month when powered on 24/7). Shutting down HDDs between backups just wears them down (they hate heat cycles), so that was no good alternative.

Building a new low-power home server

The key requirements before building Takobia were

• Small device footprint. It should fit into one IKEA Kallax square (33x33x39). I don’t care too much about the noise at it sits in the home office room anyway.
• Always on. And since energy is costly these days, it should be low power (at least during idle).
• At least 4TB of really fast storage, capable of fully-utilizing at 25GbE connection during sequential reads. However, there is no need for high redundancy as this is for nice-to-have backups, but nothing essential.
• Reasonably cheap. I don’t really have a need for enterprise-grade hardware (and I’m fine with running memtest86 once in a while). Finding a chipset with ECC support is pretty costly for Intel, the situation is better on AMDs side of things but nah.
• Connectivity. A couple of USB ports are always sweet in case I opt to connect additional add-on devices later on (e.g. scanner). I also need a single x16 PCIe slot for a 25GbE networking card.

The initial step was to settle on a specific platform. I’ve had a great experience with the Ryzen 5950X in my workstation, however, it’s not exactly energy saving. After looking at a number of idle energy consumption graphs online, I decided to take a shot again at Intel. The most recent generations of consumer Intel CPUs feature a Big-Little design (so we have low-power efficiency cores + high-power performance cores). With a little bit of tuning, we can heavily utilize the efficiency cores and avoid using the high-power cores altogether except for computationally heavy work.

The newest Raptor Lake CPUs i5 13400 and i5 13500 have both an excellent performance profile, feature a built-in GPU and a very low idle power footprint. The Passmark score of the i5 13400 sits at 25275, while the i5 13500 sits at 32468, so roughly 30% faster. I opted to go with the i5 13500 just in case I ever need the extra performance.

Next up was deciding on a case: Since my storage requirements strongly hinted towards a board with at least one M.2 w/ PCIe x4 connectivity, I ventured into the world of small-form factor PCs and discovered the ASRock DeskMeet B660. A perfect match: Cheap, motherboard + PSU were already included, 4 slots of DDR4 RAM (typically not available in that form factor) + even a PCIe 4.0 x16 extension slot. After I had the case and CPU, I just settled for readily available and cheap components to complete the build.

After tuning idle energy consumption, I’ve arrived at about 26W idle power consumption. As my energy price is fixed for the next 12 months at 0.2887€, I expect about 65€ in terms of energy costs per year. Assuming I can turn down my existing rented VPS instances (saving me 17.58€ per month) and move that workload over to Takobia, this will yield about 150€ in savings per year. In the case that this machine keeps running as long as its predecessor did (over 10 years), the savings will actually exceed the cost of buying the PC components in the first place.

For 25GbE networking I use two Mellanox MCX4121A-ACAT sourced from Ebay for 69€ a piece interconnected via an off-the-shelf SFP28 DAC cable for just over 30€. The fast interconnect enables Takobia to serve as a very fast storage extension for my main workstation.

Software stack

Takobia uses sox (which was called VirtM up until recently), a pretty web UI I built around libvirtd. It allows me to quickly turn VMs up and down, just as I need them and I can easily extend it with all the features I require myself. I got the idea to built something like sox from DigitalOcean as they (at least in the past) have been providing most of their machine management on top of libvirtd as well.

I already mentioned sox in the infra update last year, however, I recognize now that the project may require some rescoping (so I actually get around to implement some of the features I want). I’ll most likely drop anything that is required for cluster use as my only use case for now is a single machine anyway. Anyway, I’ll need to polish the UI and add some must-have features like SSH from the web UI, better observability and machine resizing. As mentioned above, the VMs are mostly there to take over the workload from Hetzner (which is mostly running Valar).

Otherwise, Takobia will end up taking over CoreDNS + dnsmasq DHCP service from the Raspberry Pi. I also run a little tool called dnsmasq-lth which generates hosts files from dnsmasq leases. To provide network-accessable file & software repository storage, Takobia runs smbd (with additional customizations to enable macOS Time Machine backups) and a small gitea instance. Back in 2020, I also had the old NAS store a basic copy of accessible sites on the WWW (mostly Wikipedia + TED talks). This was due to the unreliable ISP, nowadays 4G reception in my apartment is good enough thus I’ll drop the offline mirror support.

What’s next?

As more of my devices have support for more recent Wifi standards, I’ll have a look into upgrading this area. I’d love to have my Macbook be able to access the file-storage mentioned above with more than 1 Gbps over WiFi. To make this possible, I’ll have to find an access point which supports cable-based connectivity at these speeds. There are a couple of 2.5GbE APs around at reasonable prices, e.g. the TP-Link Omada EAP670 AX5400, but anything beyond 2.5GbE is still quite exotic.

As I did some load testing, I ran into the fact that my proxy spend about 70% of its CPU time multiplying some big.Int for the TLS handshake, which seemed kind of excessive. I published a little demo repository where you can try it yourself.

When looking at a profile for a 10s load test with RSA private keys (4096 bit)

Showing nodes accounting for 12.89s, 92.01% of 14.01s total
Dropped 218 nodes (cum <= 0.07s)
Showing top 10 nodes out of 69
      flat  flat%   sum%        cum   cum%
    10.34s 73.80% 73.80%     10.34s 73.80%  math/big.addMulVVW
     1.75s 12.49% 86.30%     12.22s 87.22%  math/big.nat.montgomery
     0.20s  1.43% 87.72%      0.20s  1.43%  math/big.mulAddVWW
     0.13s  0.93% 88.65%      0.13s  0.93%  math/big.subVV

87%! Just for setting up a secure connection. What about ECDSA with P-256?

Showing nodes accounting for 1590ms, 42.06% of 3780ms total
Dropped 170 nodes (cum <= 18.90ms)
Showing top 10 nodes out of 308
      flat  flat%   sum%        cum   cum%
     690ms 18.25% 18.25%      690ms 18.25%  vendor/golang.org/x/crypto/curve25519.ladderstep
     210ms  5.56% 23.81%      210ms  5.56%  crypto/sha256.block
     150ms  3.97% 27.78%      150ms  3.97%  runtime.futex
     100ms  2.65% 30.42%      660ms 17.46%  runtime.mallocgc
      80ms  2.12% 32.54%      470ms 12.43%  runtime.gcDrain
      80ms  2.12% 34.66%      120ms  3.17%  syscall.Syscall

18%! I guess that’s better. Finally, what about Ed25519?

Showing nodes accounting for 1640ms, 45.81% of 3580ms total
Dropped 156 nodes (cum <= 17.90ms)
Showing top 10 nodes out of 272
      flat  flat%   sum%        cum   cum%
     690ms 19.27% 19.27%      690ms 19.27%  vendor/golang.org/x/crypto/curve25519.ladderstep
     280ms  7.82% 27.09%      280ms  7.82%  crypto/sha256.block
     120ms  3.35% 30.45%      120ms  3.35%  runtime.futex
     110ms  3.07% 33.52%      110ms  3.07%  runtime.procyield

19%! It seems to use the same optimization as the ECDSA one.

418_623_557_911      cycles       # RSA
 21_593_305_599      cycles       # ECDSA
 21_083_496_865      cycles       # ED25519

So we save about 95% of CPU cycles when we switch from RSA to ECDSA when using Go’s TLS package. ECDSA is also widely supported, especially compared to Ed25519. In the case of my original proxy problem, re-issueing the certificates with new private keys increased throughput by about 30% in high-load scenarios and the bottleneck is now another system component.

The PSA part

For many services, RSA is still the default. I ran into the original issue because my Let’s Encrypt certificates all used RSA private keys. You can switch to ECDSA by simply supplying --key-type ecdsa as described in the certbot docs. Popular tools like the reverse-proxy traefik default to using RSA private keys, unnecessarily slowing down your system and prolonging TLS handshakes. So in case you don’t explicitly serve folks which prefer to use outdated crypto, you should probably switch.

Tier I: Physical layout & network

Router: Two years after my initial post I’m still happily using my EdgeRouter-X. It works fine for my needs, although I’m probably gonna switch to a self-built router at some point due to upgrading to 10G/25G Ethernet. Running 10G at home was quite pricey just a couple years ago, but these days you can find older 10GBit cards like the Mellanox ConnectX-3 cards for less than 50€.

Wireless AP: In the last post I talked about the old travel-size Wi-Fi AP I was using. Since then, I’ve switched to using a Ubiquiti FlexHD allowing me to get up to 1.7 Gbps throughput between devices on Wi-Fi. Of course, devices are throttled to 1 Gbps due to Ethernet when accessing wire-bound devices or the internet gateway.

Switching: I am still running some super low-cost low-energy 5-Port Netgear switch (GS105A) I got a couple of years ago for less than 20€.

WAN Access: I don’t get a public IPv4 (and IPv6 doesn’t work at all). I also don’t want to expose anything from my home network publicly on the internet. Luckily, I discovered the Tailscale project some time ago. The utility they provide for free is truly amazing. Thanks to Tailscale, it doesn’t really matter if I’m at home or work remotely, I’m guaranteed to have access to any machine on my home network anyway. This in combination with Wake-On-Lan on the larger machines makes for a pretty efficient compute-on-demand setup I will talk about further in the T3 section.

10GbE P2P: I have a single 10GbE connection running from my main workstation to my storage server. This is mainly to speed things up in case my workstation needs to access large datasets or perform backups. I used some cheap ConnectX-3 EN CX331A NICs I got on Ebay as well as the most basic 10G SFP+ transceivers from FS.com I could get my hands on. I guess I could’ve used a DAC cable as well, but fiber is just far more flexible and fun.

DNS: My Raspberry runs a CoreDNS instance serving as both resolver as well as nameserver for my home network DNS zone. It serves different IPs for the same zone depending which interface requests the information. For example, when a DNS request digs for the virtm.home.espe.tech entry from the Tailscale link, the CoreDNS server will respond with the Tailscale IP of my storage server. When the same request comes in from the local LAN port, it will respond with the local subnet IP from the 192.168.0.1/24 range.

You may ask Why CoreDNS? Well, It’s easy to write custom plugins for. This will be useful later when I want to add automatic DNS entry generation for my virtual machines.

Tier II: Storage & media

NAS: Back in 2020 I used a second hand Synology DS213 from Ebay. It did fine most of the time, but sometimes my TimeMachine backups wouldn’t even finish because operations timed out (which was kind of annoying!). At the end of 2020, I replaced my old workstation (a very reliable Intel 3770K) with an Ryzen 5950X-based platform, so the my 3770K is now “busy” serving files. It exposes both AFS and SMB shares.

I migrated all the old mdraid off to a new ZFS-based storage array made out of three 4TB IronWolf RED disks. I always thought that Synology uses some sort of custom RAID system? But no, it’s just a Linux software raid.

Performance wise, I’m pretty happy.

# dd if=/dev/zero of=test1 bs=2G count=1 conv=fsync
2147479552 bytes (2.1 GB, 2.0 GiB) copied, 12.2618 s, 175 MB/s
# dd if=test1 of=/dev/zero bs=2G count=1 (after cache flush)
2147479552 bytes (2.1 GB, 2.0 GiB) copied, 8.44784 s, 254 MB/s

The performance is roughly equal to single disk speeds which is totally fine for me. It’s good enough for backups and the occasional dataset scans.

The storage server keeps

• backups of my other devices
• regularly updated GDPR takeout data from major cloud services like Google and Apple
• mirror of debian testing repository (all of my Linux devices are Debian based, speeds things up quite a bit)
• up-to-date archives of important documentation and services (like Wikipedia)
• some datasets I keep for private project purposes

Earlier this year, my gateway connection got interrupted for almost a week. Having stored offline copies of some useful sites like Wikipedia was a lifesafer during that time.

I also noticed that YouTube is probably the biggest time-waste I use during the day. I deinstalled the app from my phone, which resulted in two things: YouTube is now ad-free thanks to ad-blockers. I watch less vaguely interesting videos.

ZFS allows me to do this without a lot of maintenance. I’ve read that it’s actually NOT recommended to use non-ECC RAM with ZFS due to checksumming, but so far my datasets haven’t been completely ruined because ZFS decides to rewrite them due to checksum mismatch. Although I should probably monitor for this. The block-level compression feature works pretty fine for me too and saves me around 20% of disk space.

Raspberry: The Pi is the only non-(router|switch|home-automation) device that is always on. It provides AirPlay using shareplay and the HifiBerry DAC platform. The only real issue I have with this setup is that Windows does not support AirPlay streaming. The Pi also hosts a Gitea instance for storing some repository mirroring from GitHub as well as throwaway repository storage. It also provides a simple link shortening service reachable under go/.

Tier III: Compute

I actually got around to build my own little compute platform called virtm. It runs on the aforementioned storage server. I will probably open-source it in the future, its feature set is quite minimal but for my core use cases it works pretty well. It exposes both a web-based UI as well as a gRPC API with an accompanying command line client.

It uses libvirt with qemu-kvm to manage virtual machines and is tightly integrated into my networking setup. Referring back to my old post, I wanted

I can create new Debian-based Virtual Machines with just a couple of clicks. After confirming, VirtM generates a new qcow2 image for my VM, customizes it and starts the instance.

It’s a pretty solid user flow so far. Starting, stopping and tearing down VMs works well, too. I’ll look into

• cross-machine private networks
• online migration
• displaying CPU, memory and disk usage in the UI
• easy SSH sessions via web UI or CLI (something along virtm ssh [machine]) next.

I will post more information on VirtM in the future, so stay tuned and subscribe to my Atom feed.

Just to clarify: I will focus on the environment at home, since these kind of fast-moving thought experiments don’t go well with enterprises.

What does one require?

I had to think about what kind of requirements do I have? Which kind of infrastructure do I use at home?

1. Basic internet access for

• Performing study-related task
• Entertainment
• Whatever you need the internet for
• Both my desktop and mobile devices

2. Basic local infrastructure for
   • Performing backups of both my desktop and mobile machine
   • Streaming audio to my analogue HiFi system
3. Advanced local infrastructure for
   • Running and testing computing projects
   • Learning about the latest hype cycle subject
   • Fun and profit

Another important global requirement is energy-efficiency: power costs are quite high in Germany.

Requirement decomposition

Let’s break down the different tiers into what should actually be accomplished and how.

Tier 1: Basic internet access

• DHCP service
• Traffic routing/packet forwarding to my ISP
• Local DNS resolver
• Wireless- and Ethernet-based network ports to connect my devices to

Tier 1 is instantly solved by tons of off-the-shelf components also called Home routers. I opted for a very basic one by Ubiquiti and it actually does it job very reliably, no hiccups yet. Sadly, I can’t really say that about my WiFi access point (also why I’m not listing it here, it’s running OpenWrt and branded as a travel router). Since I’ve been very satisfied with my current Ubiquiti router, I have been looking into the UniFi AP FlexHD, which is still quite pricy, but has an awesome form-factor, looks good on a desk and is powered via POE, or UniFi AP AC Lite, which is less expensive, a bit slower on 5 GHz and can be powered via POE passthrough by the ER-X.

Tier 2: Basic local infrastructure

• NFS / TimeMachine storage server
• AirPlay-supporting digital-to-analogue interface

The first requirement is accomplished by my Synology DS213+ I grabbed from Ebay (and modded a bit by replacing the default fan). The AirPlay streaming service is provided by an Raspberry Pi 4 with a Hifiberry extension board and an awesome tool called shairport-sync.

The only problem with the current configuration is that backing up my Mac over WiFi is quite slow due the aforementioned access point.

Tier 3: Advanced local infrastructure

I have two machines that serve as the basis for the compute analysis with a total of 16 Cores, 32 GB of RAM and about 1TB storage.

• Dynamically allocate storage and compute as needed
• Flexible networking
• Low configuration and maintenance overhead
• Support for multiple tenants for sharing with friends and family
• Setting up reference routes in the local network using DNS
• Only-run-what-you-use, when no compute is required it should not use a lot of power

The only thing reliably working on this list is the reference route aspect, meaning my DNS resolver on my home network is split from the ER-X router and also running on the RPI mentioned in the T2 section. The RPI runs a CoreDNS instance sourcing configuration from local storage and a frontend application I built called Koala.

Dynamically allocating storage and compute as needed can be solved either from an application perspective, where I just supply application blueprints to my compute environment, or dynamic provisioning of virtual machines, which is more flexible but also complicated. If one would choose dynamic application provisioning I guess you could probably just set up a Kubernetes cluster. But since running virtual machines is fun (and also more flexible, did I mention that?), a good off-the-shelf solution seems to be Proxmox. A few years back, I also tried to built a basic provisioning frontend but never finished it. Maybe its time to look into fjell again, since I would love to have a more refined DigitalOcean-like user experience at home.

Flexible networking is a hard one to solve. Managing virtual networks via VXLAN manually has been a horrible task in my experience, but maybe Proxmox can help me with that too.

Multi-tenancy should I be able to accomplish by setting up identity provider infrastructure. I looked into the ORY ecosystem but it seemed quite heavyweight and a bit too much for the use case I’m looking for here. At some point I found dex, a project by the folks from CoreOS (now RedHat now IBM). While I still need to look into the templating and theming (‘cause consistent user experience, duh), I have tried to set it up with a local version of Koala and it seemed to work quite well. And the connector interface seems fairly straightforward.

Only-run-what-you-use should probably be renamed to power over ethernet.

Low maintenance overhead can probably quite easy since I got only 2 machines, the possiblity of any kind of hardware failure is quite low compared to any large-scale system.

Summary

I should probably go buy a better WiFi access point. And maybe building a DigitalOcean-like interface for provisioning virtual machines, networks and storage at home is worth it?

Getting Kotlin ready

I found out that there is a Kotlin Playground very similar to the Golang Playground. It works pretty well and has an attached workshop site called Koans. If you want to execute it on your own machine, I prefer to use IntelliJ by JetBrains just because the Kotlin support is outstanding (as JetBrains is the company behind Kotlin).

Hello, world!

Because starting to learn a new language always begins with a greeting, we want to build a simple Hello-World program that outputs something to the command line.

fun main(args: Array<String>) {
  println("Hello, World!")
}

As you instantly recognize, the Java heritage is strong in this one. Since main is a static method, we do not have to wrap it with a class, although I think the Kotlin compiler puts it in a wrapper later. You can instantly see one of the great joys of Kotlin, atleast for me as a Go developer: The verbosity of Java is nowhere found. If you do not believe me, this is the same program in Java.

public class Main {
	public static void main(String[] args) {
		System.out.println("Hello, World!");
	}
}

Just by adding some Java objects to the global namespace, Kotlin has done wonders for developer productivity. This also can be seen when you want to do more complex things in Java like counting string occurences.

List<String> myWords = Arrays.asList("hello", "world", "my", "dude", "world");
int count = 0;
for (String word : myWords) {
	if (word.equals("world")) count++;
}
System.out.printf("world occurs %d times.\n", count);

We can use lambda expressions and operations to shrink that block of code massively.

val myWords = listOf("hello", "world", "my", "dude", "world")
val count = myWords.count { it == "world" }
println("world occurs $count times.")

And it’s perfectly readable and understandable. I am very pleased by the use of the equality operator for structural equality therefore testing if the content is the same and not the reference. Don’t worry, if you want to use reference equality there still is the === operator. Also if you do a variable == null check, it automatically gets extended to a variable === null expression.

Data classes

But we are still missing out on the real good stuff, a concept called data classes. Data classes are the true boilerplate killers in Kotlin, at least judging from my experience in Java. A typical Person class in Java with read-only properties would look something like the following:

public class Person {
	private final String firstName;
	private final String lastName;
	
	public Person(String firstName, String lastName) {
		this.firstName = firstName;
		this.lastName = lastName;
	}
	
	public String getFirstName() {
		return firstName;
	}
	
	public String getLastName() {
		return lastName;
	}
}

The same, but different in Kotlin looks like this.

data class Person(val firstName: String, val lastName: String)

Voilá, nothing more to it. Same method, same functionality. I begin to grasp why Kotlin is a loved programming language. It feels a lot like Swift but without the Objective-C legacy bullshit.

Time to go ashore

But, don’t despair! We can deploy our own load balancer using the Kubernetes Ingress object. It allows us to define routing rules and use a backend of our choice that uses these rules to route traffic to our cluster. And the most exiting thing: to build an Ingress controller is really simple.

But, since I don’t want to get into building your own controller yet (at least this time), we will use a proven ecosystem. Luckily, the nginx-ingress is maintained by the Kubernetes project.

cURL and apply!

Setting up the basic infrastructure is once again copying and pasting commands.

# For more information, see https://kubernetes.github.io/ingress-nginx/deploy/#mandatory-command
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml

Now our backend should be configured successfully. Maybe you already noticed that we did not expose anything yet, no IP to access, nothing. That is mostly because the frontend service is custom for each cluster deployment.

echo > ingress-nginx.yaml <<EOF
apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
spec:
  externalTrafficPolicy: Cluster
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: ingress-nginx
  sessionAffinity: None
  type: NodePort
  externalIPs:
    - <your-master-ip>
EOF
kubectl apply -f ingress-nginx.yaml

Remember to replace the external IP with your own master node IP. If you have your own DNS running (or this is a public cluster), I recommend to point add a A record to it.

Exposing our blog

Let’s assume you have your blog running on Kubernetes and exposed it using a Service called blog. To expose this application using your new nginx-ingress, you just need to add the routing rules to your cluster.

echo > blog-ingress.yaml <<EOF
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: blog
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  backend:
    serviceName: blog
    servicePort: 80
EOF
kubectl apply -f blog-ingress.yaml

In the case, you have a domain name pointing to your Ingress, you can also use host-based routing.

echo > blog-ingress.yaml <<EOF
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: blog
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: <your-blog-domain>
    http:
      paths:
      - backend:
          serviceName: blog
          servicePort: 80
        path: /
EOF
kubectl apply -f blog-ingress.yaml

Adding certificates using Let’s Encrypt

Of couse, if you publish content on the web, you want to serve it encrypted to your users. Once upon a time, using HTTPS was hard and expensive because you had to buy a certificate from a trusted company. Luckily, again, times have changed, and there is this wonderful initiative called Let’s Encrypt and a cluster component called certmanager.

To install certmanager and setup our ClusterIssuers, we have to run a few commands again.

# install certmanager
kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/master/contrib/manifests/cert-manager/with-rbac.yaml
# setup ClusterIssuer for letsencrypt-staging
echo > letsencrypt-staging.yaml <<EOF
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    server: https://acme-staging.api.letsencrypt.org/directory
    email: <your-email-address>
    privateKeySecretRef:
      name: letsencrypt-staging
    http01: {}
EOF
kubectl apply -f letsencrypt-staging.yaml
# setup ClusterIssuer for letsencrypt-prod
echo > letsencrypt-prod.yaml <<EOF
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: <your-email-address>
    privateKeySecretRef:
      name: letsencrypt-prod
    http01: {}
EOF
kubectl apply -f letsencrypt-prod.yaml

Remember to replace <your-email-address> with your own email. When you want to test your deployments, you can use letsencrypt-staging as a test issuer to check out if things work out as expected (because you do not have to deal with rate limits and such). Of couse, when you have a stable version running you should move to use letsencrypt-prod because the staging issuer is not trusted by default.

We finally have to update our Ingress, so we can test if our auto-encryption works well!

echo > blog-ingress.yaml <<EOF
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: blog
  annotations:
    kubernetes.io/ingress.class: nginx
    certmanager.k8s.io/cluster-issuer: letsencrypt-staging
spec:
  tls:
  - hosts:
    - <your-blog-domain>
    secretName: blog-tls
  rules:
  - host: <your-blog-domain>
    http:
      paths:
      - backend:
          serviceName: blog
          servicePort: 80
        path: /
EOF
kubectl apply -f blog-ingress.yaml

Finally, our blog should be exposed using HTTPS! Remember to replace letsencrypt-staging with letsencrypt-prod when you are going public.

We need a ship (or a whale)

A little disclaimer to start with: the configuration here provides no recovery mechanisms since it is based on a single master node. If you want to set up a high availability cluster, I recommend you consult the excellent Kubernetes documentation.

But before we can install kubeadm itself, we first need a container runtime. Luckily most recent Linux distros come with Docker available through the package manager. So depending which distribution you run, you consult your package manager and ask it for Docker.

sudo apt-get update
sudo apt-get install -qy docker.io

Even if your package manager ships an older version of Docker, don’t worry: Kubernetes works rather well with older versions.

We got a ship, we need a captain!

Installing Kubernetes is rather straightforwar but we first need to add the official Google package sources to our local listings.

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg \
    | sudo apt-key add -
echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" \
    | sudo tee /etc/apt/sources.list.d/kubernetes.list

If adding the key does not work, you may still need to install the apt-transport-https package. Now we can update our package manager and install the kubeadm, kubelet and kubernetes-cni tool.

sudo apt-get update
sudo apt-get install -qy \
    kubelet kubeadm kubernetes-cni

Careful here, stormy waters ahead

But before can actually use these tools, you need to further prepare the runtime system for a new captain. Since Kubernetes does not work well with Linux swap files/partitions, it is strongly recommended to disable swap space.

The easiest way to perform this action is to edit your /etc/fstab, comment out the swap line(s) and reboot the system. After rebooting, we need to get the IP we want to advertise our cluster on. When you run this action in a datacenter, you may pick the private networking IP provided by your hoster. When we run this at home, you may have only one IP assigned by your router or hypervisor so we will pick this one.

ip -c a

...
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> ...
    link/ether 52:54:00:7e:76:cb brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.234/24 brd 192.168.10.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 ...

When I run the command on my local machine, I see that I may use 192.168.10.234 in this case. One command is actually is enough to tell Kubernetes to setup our cluster using bootstrapping!

sudo kubeadm init \
    --pod-network-cidr=10.244.0.0/16 \
    --apiserver-advertise-address=192.168.10.234 \
    --kubernetes-version stable-1.10

If you want to add more nodes later on, I recommend to save the kubeadm join command printed out on your screen. It allows an easy extension of your lonely one-node cluster.

The kubeadm stores the configuration required to access the cluster via kubectl in a file under /etc/kubernetes/admin.conf. By copying it to our home directory, we can access the cluster without using sudo all the time.

mkdir ~/.kube/
sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

If you want to access the cluster from another machine, you need to copy the configuration file to it.

scp <username>@<master-ip>:~/.kube/config kubeconfig
export KUBECONFIG=./kubeconfig

You may want to copy the kubeconfig file into $HOME/.kube/config to use it as your default kubectl configuration. Be careful, you overwrite existing configurations when copying. As an alternative, you can use multiple files as your kubeconfig by separating each file path using a : (e.g. KUBECONFIG=~/kubeconf1:~/kubeconf2 kubectl get nodes).

We got a ship and a captain but we still need a sail

In our case the sail will be flannel, a popular layer 3 network fabric by CoreOS. Because even though Kubernetes allocates an IP for each pod, it actually takes not the responsibility of routing the traffic between the pods. Now, there are actually two approaches for solving this problem: either you use a software-defined network that encapsulates the traffic or you can actually setup physical infrastructure with real switches and router which is more performant but a little bit overkill for a home cluster. If you want to learn more about the Kubernetes network model, you can check out the documentation.

Setting up virtual networking was really complicated once but since we are living in the days of modern DevOps, we can setup virtual infrastructure by just running a few (actually just one) command.

kubectl apply -f \
    https://raw.githubusercontent.com/coreos/flannel/v0.10.0/Documentation/kube-flannel.yml

The supplied file actually does a lot of things I will not go deeper into, but it sets up roles for RBAC, a security model for the Kubernetes API, creates service accounts, configurations and a daemon-set. This is interesting because the container networking infrastructure then actually runs on top of Kubernetes itself.

The lonely master

Since we only have one node in our cluster the master has to do all the work. To allow it to do the heavy lifting we must actually taint it.

kubectl taint nodes --all node-role.kubernetes.io/master-

Now our master node is captain and cabin boy at the same time, how wonderful!

Running the dashboard

Since using kubectl is a nice way to interact your cluster, you stil want to see those nice, green graphs. To get this experience, we can install the Kubernetes dashboard.

kubectl apply -f \
    https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml

Since we use the secure RBAC role model, we need some more configuration files. First of all, we need a ServiceAccount and a ClusterRoleBinding.

cat > account.yaml << EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kube-system
EOF
cat > rolebinding.yaml << EOF
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kube-system
EOF
kubectl apply -f account.yaml
kubectl apply -f rolebinding.yaml

We can now fetch the secret token from kube-system.

kubectl -n kube-system describe secret \
    $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}') \
    | grep "token:"

token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.ey...

Copy the token to your clipboard, we can now forward the Kubernetes API to our development machine (I assume you have copied the kubeconf to your local machine, you may need SSH port forwarding else).

kubectl proxy

You can reach the dashboard via a very long pretty URL. Use the token retrieved from above to sign in.

Congratulations, you actually set up your own local Kubernetes cluster! Have fun experimenting and always remember to sail safe.

If you want to add more nodes to your cluster, repeat all steps until you reach kubeadm init. Use the kubeadm join printed out when you set up your master and you are ready to go.

You can find the codebase of this introductory series on GitHub.

Extending the blueprint

Well, we have a nice blueprint of the handlers we want, but shoving around global variables is neither fun nor good style. So we will extend each of our handlers to use Go’s support of higher-order functions, that allows us to use functions as values. We will do this for each of our handlers. Our final result should look something like this. Notice the changes in the main method when registering the handlers.

// import ...

func listCitiesHandler(service weather.Service) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
	    fmt.Fprintln(w, "/city")
    }
}

func showTemperatureHandler(service weather.Service) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintln(w, "/city/")
    }
}

func sendReportHandler(service weather.Service) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintln(w, "/report")
    }
}

func main() {
    service := weather.NewInMemoryService()
    http.HandleFunc("/city", listCitiesHandler(service))
    http.HandleFunc("/city/", showTemperatureHandler(service))
    http.HandleFunc("/report", sendReportHandler(service))
    // if err := ...
    // ...
}

Listing the stations

Let’s focus on the first handler on our blueprint, the city listing. To remind you of our handler description, we want something like the following.

Example: Requesting a list of stations

GET /city => {
    "munich",
    "zurich",
    "oberjoch"
}

When we look up the documentation of the weather package we imported, we will learn that calling service.Cities would give us a slice of city names. But since we want the list in a JSON format, we will have to encode it. Luckily, Go provides a dead-simple package for serializing and deserializing structures from/to JSON called encoding/json.

We can divide our job to write the list of cities to the HTTP response into three basic steps:

• fetch the cities string slice
• construct a new json.Encoder for our ResponseWriter
• use our json.Encoder to serialize the string slice (and ignore the error return value for now)

This literally translates into the following Go code.

cities := service.Cities()
json.NewEncoder(w).Encode(cities)

Since this is all of the needed business logic in our listCitiesHandler handler, we are pretty much done here except for error handling.

func listCitiesHandler(service weather.Service) http.HandleFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        if r.Method != "GET" {
            http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
            return
        }
        cities := service.Cities()
        encoder := json.NewEncoder(w)
        json.NewEncoder(w).Encode(cities)
    }
}

Luckily, again, Go provides another set of comfortable methods and constants to propagate errors back to the user using HTTP. The first case to handle is when the user does request our listCitiesHandler service using the wrong HTTP method (something absurd like POST) and since we provide a strict REST API we shall not honor these fools with a valid response. To show them the bad things they have done, we respond instead with a nice method not allowed status and text using the http.Error method. It accepts a ResponseWriter, a plain text error message and the HTTP error code as parameters.

A similar thing happens at the end of our handler when we fail on ourselves by somehow being not able to properly encode a list of strings.

Accessing temperature data

Our API users are now able to access a list of cities which have reported temperatures associated with. Well, of course, the user wants to know the current temperature of a specific station. This is exactly, what our showTemperatureHandler has to do.

1. Receive a station parameter from the user
2. Lookup station data
3. Convert to requested unit
4. Print out temperature

This is a rather massive procedure, but we can easily decompose it into subproblems. First of all, how can we take a station or city parameter from the user? We can simply use the HTML URI for that one!

city := strings.TrimPrefix(r.URL.Path, "/city/")

Using URL.Path we can get the request path and strip away the unwanted path prefix with the standard library method strings.StripPrefix. Since the unit of measurement is still missing and we already know how to extract data from web URIs we chould choose to do the same for our optional parameter. But careful: this would be extremely imprecise! We rather want to use a URI query parameter for our case.

unit := r.URL.Query()["unit"]
if unit == nil {
    // provide default value
    unit = []string{"celsius"}
}

We now got a string describing our unit of measurement, but we need to convert it into a discrete constant.

This way, we can easily provide a default value in case the tag is not provided and make use of the symbolic language of a question mark, visually marking the tag as optional.

Parsing the temperature unit

The rest of the handler is very similar to the routine of the listCitiesHandler but instead of serializing a struct from a library, we will convert the temperature using a method called parseTemperatureUnit and use a custom response struct.

func parseTemperatureUnit(unit string) (weather.Unit, bool) {
	switch strings.ToLower(unit) {
	case "kelvin":
		return weather.Kelvin, true
	case "celsius":
		return weather.Celsius, true
	case "fahrenheit":
		return weather.Fahrenheit, true
	}
	return weather.Kelvin, false
}

We use a standard Go switch statement to match the unit. If we are not able to find a perfect match, we default to using Kelvin units. Now we can finally parse our unit parameter and fetch the report from the datastore!

requestedUnit, ok := parseTemperatureUnit(unit[0])
if !ok {
    http.Error(w, "unknown unit of measurement", http.StatusBadRequest)
    return
}
temp, err := service.TemperatureIn(city, requestedUnit)
if err != nil {
    http.Error(w, "city not found", http.StatusNotFound)
    return
}

Installing the go command line tool

Before you can start writing Go code, you need the go command line tool that bundles the compiler and a bunch of other development tools for you. Just visit golang.org/dl/ to fetch the latest version and install it on your workstation. Be sure to yet your $GOPATH correctly!

Printing out the holy mantra

Let start with something simple - why not hello world? Create a new file called hello.go or whatever you want to name it and type the following code in your editor of choice.

package main

import "fmt"

func main() {
	fmt.Println("hello, world")
}

If you want to run your program, just enter go run hello.go to execute it. Pretty straightforward, huh? Let us dive a bit deeper into the structure of your first Go program. Before you continue with this post, I recommend to get a bit more accustomed to the language by doing the Go tour.

Dive into the magic

In the very first line, we declare a package. Packages let us bundle batches of files into one component that lie under a common namespace. Since we want to run our piece of work, it must be in the package main to be executable.

In the second line, we import a package called fmt (spoken like format or fomt, you decide). It is a collection of methods and structures to help us print stuff to stdout and other data streams. In our program we will only use fmt.Println which name is very descriptive: it just prints its arguments separated by spaces and adds a new line to the end.

In the third line, we declare a function. I assumed at the top that you are not completely new to the concepts of programming, so it is pretty much the same thing as in every other language you have used so far. We call the function main, so it will be called right when we run our code. Since we finished the holy mantra, we are blessed to go on with our journey. Now we will start building real server shit!

Designing a simple API

As I mentioned above, the power of the language and its standard library enables us to build server systems insanely fast. We will abuse some of that power and write a simple REST API for a community-run weather service. A client will be able to report and fetch current temperature values for a set of cities.

Example: Report measured temperature in Munich

$ curl -X POST -d @- localhost:8080/reports
{
	"city": "munich",
	"temperature": -12.0,  
	"unit": "celsius"
}
OK

Example: List locations with available reports

$ curl -X GET localhost:8080/reports
["munich"]

Example: Get the latest report from Munich

$ curl -X GET localhost:8080/reports/munich
{"city":"munich","temperature":-12,"unit":"celsius","timestamp":1519520293}

Organizing packages

To make things a bit cleaner, we want to organize our API using the $GOPATH. If you have a GitHub account, you can easily publish packages by setting up a repository in the $GOPATH directory. Let’s assume you have this awesome library called awesome-lib located in $GOPATH/src/github.com/yourgithubaccount/awesome-lib and you have published it under the same repository name on GitHub, everyone can import your library using the statement import "github.com/yourgithubaccount/awesome-lib". There is no need for a bloaty package manager!

We also want to do it this way and create a new folder called $GOPATH/src/github.com/yourgithubaccount/weather-api. In this folder, we once again create a new file called main.go with the following content. You can look up the current snapshot of the package on my GitHub account.

A first blueprint

package main

import (
	"fmt"
	"net/http"

    "github.com/gorilla/mux"
	"github.com/lnsp/go-demos/weather"
)

type weatherAPI struct {
    backend *weather.Service
    mux *mux.Router
}

func (api *weatherAPI) ListCities(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintln(w, `["munich"]`)
}

func (api *weatherAPI) CreateReport(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintln(w, "OK")
}

func (api *weatherAPI) ShowTemperature(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintln(w, `{"city":"munich","temperature":-12,"unit":"celsius","timestamp":1519520293}`)
}

func (api *weatherAPI) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    api.mux.ServeHTTP(w, r)
}

func newWeatherAPI(backend weather.Service) *weatherAPI {
    api := &weatherAPI{
        backend: backend,
        mux: mux.NewRouter(),
    }
    api.mux.HandleFunc("/reports", api.ListCities).Methods("GET")
    api.mux.HandleFunc("/reports", api.CreateReport).Methods("POST")
    api.mux.HandleFunc("/reports/{city}", api.ShowTemperature).Methods("GET")
    return api
}

func main() {
    backend := weather.NewInMemoryService()
    handler := newWeatherAPI(backend)
	if err := http.ListenAndServe(":8080", handler); err != nil {
		panic(err)
	}
}

Installing dependencies

Since we have external dependencies this time, you have to run go get -v to install these. If you want to test the system, just run go run main.go in your terminal, open a second tab and run curl localhost:8080/reports to send a request to your server. curl is an awesome tool to easily test simple APIs and we will continue to use it in this blog post.

Inspecting the blueprint

We see once again the package main statement and once again it means that we want to this piece of code to be executable. But this time, the import statement is different. Instead of a single package we give it a list of packages. These combined statements help us to visually group imports and keep our code clean and simple.

In the second import line, there is a package we don’t know about yet: net/http. It provides a complete HTTP stack including an web server, HTTPS / HTTP2 support and a client implementation for sending out requests.

The third and fourth import lines define a third-party package, a package that is not included in the standard library. In our case, we import one of my published packages called weather (the last part of the fully-qualified import path) and the wonderful mux package by the Gorilla project. The weather package was written for this tutorial and exposes a collections of structures that will provide the “business logic” for this project (mostly a simple in-memory datastore).

Spawning an API instance

To easily be able to plug different implementations of the weather service backend into our RESTful API frontend, we create a structure that contains our systems dependencies called weatherAPI. This structure also implements the http.Handler interface, so we can just put it into the handler field of http.ListenAndServe. On instance creation, we have to make sure that our routes are connected in the right way – using the declarative syntax of mux.Router to only allow specific kinds of requests.

The first two http.HandleFunc calls are completely standard, nothing off the ordinary. In the third route we use something special called a route variable. These are provided by the mux.Router and allow us to easily extract parameters from a requests URI path.

About request handlers

The methods thrown into our router are pretty similar, all require a http.ResponseWriter and a *http.Request as parameters. This function signature is defined in the package http as the type http.HandlerFunc. Since Go supports first-class functions, these functions can be thrown around just like another variable or constant. We can use their special signature to directly wire them to our web server.

All of the three handlers have a special role assigned.

• ListCities, sends back a list of all available measurement locations
• ShowTemperature, sends back the last measured temperature at the specified place
• CreateReport, accepts new temperature measurement reports from client

In each handler we use the legless sibling of the fmt.Println method we used earlier called fmt.Fprintln. The only difference between these two is that the fmt.Fprintln version takes an implementation of the io.Writer interface as the first parameter. Luckily, the http.ResponseWriter instance we received from our server implements this exact interface.

In the main method we first initialize our package variable service using a method from the weather package called NewInMemoryService which provides a simple in-memory implementation of the weather.Service interface. Then we call newWeatherAPI, it instantiates a new API handler, sets up the needed routes and returns it.

In case you don’t have it discovered yet: every reference in this post mentioning a library package or an item in one of theses packages, is a link to the matching documentation. Go provides excellent tooling including a documentation tool called godoc. godoc.org, a hosted version of the godoc tool, provides documentation for each publicly available package.

Moving on

I recommend to hack a bit around using the code above. If you don’t have any ideas, here are some of mine.

• temper with the output, let it print other messages the writer
• add an additional handler

In the second part of this article series, we will learn about error handling, more standard library packages and JSON encoding and decoding.