This week I was debugging a misbehaving Python program that makes significant use of Python’s asyncio. The program would eventually take very long periods of time to respond to network requests. My first suspicion was a CPU-heavy coroutine hogging the thread, preventing the socket coroutines from running, but an inspection with pdb showed this wasn’t the case. Instead, the program’s author had made a couple of fundamental mistakes using asyncio. Let’s discuss them using small examples.

As a computer engineer, my job is to use computers to solve important problems. Ideally my solutions will be efficient, and typically that means making the best use of the resources at hand. Quite often these resources are machines running Windows and, despite my misgivings about the platform, there is much to be gained by properly and effectively leveraging it.

Sometimes targeting Windows while working from another platform is sufficient, but other times I must work on the platform itself. There are various options available for C development, and I’ve finally formalized my own development kit: w64devkit.

This article was discussed on Hacker News.

Suppose you’re writing a command line program that prompts the user for a password or passphrase, and Windows is one of the supported platforms (even very old versions). This program uses UTF-8 for its string representation, as it should, and so ideally it receives the password from the user encoded as UTF-8. On most platforms this is, for the most part, automatic. However, on Windows finding the correct answer to this problem is a maze where all the signs lead towards dead ends. I recently navigated this maze and found the way out.

This article was discussed on Hacker News.

I’ve noticed a small pattern across a few of my projects where I had vectorized and parallelized some code. The original algorithm had a “push” approach, the optimized version instead took a “pull” approach. In this article I’ll describe what I mean, though it’s mostly just so I can show off some pretty videos, pictures, and demos.

I had recently lamented that due to Go’s strict module security policy it was unreasonably difficult to experiment and practice with modules. Modules can only be fetched from servers with valid TLS certificates, including both the module path and repository servers. Setting up a small, local experiment meant creating a certificate authority, generating and signing certificates, and installing these all in the right places. I’d much rather relax Go’s security policy for the experiment.

As a result of that complaint, I learned that the upcoming Go 1.14 has as a new feature: GOINSECURE. It’s like the old -insecure option, but safer due to being finer grained: a whitelist of exceptions. It’s exactly what I needed. Since then I’ve been using it to run small module experiments. It started as some scripts, but I eventually formalized it into its own little project.

https://github.com/skeeto/go-module-testbed [requires Go 1.14]

Each of my Emacs packages has a Makefile to byte-compile all source files, run the tests, build a package file, and, in some cases, run the package in an interactive, temporary, isolated Emacs instance. These portable Makefiles have a similar structure and follow the same conventions. It would require more thought and feedback before I’d try to make it a standard, but these are conventions I’d like to see in other package Makefiles.

This article was discussed on Hacker News, on reddit, and on Lobsters.

Regardless of your opinions of the Go programming language, the primary implementation of Go, gc, is an incredible piece of software engineering. Everyone ought to be blown away by it! Yet not only is it undervalued in general, even the Go community itself doesn’t fully appreciate it. It’s not perfect, but it has unique features never before seen in a toolchain.

I saw (part of) a video, OS hacking: Purgeable memory, by Andreas Kling who’s writing an operating system called Serenity and recording videos his progress. In the video he implements purgeable memory as found on some Apple platforms by adding special support in the kernel. A process tells the kernel that a particular range of memory isn’t important, and so the kernel can reclaim if it the system is under memory pressure — the memory is purgeable.

This article was discussed on Hacker News and on reddit.

Despite the goal of JSON being a subset of JavaScript — which it failed to achieve (update: this was fixed) — parsing JSON is quite unlike parsing a programming language. For invalid inputs, the specific cause of error is often counter-intuitive. Normally this doesn’t matter, but I recently ran into a case where it does.

Suppose you don’t like the names car and cdr, the traditional identifiers for two halves of a lisp cons cell. This is misguided. A cons is really just a 2-tuple, and the halves don’t have any particular meaning on their own, even as “head” and “tail.” However, maybe this is really important to you so you want to do it anyway. What’s the best way to go about it?