Shrinking a Windows 10 installation for small harddrives

Cheap Windows 10 capable devices often come with very limited internal storage space. To make use of this, it is of course crucial that Windows itself takes up as little space as possible. To combat the bloat, there’s a little known feature in compact.exe, the built-in disk compression tool, that compresses the operating system itself, often saving several of those valuable gigabytes.

Microsoft does have some documentation on this feature, right here.

Basically, all you do is open cmd.exe as an administrator and run:

cd %windir%\system32
Compact.exe /CompactOS:always

You’ll see something like this:

To know whether or not CompactOS is enabled on your Windows installation, issue the following commands:

cd %windir%\system32
Compact.exe /CompactOS:query

Whoa! Look at all that space, mom!

Checking for required packages on a Debian (or derivative) system

Tasked with cleaning up an installation script, I noticed it was calling dpkg dozens of times to verify that the required packages were in place. On a Raspberry Pi, or similar low powered device, this takes ages. I replaced it with the following:

c=0
while read -r pkg; do
    printf 'Missing package: %s\n' "$pkg" >&2
    (( ++c ))
done < <(dpkg-query -W -f='${binary:Package}\n' | cut -d ':' -f 1 | sort | comm -13 - <(sort <<-EOF
    libc-ares2
    libssl1.0.0
    zliblg
    mosquitto
    gdb
    tcpdump
    vim
EOF
))
printf 'Total missing packages: %d\n' "$c" >&2

In short:

  • We query dpkg once, listing every installed package, cut off any version numbers and sort the list
  • We sort a list of manually specified required packages (libc-ares2 … vim)
  • We compare the sorted lists (sorting is a requirement for comm), specifying that we want to not display columns 1 and 3, which are lines unique to the left side (installed ones), and lines that appear on both sides (installed and required), respectively. This leaves colum 2, the ones unique to the right side, which are the required packages missing from dpkg’s list.

Sample output:

$ ./check
Missing package: gdb
Missing package: mosquitto
Missing package: zliblg
Total missing packages: 3

Archived here for future reference, and in case it’s useful to anyone 🙂

Avoid having Visual Studio put all its floating windows on top

Working with multiple screens, I often use several windows in Visual Studio. For a while now, I’ve been annoyed by all the miscellaneous windows forcing themselves on top so I couldn’t open a browser on one screen and switch to the code on another screen without the window on the browser’s screen popping up to obscure the browser.

I finally took the time to figure this one out, and the answer is right there in the options menu. Deactivate both of the highlighted options and your windows are free to move around as you wish. They even get their own icon on the task bar, if you have a task bar on all your screens!

Sleeping without a subprocess in Bash, and how to sleep forever

No subprocess, no sleep command, no coproc, no nothing? Yes.

Sleeping in bash script is traditionally done with the sleep (1) command, which is external to bash, the command /bin/sleep. However, if you have a bunch of scripts running that all sleep this way, the output of ps looks like a mess, pstree looks like a bigger mess, and every OCD sensor in my brain goes off.

Sample output of pstree:

$ sudo pstree -ps 2828
systemd(1)───urxvt(2826)───bash(2828)───bash(14252)───sleep(14255)

Here, my terminal (urxvt) runs a shell (bash, 2828), that runs a test script (bash, 14252), that runs sleep (14255).

Several bad ideas

This post on Stack Exchange contains plenty of horrible proposed solutions, but does also point out that several distributions of Linux ship a package with loadable bash modules. Among them is an internal sleep command. I didn’t want to rely on that, however.

Stack Overflow has a post on how to sleep forever. Again there are several horrendous ideas, but the answer by Tino is rather clever:

bash -c 'coproc { exec >&-; read; }; eval exec "${COPROC[0]}<&-"; wait'

coproc is a relatively new feature, however, and it uses eval, which, as wooledge.org points out, is a common misspelling of “evil”. We can do better.

Finally asleep

Waiting to read a file descriptor that will never output anything is a clever solution, but we can achieve that without using coproc. instead opting for good old fashioned process substitution.

So I wrote the following function:

snore()
{
    local IFS
    [[ -n "${_snore_fd:-}" ]] || { exec {_snore_fd}<> <(:) && read -r -t 0 -u $_snore_fd; } 2>/dev/null ||
    {
# workaround for MacOS and similar systems local fifo
fifo=$(mktemp -u) mkfifo -m 700 "$fifo" exec {_snore_fd}<>"$fifo" rm "$fifo" } read ${1:+-t "$1"} -u $_snore_fd || : }

So what does that do? Well, this:

local IFS Reset IFS in case it’s set to something weird.
[[ -n “${_snore_fd:-}” ]] Checks if the $_snore_fd variable has already been declared. If so, we are good to go. The :- is there to subtitute with an empty string in case you’re using “set -eu”, which would exit with an error if the variable wasn’t set already.
exec {_snore_fd}<> Assigns the next available file descriptor to the “_snore_fd” variable. “_snore_fd” will be a number signifying the assigned file descriptor after this.
<(:) Process substitution: reading from a subshell that simply runs “:”, or “true” if you will, and then exits
read Attempts to read input, though it won’t get any
${1:+-t “$1”} Parameter expansion: If the snore() function was provided a parameter, it will pass it along to read as an argument for -t (timeout).
If no parameters were provided, -t will not be specified, and read will hang forever.
-u $_snore_fd Specifies that read should use the value of $_snore_fd as its input file descriptor
|| : Making sure read returns 0, for coding with -e set. This will run : if read fails, and : always returns 0.

Let’s test it!

Here’s a short script to compare the efficiency of snore() to that of /bin/sleep. It runs each operation 1000 times, for a total of what should be 10 seconds for each.

#!/usr/bin/env bash
set -u
set -e

snore()
{
    local IFS
    [[ -n "${_snore_fd:-}" ]] || exec {_snore_fd}<> <(:)
    read ${1:+-t "$1"} -u $_snore_fd || :
}

time for ((i=0; i<1000; i++)); do snore 0.01; done
time for ((i=0; i<1000; i++)); do sleep 0.01; done

The snore() function runs faster than /bin/sleep, at least on my system. That’s not to say it sleeps too quickly – one second is still one second – but if called in quick succession, one can see that the snoring loop is faster than the sleeping one:

$ /tmp/test </dev/null

real	0m10.226s
user	0m0.144s
sys	0m0.036s

real	0m11.674s
user	0m0.060s
sys	0m0.232s

As you can see, calling snore() 1000 times has a combined overhead of 0.226 seconds, while /bin/sleep measured 1.674 seconds. This is of course utterly insignificant in real world applications, but it’s interesting none the less.

No more sleep processes

Aside from the completely insignificant performance differences, my OCD was satisfied, as a script running snore() has no child process to wait for, and the subshell we spawn (once) disappears immediately. Here’s pstree while I run a script that snores:

$ sudo pstree -ps 2828
systemd(1)───urxvt(2826)───bash(2828)───bash(19247)

So my terminal runs a shell, and that shell runs the script, but there’s no sleep call, and no other subprocess. There’s simply the interactive shell waiting for the script. Excellent.

As an added bonus, there will no longer be any of the usual issues of various sleep processes hanging around after killing processes, or preventing them from being killed in the first place.

Halt and Catch Fire

Going back to the question on stack overflow, you may have noticed the parameter processing of snore() allowing for no parameters to be passed. This means that if you don’t pass any parameters to snore(), -t (timeout) will not be specified for the call to read, and read will hang forever. I don’t know why you’d want this, but now you can.

Update (June 12, 2019)

Added a workaround for MacOS and similar systems, using a short-lived FIFO to read from (only created on the first call to snore()).

Update (July 23, 2023)

Added a workaround for Bash 5.2 (thank you, anonymous commenter, for letting me know about the issue). Bash 5.2 would let the subprocess spawned by <(:) linger until the first read call completed. By using read -r -t 0 on it initially, this subprocess should go away immediately and not clutter up process lists, as intended.

Making Windows keep the system clock in UTC

Some hypervisors for virtual machines do not properly support sending a fake system time to the guest operating system, thus making Windows guests display the wrong time if their timezone is set to anything except UTC. This happens because Windows, by default, keeps the system clock set to the local time. This is stupid.

The same problems can occur on dual-booted computers, for instance where Windows and Linux attempt to co-exist on the same hardware. Linux will, unless told to do otherwise, set the system clock to UTC, and Windows will keep changing it to whatever the local time is. Linux can of course be told to keep the system time in the local time zone, but a less known feature of Windows allows you to do the opposite.

The magic registry key is HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\TimeZoneInformation\RealTimeIsUniversal

Create a new 32-bit DWORD and set it to 1, then reboot.

There’s exhaustive reading material on the subject here (local archive) if you’re interested.

Finding the most expensive recent SQL queries on SQL Server

Mostly a note to self, original source here.

SELECT TOP 10 SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,
((CASE qs.statement_end_offset
WHEN -1 THEN DATALENGTH(qt.TEXT)
ELSE qs.statement_end_offset
END - qs.statement_start_offset)/2)+1),
qs.execution_count,
qs.total_logical_reads, qs.last_logical_reads,
qs.total_logical_writes, qs.last_logical_writes,
qs.total_worker_time,
qs.last_worker_time,
qs.total_elapsed_time/1000000 total_elapsed_time_in_S,
qs.last_elapsed_time/1000000 last_elapsed_time_in_S,
qs.last_execution_time,
qp.query_plan
FROM sys.dm_exec_query_stats qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qt
CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp
ORDER BY qs.total_logical_reads DESC -- logical reads
-- ORDER BY qs.total_logical_writes DESC -- logical writes
-- ORDER BY qs.total_worker_time DESC -- CPU time

Creating a local Certificate Authority using OpenSSL

I was recently tasked with creating a local CA for a project, where we needed to verify custom client certificates, have the ability to revoke them at will, and we wanted to add additional custom fields to the certificates. Cool.

The first stop after searching a bit was this excellent howto by Jamie Nguyen. There’s a local mirror here.

Frankly, the only thing Jamie doesn’t go into detail about is how to add custom properties to, in my case, client certificates. Dustin Oprea has a write-up on this here (mirror).