Shrinking a Raspbian installation and re-enabling auto expanding for distribution of customized images

UPDATE: See the automated script at https://blog.dhampir.no/content/script-for-creating-a-compressed-image-file-from-a-raspbian-sd-card

Raspbian, by default, expands to fill the SD card it finds itself on, the first time it boots. After having customized an image to your liking it would be favourable to avoid copying 16 gigabytes of data, or however large your chosen SD card is, each time you want to duplicate your setup. So let’s go through some simple steps to reduce the size of the resulting image file.

The Works

  1. Prepare the Raspbian image by re-enabling auto expanding
    1. Edit /boot/cmdline.txt and append init=/usr/lib/raspi-config/init_resize.sh
      In my case, this meant replacing

      dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=PARTUUID=1ba1cea3-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait logo.nologo net.ifnames=0

      With

      dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=PARTUUID=1ba1cea3-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait logo.nologo net.ifnames=0 init=/usr/lib/raspi-config/init_resize.sh

      This takes care of resizing the partition, but not the file system.

    2. Grab the resize2fs_once script from git (or from my archive copy here) and enable it
      $ sudo wget -O /etc/init.d/resize2fs_once https://github.com/RPi-Distro/pi-gen/raw/dev/stage2/01-sys-tweaks/files/resize2fs_once
      $ sudo chmod +x /etc/init.d/resize2fs_once
      $ sudo systemctl enable resize2fs_once

      This will expand the file system to match the resized partition on the first boot.

  2. Cleanly shut down the Raspberry Pi
  3. Put the SD card in a card reader on a Linux machine. The machine can be your Raspberry Pi, as long as you’re booting from another SD card.
    The following steps assume you are not currently booting from the SD card you want to modify.
  4. Confirm the device name with dmesg
    $ dmesg
    ...
    [1314286.573659] mmc0: new ultra high speed SDR50 SDHC card at address 59b4
    [1314286.574319] mmcblk0: mmc0:59b4 USDU1 14.7 GiB 
    [1314286.575783] mmcblk0: p1 p2
  5. Launch gparted, or a CLI tool if you prefer, and shrink the EXT4 file system and partition. I usually leave a hundred megs or so of free space to avoid issues with programs that write stuff on the first boot.

    Close gparted.
  6. Optional extra #1: You don’t have to do this step. Skip ahead if you want.
    If you want, this would be the time to defragment the file system, as any fragmentation will be written as is to the next card.

    $ mkdir /tmp/sd_root
    $ sudo mount /dev/mmcblk0p2 /tmp/sd_root
    $ sudo e4defrag /dev/mmcblk0p2
    ext4 defragmentation for device(/dev/mmcblk0p2)
    [7/51184]/tmp/sd_root/etc/dhcp/debug:       100%     [OK]
    [14/51184]/tmp/sd_root/sbin/mntctl:         100%     [OK]
    [310/51184]/tmp/sd_root/sbin/killall5:      100%     [OK]
    ............
    
        Success:          [ 40336/51184 ]
        Failure:          [ 10848/51184 ]
    $ sudo umount /tmp/sd_root
    $ sudo rmdir /tmp/sd_root

    Note that some failures are expected. This is normal.

  7. Optional extra #2: You don’t have to do this step. Skip ahead if you want.
    If you want your image to compress extremely well, you can at this point mount the image and zero fill the free space you left. Large chunks of zeros are exceptionally easy to compress.

    $ mkdir /tmp/sd_root
    $ sudo mount /dev/mmcblk0p2 /tmp/sd_root
    $ sudo dd if=/dev/zero of=/tmp/sd_root/delete.me
    dd: writing to '/tmp/sd_root/delete.me': No space left on device
    41110+0 records in
    41109+0 records out
    21047808 bytes (21 MB, 20 MiB) copied, 0.437186 s, 48.1 MB/s
    $ sudo rm /tmp/sd_root/delete.me
    $ sudo umount /tmp/sd_root
    $ sudo rmdir /tmp/sd_root
  8. Use fdisk -l or a similar command to find the end of the resized partition
    $ sudo fdisk -l /dev/mmcblk0
    Disk /dev/mmcblk0: 14.7 GiB, 15753805824 bytes, 30769152 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: dos
    Disk identifier: 0x44b1ee6a
    
    Device Boot Start End Sectors Size Id Type
    /dev/mmcblk0p1 8192 93486 85295 41.7M c W95 FAT32 (LBA)
    /dev/mmcblk0p2 94208 3985407 3891200 1.9G 83 Linux

    This partition ends at block 3985407, and the block size is 512 bytes. We’ll need these numbers.

  9. Use dd to copy the image to where you want it. Here we use the values from the previous step, but add 1 to the number of blocks, as blocks are 0-indexed. To clarify, the first block is block 0, so if the last partition ended at block 5, the full image would be 6 blocks long.
    $ sudo dd if=/dev/mmcblk0 of=/tmp/my_raspbian.img bs=512 count=3985408
  10. Compress the image with gzip, zip, 7z, or whatever tool you fancy, put it somewhere and tell your friends 🙂

2 Comments

  • Shalom Crown says:

    Hi!
    After filling free space with zeroes, you need to run ‘sync’ to make sure it is all actually written to the SD card.

    • bolt says:

      This is not needed. As per the spec, unmounting the file system will first complete all pending write operations. Also, since we’re reading the image back, whether we’re reading from cache or the disk itself doesn’t really matter, so even if the data wasn’t on disk for some reason, it would still compress well.

Leave a Reply

Your email address will not be published. Required fields are marked *