Scratch space using tmpfs and friends

How I use tmpfs for my scratch space
Published on July 20, 2018 under the tag tmpfs, overlayfs

When programming I tend to generate many files that I don’t want to save persistently on my storage media. Pdf documents I download, github repositories I clone to check something in source code and many other types of files should be stored somewhere but I would be glad to get rid of them when I power off my laptop.

For this purpose I use the following script which creates tmpfs memory backed file system which is mounted at ~/scratch. I use this directory to store all temporary files and I know they will disappear after I reboot.

#!/bin/sh
sudo mount -t tmpfs -o size=8G none ~/scratch/

Another use case is when I want to temporarily install some packages to try new stuff. For example if I want to compile some haskell application using stack. Stack is going to pull and compile many new packages and it will store them at ~/.stack. If I don’t want to persistently store all these packages on my storage media, I can do the following trick. Create ~/scratch/stack-shadow directory. Use overlayfs to create new file system that mirrors the old directory and stores all modified or newly created files in new directory. Using mount -o bind to bind the overlayfs directory to the ~/.stack location

mkdir -p ~/scratch/stack-shadow/upperdir \
         ~/scratch/stack-shadow/workdir  \
         ~/scratch/stack-shadow/export

sudo mount -t overlay overlay \
           -olowerdir=~/.stack \
           -oupperdir=~/scratch/stack-shadow/upperdir \
           -oworkdir=~/scratch/stack-shadow/workdir  \
           ~/scratch/stack-shadow/export

sudo mount -o bind ~/scratch/stack-shadow/export ~/.stack

We can build a bash script to automate these commands.

#!/bin/bash

function usage {
    echo "usage: $0 <command> <src> <dst>"
    echo "commands:"
    echo -e "\tnew - bind dst to location at src"
    echo -e "\tshadow  - overlay src to dst, bind dst to src"
    echo -e "\tdup - overlay src to dst"
    echo -e "\tclear - umount src and dst"
    exit 1
}

function new {
    local src=$1
    local dst=$2
    mkdir -p $dst
    sudo mount -o bind $dst $src
}

function shadow {
    local src=$1
    local dst=$2
    dup $src $dst
    sudo mount -o bind $dst/export $src
}

function dup {
    local src=$1
    local dst=$2
    mkdir -p $dst/upperdir $dst/workdir $dst/export
    sudo mount -t overlay overlay -olowerdir=$src -oupperdir=$dst/upperdir -oworkdir=$dst/workdir $dst/export
}

function clear {
    local src=$1
    local dst=$2
    test -d $src && sudo umount $src
    test -d $dst/export && sudo umount $dst/export
}

if [ $# -lt 3 ]; then usage; fi

set -x

case "$1" in
    new)
        new $2 $3
        ;;
    shadow)
        shadow $2 $3
        ;;
    dup)
        dup $2 $3
        ;;
    clear)
        clear $2 $3
        ;;
    *)
        usage
esac