How to create a custom Ubuntu live from scratch

This procedure shows how to create a bootable and installable Ubuntu Live (along with the automatic hardware detection and configuration) from scratch.

Prerequisites (GNU/Linux Debian/Ubuntu)

Install applications we need to build the environment.

sudo apt-get install \
    binutils \
    debootstrap \
    squashfs-tools \
    xorriso \
    grub-pc-bin \
    grub-efi-amd64-bin \
    mtools

mkdir $HOME/live-ubuntu-from-scratch

Bootstrap and Configure Ubuntu

• Checkout bootstrap

  sudo debootstrap \
     --arch=amd64 \
     --variant=minbase \
     focal \
     $HOME/live-ubuntu-from-scratch/chroot \
     http://us.archive.ubuntu.com/ubuntu/
  

  debootstrap is used to create a Debian base system from scratch, without requiring the availability of dpkg or apt. It does this by downloading .deb files from a mirror site, and carefully unpacking them into a directory which can eventually be chrooted into.

• Configure external mount points

  sudo mount --bind /dev $HOME/live-ubuntu-from-scratch/chroot/dev
  
  sudo mount --bind /run $HOME/live-ubuntu-from-scratch/chroot/run
  

  As we will be updating and installing packages (grub among them), these mount points are necessary inside the chroot environment, so we are able to finish the installation without errors.

Define chroot environment

A chroot on Unix operating systems is an operation that changes the apparent root directory for the current running process and its children. A program that is run in such a modified environment cannot name (and therefore normally cannot access) files outside the designated directory tree. The term "chroot" may refer to the chroot system call or the chroot wrapper program. The modified environment is called a chroot jail.

Reference: https://en.wikipedia.org/wiki/Chroot

1. Access chroot environment

   sudo chroot $HOME/live-ubuntu-from-scratch/chroot
   

2. Configure mount points, home and locale

   mount none -t proc /proc
   
   mount none -t sysfs /sys
   
   mount none -t devpts /dev/pts
   
   export HOME=/root
   
   export LC_ALL=C
   

   These mount points are necessary inside the chroot environment, so we are able to finish the installation without errors.

3. Set a custom hostname

   echo "ubuntu-fs-live" > /etc/hostname
   

4. Configure apt sources.list

   cat <<EOF > /etc/apt/sources.list
   deb http://us.archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse
   deb-src http://us.archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse
   
   deb http://us.archive.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse
   deb-src http://us.archive.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse
   
   deb http://us.archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse
   deb-src http://us.archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse
   EOF
   

5. Update indexes packages

   apt-get update
   

6. Install systemd

   apt-get install -y libterm-readline-gnu-perl systemd-sysv
   

   systemd is a system and service manager for Linux. It provides aggressive parallelization capabilities, uses socket and D-Bus activation for starting services, offers on-demand starting of daemons, keeps track of processes using Linux control groups, maintains mount and automount points and implements an elaborate transactional dependency-based service control logic.

7. Configure machine-id and divert

   dbus-uuidgen > /etc/machine-id
   
   ln -fs /etc/machine-id /var/lib/dbus/machine-id
   

   The /etc/machine-id file contains the unique machine ID of the local system that is set during installation or boot. The machine ID is a single newline-terminated, hexadecimal, 32-character, lowercase ID. When decoded from hexadecimal, this corresponds to a 16-byte/128-bit value. This ID may not be all zeros.

   dpkg-divert --local --rename --add /sbin/initctl
   
   ln -s /bin/true /sbin/initctl
   

   dpkg-divert is the utility used to set up and update the list of diversions.

8. Upgrade packages

   apt-get -y upgrade
   

9. Install packages needed for Live System

   apt-get install -y \
       ubuntu-standard \
       casper \
       lupin-casper \
       discover \
       laptop-detect \
       os-prober \
       network-manager \
       resolvconf \
       net-tools \
       wireless-tools \
       wpagui \
       locales \
       linux-generic
   

   The next steps will appear, as a result of the packages that will be installed from the previous step, this will happen without anything having to be informed or executed.

   1. Configure grub

      

   2. Don’t select any options

      

   3. Only confirm “Yes”

      

10. Graphical installer

    apt-get install -y \
       ubiquity \
       ubiquity-casper \
       ubiquity-frontend-gtk \
       ubiquity-slideshow-ubuntu \
       ubiquity-ubuntu-artwork
    

    The next steps will appear, as a result of the packages that will be installed from the previous step, this will happen without anything having to be informed or executed.

    1. Configure keyboard

       

       

    2. Console setup

       

11. Install window manager

    apt-get install -y \
        plymouth-theme-ubuntu-logo \
        ubuntu-gnome-desktop \
        ubuntu-gnome-wallpapers
    

12. Install useful applications

    apt-get install -y \
        clamav-daemon \
        terminator \
        apt-transport-https \
        curl \
        vim \
        nano \
        less
    

13. Install Visual Studio Code (optional)

    1. Download and install the key

       curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg
       
       install -o root -g root -m 644 microsoft.gpg /etc/apt/trusted.gpg.d/
       
       echo "deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main" > /etc/apt/sources.list.d/vscode.list
       
       rm microsoft.gpg
       

    2. Then update the package cache and install the package using

       apt-get update
       
       apt-get install -y code
       

14. Install Google Chrome (optional)

    1. Download and install the key

       wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
       
       echo "deb http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list
       

    2. Then update the package cache and install the package using

       apt-get update
       
       apt-get install google-chrome-stable
       

15. Install Java JDK 8 (optional)

    apt-get install -y \
        openjdk-8-jdk \
        openjdk-8-jre
    

16. Remove unused applications (optional)

    apt-get purge -y \
        transmission-gtk \
        transmission-common \
        gnome-mahjongg \
        gnome-mines \
        gnome-sudoku \
        aisleriot \
        hitori
    

17. Remove unused packages

    apt-get autoremove -y
    

18. Reconfigure packages

    1. Generate locales

       dpkg-reconfigure locales
       

       1. Select locales

          

       2. Select default locale

          

    2. Reconfigure resolvconf

       dpkg-reconfigure resolvconf
       

       1. Confirm changes

          

          

          

    3. Configure network-manager

       cat <<EOF > /etc/NetworkManager/NetworkManager.conf
       [main]
       rc-manager=resolvconf
       plugins=ifupdown,keyfile
       dns=dnsmasq
       
       [ifupdown]
       managed=false
       EOF
       

    4. Reconfigure network-manager

       dpkg-reconfigure network-manager
       

19. Cleanup the chroot environment

    1. If you installed software, be sure to run

       truncate -s 0 /etc/machine-id
       

    2. Remove the diversion

       rm /sbin/initctl
       
       dpkg-divert --rename --remove /sbin/initctl
       

    3. Clean up

       apt-get clean
       
       rm -rf /tmp/* ~/.bash_history
       
       umount /proc
       
       umount /sys
       
       umount /dev/pts
       
       export HISTSIZE=0
       
       exit
       

Unbind mount points

sudo umount $HOME/live-ubuntu-from-scratch/chroot/dev

sudo umount $HOME/live-ubuntu-from-scratch/chroot/run

Create the CD image directory and populate it

1. Access build directory

   cd $HOME/live-ubuntu-from-scratch
   

2. Create directories

   mkdir -p image/{casper,isolinux,install}
   

3. Copy kernel images

   sudo cp chroot/boot/vmlinuz-**-**-generic image/casper/vmlinuz
   
   sudo cp chroot/boot/initrd.img-**-**-generic image/casper/initrd
   

4. Copy memtest86+ binary (BIOS)

   sudo cp chroot/boot/memtest86+.bin image/install/memtest86+
   

5. Download and extract memtest86 binary (UEFI)

   wget --progress=dot https://www.memtest86.com/downloads/memtest86-usb.zip -O image/install/memtest86-usb.zip
   
   unzip -p image/install/memtest86-usb.zip memtest86-usb.img > image/install/memtest86
   
   rm -f image/install/memtest86-usb.zip
   

Grub configuration

1. Access build directory

   cd $HOME/live-ubuntu-from-scratch
   

2. Create base point access file for grub

   touch image/ubuntu
   

3. Create image/isolinux/grub.cfg

   cat <<EOF > image/isolinux/grub.cfg
   
   search --set=root --file /ubuntu
   
   insmod all_video
   
   set default="0"
   set timeout=30
   
   menuentry "Try Ubuntu FS without installing" {
      linux /casper/vmlinuz boot=casper nopersistent toram quiet splash ---
      initrd /casper/initrd
   }
   
   menuentry "Install Ubuntu FS" {
      linux /casper/vmlinuz boot=casper only-ubiquity quiet splash ---
      initrd /casper/initrd
   }
   
   menuentry "Check disc for defects" {
      linux /casper/vmlinuz boot=casper integrity-check quiet splash ---
      initrd /casper/initrd
   }
   
   menuentry "Test memory Memtest86+ (BIOS)" {
      linux16 /install/memtest86+
   }
   
   menuentry "Test memory Memtest86 (UEFI, long load time)" {
      insmod part_gpt
      insmod search_fs_uuid
      insmod chain
      loopback loop /install/memtest86
      chainloader (loop,gpt1)/efi/boot/BOOTX64.efi
   }
   EOF
   

Create manifest

In the next steps the creation of the manifest is important because it tells us which version of each package installed in the Live version and which packages will be removed or maintained in the version that will be installed (persisted in the hard drive).

1. Access build directory

   cd $HOME/live-ubuntu-from-scratch
   

2. Generate manifest

   sudo chroot chroot dpkg-query -W --showformat='${Package} ${Version}\n' | sudo tee image/casper/filesystem.manifest
   
   sudo cp -v image/casper/filesystem.manifest image/casper/filesystem.manifest-desktop
   
   sudo sed -i '/ubiquity/d' image/casper/filesystem.manifest-desktop
   
   sudo sed -i '/casper/d' image/casper/filesystem.manifest-desktop
   
   sudo sed -i '/discover/d' image/casper/filesystem.manifest-desktop
   
   sudo sed -i '/laptop-detect/d' image/casper/filesystem.manifest-desktop
   
   sudo sed -i '/os-prober/d' image/casper/filesystem.manifest-desktop
   

Compress the chroot

After everything has been installed and preconfigured in the chrooted environment, we need to generate an image of everything that was done by following the next steps.

1. Access build directory

   cd $HOME/live-ubuntu-from-scratch
   

2. Create squashfs

   sudo mksquashfs chroot image/casper/filesystem.squashfs
   

   Squashfs is a highly compressed read-only filesystem for Linux. It uses zlib compression to compress both files, inodes and directories. Inodes in the system are very small and all blocks are packed to minimize data overhead. Block sizes greater than 4K are supported up to a maximum of 64K. Squashfs is intended for general read-only filesystem use, for archival use (i.e. in cases where a .tar.gz file may be used), and in constrained block device/memory systems (e.g. embedded systems) where low overhead is needed.

3. Write the filesystem.size

   printf $(sudo du -sx --block-size=1 chroot | cut -f1) > image/casper/filesystem.size
   

Create diskdefines

README file often found on Linux LiveCD installer discs, such as an Ubuntu Linux installation CD; typically named “README.diskdefines” and may be referenced during installation.

1. Access build directory

   cd $HOME/live-ubuntu-from-scratch
   

2. Create file image/README.diskdefines

   cat <<EOF > image/README.diskdefines
   #define DISKNAME  Ubuntu from scratch
   #define TYPE  binary
   #define TYPEbinary  1
   #define ARCH  amd64
   #define ARCHamd64  1
   #define DISKNUM  1
   #define DISKNUM1  1
   #define TOTALNUM  0
   #define TOTALNUM0  1
   EOF
   

Create ISO Image for a LiveCD (BIOS + UEFI)

1. Access image directory

   cd $HOME/live-ubuntu-from-scratch/image
   

2. Create a grub UEFI image

   grub-mkstandalone \
      --format=x86_64-efi \
      --output=isolinux/bootx64.efi \
      --locales="" \
      --fonts="" \
      "boot/grub/grub.cfg=isolinux/grub.cfg"
   

3. Create a FAT16 UEFI boot disk image containing the EFI bootloader

   (
      cd isolinux && \
      dd if=/dev/zero of=efiboot.img bs=1M count=10 && \
      sudo mkfs.vfat efiboot.img && \
      LC_CTYPE=C mmd -i efiboot.img efi efi/boot && \
      LC_CTYPE=C mcopy -i efiboot.img ./bootx64.efi ::efi/boot/
   )
   

4. Create a grub BIOS image

   grub-mkstandalone \
      --format=i386-pc \
      --output=isolinux/core.img \
      --install-modules="linux16 linux normal iso9660 biosdisk memdisk search tar ls" \
      --modules="linux16 linux normal iso9660 biosdisk search" \
      --locales="" \
      --fonts="" \
      "boot/grub/grub.cfg=isolinux/grub.cfg"
   

5. Combine a bootable Grub cdboot.img

   cat /usr/lib/grub/i386-pc/cdboot.img isolinux/core.img > isolinux/bios.img
   

6. Generate md5sum.txt

   sudo /bin/bash -c "(find . -type f -print0 | xargs -0 md5sum | grep -v -e 'md5sum.txt' -e 'bios.img' -e 'efiboot.img' > md5sum.txt)"
   

7. Create iso from the image directory using the command-line

   sudo xorriso \
      -as mkisofs \
      -iso-level 3 \
      -full-iso9660-filenames \
      -volid "Ubuntu from scratch" \
      -eltorito-boot boot/grub/bios.img \
      -no-emul-boot \
      -boot-load-size 4 \
      -boot-info-table \
      --eltorito-catalog boot/grub/boot.cat \
      --grub2-boot-info \
      --grub2-mbr /usr/lib/grub/i386-pc/boot_hybrid.img \
      -eltorito-alt-boot \
      -e EFI/efiboot.img \
      -no-emul-boot \
      -append_partition 2 0xef isolinux/efiboot.img \
      -output "../ubuntu-from-scratch.iso" \
      -m "isolinux/efiboot.img" \
      -m "isolinux/bios.img" \
      -graft-points \
         "/EFI/efiboot.img=isolinux/efiboot.img" \
         "/boot/grub/bios.img=isolinux/bios.img" \
         "."
   

Make a bootable USB image

It is simple and easy, using "dd"

sudo dd if=ubuntu-from-scratch.iso of=<device> status=progress oflag=sync