Remaster a Sandbox

[s243a]

The differences between: build scripts, sandbox remaster, and normal remasters

This is someone between a remaster script and a build system in the spectrum of creating customized operating systems and derivatives. The idea of puppy's remaster script (i.e. remasterpup2) is for files/folders in /etc /var and /root to only be include in the ISO only files/folders that we know are safe to include otherwise we use the info from the original iso for the contents of these folders.

In contrast, in the build system we might take the entire root file system that we constructed both because:
1. we know what we put there
2. we likely didn't add any private info to any of these folders (e.g. ssh keys)
3. the build system is tested multiple times so if there is anything that we need to remove we are likely to know what it is.

In a remaster script it is okay, to leave out a lot of info from the folders /etc, /root, and /var because much of it is either configuration stuff that isn't needed due to sensible defaults or it is auto-generated by the boot process on the initial run.

The sandbox remaster script that I'm working on takes an intermediate approach. It assumes that it is okay to copy /root and /etc as is. However, the script is a bit more picky with the /var folder because I know it contains stuff that I will want to remove (pid files and log files) but I'm not sure at the moment how to pragmatically identify it and remove it. Sometimes the /var folder even has temporary files and private keys (e.g. tor hidden service).

So the var section of my code could use work but it isn't that critical because puppy doesn't depend too much on the /var folder and the stuff it depends on within this folder (e.g. /var/packages), the script properly deals with.

Package Manager Considerations and Building an Adrv

The script is designed so that you can either build a single layer (e.g. an Adrv) or alternatively combine all layers into a single sfs file. In the later case you copy directly from the aufs file system. I'm using this with my psandbox tool (based on jamesbond's sandbox.sh). In this tool the aufs file system is located at:

Code: Select all

/mnt/sb/fakeroot

If you are building an sfs which combines all layers, then package metadata should be written to:

Code: Select all

/var/packages/woof-installed-packages

If you you are building a single layer (e.g. an adrv) then you might write the package meta-data to a different file. For example:

Code: Select all

/var/packages/adrv-installed-packages

In either case we move the list of package contents to the folder:

Code: Select all

/var/packages/builtin_files

Here are some implementation details:

Code: Select all

	  while read -r ONEPKG
	  do
      ONEFILE="/tmp$PKG_FILES_DIR/`echo "$ONEPKG" | cut -f1 -d '|'`.files"
      ONENAME="/tmp${PKGS_DIR}/builtin_files/`echo "$ONEPKG" | cut -f2 -d '|'`"
      [ -f "$ONEFILE" ] && mv -f "$ONEFILE" "$ONENAME"
      [ -f "$ONENAME" ] && echo "$ONEPKG" >> /tmp${PKGS_DIR}/${INST_PKG_PREFIX}-installed-packages
	  done < <(cat "${a_root}${PKGS_DIR}"/*-installed-packages)
	  #TODO, maybe look up the meta info of any package that wasn't moved
	  
	  #cat "${a_root}${PKGS_DIR}"/*-installed-packages >> /tmp${PKGS_DIR}/${INST_PKG_PREFIX}-installed-packages
	  sort -u --key=1 --field-separator="|" /tmp${PKGS_DIR}/${INST_PKG_PREFIX}-installed-packages > /tmp/${INST_PKG_PREFIX}-installed-packages-tmp #110722
	  mv -f /tmp/${INST_PKG_PREFIX}-installed-packages-tmp /tmp${PKGS_DIR}/${INST_PKG_PREFIX}-installed-packages
	  echo -n "" > /tmp${PKGS_DIR}/user-installed-packages #v431	

https://pastebin.com/EcpKrsnd

Calling the remaster script

The intent of my script is to be more command-line driven than the official puppyremaster2. There are still some legacy gui stuff from puppies remaster script that do appear when you run this script. My plan is to provide an option to turn off these gui prompts.

In this example we build both an adrv for precise light and also a basesfs that has the adrv merged into it.

Code: Select all

#!/bin/bash
exec &> >(tee ./mk_adrv_log)
bash -x ./mk_adrv.sh -s "$(cat << EOF
/initrd/mnt/dev_save/precise/light/5.7.2/precisesave adrv_precise_light-5.7.2.sfs adrv 
/mnt/sb/fakeroot puppy_precise_light-5.7.2.sfs woof
EOF
)"

The -s option means (string). -f is for file. File descriptors don't seem to work properly (in place of file for my script) so for now you have to use the "-s" option if using heredocs (instead of files) but you can use the -f option if you want to create a separate file.

Each line of this file (or string) -- for example see the above herdoc -- has the following fields:
"folder (e.g. mount point)", "output file name", "metadata prefix"

I significantly cut down remasterpup2

My code significantly cuts down remasterpup2. The reason that I did this was to simplify things for development. In fact, at this point it doesn't even build an ISO. It only builds the sfs files. I will likely be adding much of the stuff I stripped out back in later when development is further along. A lot of the comments contain references to fixes in the original remasterpup2 script. I will have to add the comments back later that explain these changes.

Known Issues

While writing this I realized that the /home folder is excluded. This is fine for building an adrv but not for a combined base sfs file (i.e. PUPSFS). I will fix this shortly. Also the code is very preliminary. It appears to work but I recommend that people wait a bit before trying it.

Name Suggestions

p.s. I think a better name for this script than mk_adrv.sh. Suggestions welcome

[s243a]

As a side note, in the above example the output is adrv_precise_light-5.7.2.sfs because the name of this has to match what is in the initrd version of /etc/DISTRO_SPECS. I will create an option to update the initrd with an alternative name but I'll note that if one does this then the script will take longer to run.

[s243a]

I added this script to gitlab:
https://gitlab.com/s243a/psandbox/-/blo ... mk_adrv.sh

Home folder is still not included in the remaster. I'll make this fix soon.