Fatdog64-802/801/800 Final [21 May 2019]

[rufwoof]

I have to use Firefox, when I transfer my age pension to my self I use a XXXX company and their web site does not work with Seamonkey.

It can be trivial to elevate from spot to root. Best IMO to not rely upon running as spot as being secure. Instead look to keep a 'clean' version of the system and browser and boot/use that to go directly to your company web site, nowhere else before or after as that way even when running as root with older software that is pretty secure/safe.

Ideally that means booting from a DVD or USB. If you use the multi-session save method then you can initially boot that, configure it as you like preferably keeping any online activity during that set up stage to a minimum, install firefox, create the save and then put that dvd/usb aside for your 'banking' activities i.e. boot it, do your online banking and shut down afterwards without saving. To prevent system auto saving, set the Event Managers Save Session Interval to a value of zero (Control Panel, Desktop tab, FatDog64 Event Manager, RAM save interval).

I boot such a usb as my daily workhorse. See the link in my signature below. From a security perspective Fatdog is the leader of the Puppy's, but security is a process not a product so you have to use it the correct way.

[spotted]

@ step
I have it sorted now. Thanks Fresh install, new savefile, 'update firefox' from control panel. Firefox is downloaded to root, if run from there, .mozilla is placed in home/spot, all fixed. I had been running firefox from the next petition saved from a previous download, and that is when firefox is hardwired to put .mozilla into root.
@ rufwoof
Yes, I have a separate Flash drive for banking, another for experimenting with, and a SSD for normal internet use.

[rufwoof]

Fatdog has earlyshell and lateshell boot parameters already.

Proposed change : to add a midshell boot parameter in addition to those, where initrd's /sbin/system-init, around line 1450 is extended to include

grep -q midshell /proc/cmdline && echo 'Starting midshell. Type "exit" to continue or reboot -f to reboot' && setsid cttyhack /bin/sh

so it looks like ...

Code: Select all

# earlyshell - after load modules and pkeys so that usb keyboard works
grep -q earlyshell /proc/cmdline && echo 'Starting earlyshell. Type "exit" to continue.' && setsid cttyhack /bin/sh

# waitdev and wait for usb disk
process_waitdev

# configure network - after load modules, before basesfs
process_net

# process keyfile - after load modules, before basesfs
process_keys

# midshell
grep -q midshell /proc/cmdline && echo 'Starting midshell. Type "exit" to continue or reboot -f to reboot' && setsid cttyhack /bin/sh

# configure mdadm and lvm if requested, before basesfs and savefile
process_dofsck
process_mdadm # note: mdadm before lvm
process_lvm

Why: Because that's before fd64.sfs gets loaded, but is potentially after keyboard and wireless or ethernet net connection has been established. i.e. if you boot with the net boot parameter such as ... (assuming grub4dos menu.lst (note I've changed my actual wireless ssid and password in this example to VM999999 and xxxxxxxx respectively))

Code: Select all

title Fatdog midshell with wlan0
root (hd0,0)
kernel /vmlinuz midshell net=wpa2:VM999999:xxxxxxxx:wlan0:dhcp pkeys=uk video=640x400 savefile=none
initrd /initrd

then that boots to (initrd) cli and I can telnet, ssh ...etc. as desired, which includes being able to potentially wget, curl, scp or whatever

I've revised mine to call /root/menu.sh (inside initrd, so the earlier command now looks like

Code: Select all

grep -q midshell /proc/cmdline && echo 'Starting midshell. Type "exit" to continue or reboot -f to reboot' && setsid cttyhack /root/menu.sh

instead of running /bin/sh) where that menu.sh file is as attached (<userid> in that code is replaced with my actual hashbang and sdf userid's)

[rufwoof]

Attached script works well for me for initrd cli based wifi/network connecting. Setting the kernel net.... boot parameter to have invalid ssid and password, so that the modules/drivers get loaded without leaving sensitive values lying around that might potentially be snooped, enables that script to function OK.

For releasing (to perhaps switch to another ssid), I'm using code borrowed out of network-setup.sh

Code: Select all

kill_prog() {
	local p
	for p in $(pidof $1); do
		grep -q $2 /proc/$p/cmdline && kill $p
	done
}
deactivate_ip() {
	kill_prog dhcpcd $1
	kill_prog udhcpc $1
	ifconfig $1 0.0.0.0 > /dev/null # release IP address 
}

deactivate_interface() {
	deactivate_ip $1
	kill_prog wpa_cli $1	
	kill_prog wpa_supplicant $1
	ifconfig $1 down > /dev/null 
	return 0
}

i.e. run 'deactivate_interface wlan0'

[rufwoof]

Fatdog init cli, and you can't just compile or copy across binaries into that and run them. Even with the appropriate lib's installed. You can however run them in a chroot, and as we already have a fd64.sfs to hand that can be mounted and that mount point used as the chroot folder. Being a read only system however you have to pull a few tweaks - but even then takes up near zero overhead.

Scenario: initrd with no fd64.sfs within that. Booted to init cli (as per midshell that I outlined earlier), where the fd64.sfs is stored elsewhere (remote or on a local HDD) that is then copied/downloaded into the ramdisk (/fd64.sfs).

As after mounting fd64.sfs and chrooting into that, that is read only (can't create a folder), I've opted to use the pre-existing /mnt/floppy folder to mount my HDD (sda1) - as nowadays /mnt/floppy is unlikely to otherwise be used. I've also bind mounted /tmp so that is writable - security isn't a issue here. I created a /root/chroot.sh script ...

Code: Select all

#!/bin/sh
mkdir /CHR
mount /fd64.sfs /CHR

export LC_ALL=C 
mount --bind /dev /CHR/dev 
mount --bind /proc /CHR/proc 
mount --bind /sys /CHR/sys 
mount -t devpts devpts /CHR/dev/pts 
mount --bind /tmp /CHR/tmp
mount --bind /etc /CHR/etc   # so network works i.e. /etc/resolv.conf
echo "#!/bin/sh" >/tmp/CHRinit
echo "HOME=/mnt/floppy export HOME" >>/tmp/CHRinit
# using /mnt/floppy as a mount point here as sfs mount is ro
# and /mnt/floppy exists within that, but is unlikely to be used
echo "mount /dev/sda1 /mnt/floppy" >>/tmp/CHRinit
echo "/bin/sh" >>/tmp/CHRinit
chmod +x /tmp/CHRinit
#xhost + 
#mkdir -p /mnt/sdb1/cnt/tmp/.X11-unix 
#mount --bind /tmp/.X11-unix /mnt/sdb1/cnt/tmp/.X11-unix 
#chroot /CHR /usr/bin/htop "$@"
chroot /CHR /tmp/CHRinit "$@"

... and running that results in a shell prompt within the chroot environment, from which I can run things within fd64.sfs such as mc, or compiled binaries on my HDD (which is mounted to /mnt/floppy) i.e. my ncurses based 'menu'.

Afterwards, I can exit the chroot, and then exit the midshell ... to resume booting to the full/normal fd64 switched root gui desktop. Or just shutdown/reboot.

Note that for networking to work within the chroot I simply bind mounted /etc (so /etc/resolv.conf etc. are shared between the main and chrooted environments).

Booting from usb and the 74MB initrd boots really quickly. I've been using a fd64.sfs stored on hdd so copying that into the ramdisk is also very quick. I then have a TUI/cli desktop that's net (wireless) connected and I can ssh/whatever using that - but for added niceties mounting/chrooting the /fd64.sfs considerably extends that tui's functionality. I can then shutdown after I'd checked my mails/irc/bbs's etc. or alternatively boot to the full fatdog gui desktop if needed (exit the midshell). Makes my fatdog more modular. There's also the option to create a bespoke alternative to fd64.sfs as the sfs that is mounted as part of running midshell, one that might for instance just contain tui type programs (freepascal's IDE, mc, w3m ...etc.).

I've tried that boot usb out on a range of desktops (radeon/intel graphics) and all booted with wireless net connected fine. One booted OK but with no net connection due to that desktop having no wireless, creating a script did the trick for that

Code: Select all

ifconfig eth0 up
echo "auto eth0" >>/etc/network/interfaces
echo "iface eth0 inet dhcp" >>/etc/network/interfaces
ifup eth0

Including the net... kernel boot parameter alongside vga=ask and nomodeset seems to be a good general choice. As then pressing F12 after power on with the usb inserted, provided secure mode has been disabled, booted OK in each instance.

[rufwoof]

So I'm booting fatdog usb stick frugal installed (grub4dos) with the initrd having no fd64.sfs inside it (around a 75MB sized initrd), with multi-session saves, back to usb. Event manager save interval is set to zero, so I can unplug the usb once booted, only need to reattach it if I want to run a save action, or boot again.

Code: Select all

kernel /vmlinuz midshell net=wpa2:xxssidxx:xxpasswordxx:wlan0:dhcp pkeys=uk savefile=direct:multi:uuid:2d5fb1e2-5276-4950-bc8c-0b3c1ec9cfa6:: video=640x400

At my midshell point, i.e. just before initrd loads fd64.sfs, in the initrd cli I have been copying in a fd64.sfs to / in the ramdisk so it uses that when initrd's execution is resumed to load the gui desktop. I have a fd64.sfs on my hdd (sda1), stored in the root folder to facilitate such copying. But I thought I'd try just sym linking that instead, so I mounted sda1, sym linked the hdd fd64.sfs to /fd64.sfs inside the ramdisk, and then resumed initrd boot ... and it worked. htop shows significantly less memory being used, normally after first booting/loading the gui desktop htop is up at around over 1GB used but now its down at 692MB type level. A 300MB saving which ballpark compares to the size of the 340MB fd64.sfs size.



My (muti-session) save file is quite large, contains google chrome etc. and is around 190MB that iirc expands to around 400MB, which is all loaded into ram.

I hadn't expected that sym linking of fd64.sfs from within initrd/ramdisk to a hdd based fd64.sfs to work, I thought that the ramdisks sym link to sda1 fd64.sfs would be lost/broken, or inaccessible after the switch root, but seemingly not. I even fully loaded 3GB+ of ram usage so that swap was being used, and the system continued to work fine. I had also expected that sda1 would have to remain mounted, but no, I can mount and unmount it from within the main gui desktop.

Don't know what's going on, but that's pretty amazing in my book. Currently, with chrome loaded and posting this ...

Code: Select all

# free -m
              total        used        free      shared  buff/cache   available
Mem:           3406         422        2066         579         917        2022
Swap:         16383           0       16383
# 

To double check, I umounted sda1 (my HDD) and using a terminal cd'd to /mnt/sda1 ... and it was empty. Remounting again and /mnt/sda1 showed the files/folders. I guess fd64.sfs has just been cached, rather than allocated to fixed ram ???

EDIT: Noticed in /dev/initrd.err

==== end of initrd ====
/usr/bin/auchk: Checking /aufs/pup_save for aufs
/usr/bin/auchk: [Pass 1] Illegal whiteout
/usr/bin/auchk: [Pass 2] Remained pseudo-links
/usr/bin/auchk: [Pass 3] Remained temp files
rmdir: failed to remove '/mnt/+*': No such file or directory

So it could be a feature (undocumented bug) that /mnt/sda1 in initrd isn't released. Also see from the linux documentation that initrd :

Note: /dev/initrd is read-only and it can only be used once. As soon
as the last process has closed it, all data is freed and /dev/initrd
can't be opened anymore.

So in /mnt/sda1 being mounted in initrd, after switch-root the initrd isn't freed because of the open mount process. ???

[rufwoof]

Might I propose adding Terminus bold 32 console font to the next release of Fatdog. At 3KB that's a relatively minor increase whilst adding a better IMO font for the likes of displaying BBS's. Better than the current big font - that is OK, but isn't as good at displaying ansi graphics.

Add to /lib/boot/consolefonts within initrd and /usr/share/consolefonts within the main fd64.sfs ... and use setfont <filename> to activate it.

If that's renamed to bbs.psf.gz then that would fit well alongside the big and bbig choices i.e. ctrl-alt-F2 and login, and then run 'setfont bbs' before telnet'ing to a BBS i..e TERM=linux telnet blackflag.acid.org

[dr. Dan]

New (to me) laptop, and it's up and running with 802. It has a 1920x1080 screen, so I've been making adjustments for readability.

I'm seeing an issue with the new control Panel. As the screen capture shows, it doesn't resize smoothly. The Htop screenshots show an issue in the processes running the tabs after resizing. It doesn't happen all at once. I changed the initial dimensions in the script, so I don't have to resize it in the first place. (The high-dpi setting in the script didn't kick in for me.)

Also, I've attached a screenshot of the popup window when a program isn't shutting down correctly. It's been like this for me about from the beginning (4 years ago), but I haven't gotten around to asking about it. Where do I find the file to edit the get readable text?

I hope I've kept the files small.

Thanks all, it continues to be a pleasure.

Dan

[step]

Dan, re the popup window look, are you using the default Fatdog theme or switched to something else? Does the popup always look like that or does it look right sometimes? What happens if you boot without a savefile, can you test the popup window? Here some steps to open the popup at will:
1. open a terminal window
2. type yad --text-info and press ENTER
3. when yad is displayed switch to the terminal and press Ctrl+Z, this will stop the yad process
4. now close the yad window, either from its X corner or the app bar; yad will not close
5. after a few seconds you'll see the popup dialog.
To eventually close yad go back to the terminal and type fg then press ENTER.

Code: Select all

# yad --text-info
^Z
[1]+  Stopped(SIGTSTP)        yad --text-info
# fg
yad --text-info
Terminated
# 

As regards your control panel issue, I can take a deeper look. Contact me privately next week, please. Thank you.

[step]

Dan, some quick thoughts. What's the value of key Xft.dpi in file /root/.Xdefaults?
What happens if you edit /usr/sbin/fatdog-control-panel.sh and insert this line before line 103, so that the new line becomes 103

Code: Select all

	"--fixed" \

Be careful when you paste from the forum; make sure that there are no extra spaces after the end of the line (after the backslash character in this case).

[SFR]

@Dr. Dan: I can reproduce the pop-up window issue.
It's a "feature" of 23oz Openbox theme.
The relevant colors are defined in /usr/share/themes/23oz/openbox-3/themerc, lines 231 & 239:

Code: Select all

osd.button.unpressed.bg.color:			#303030
...
osd.button.focused.bg.color:			#303030

I guess we should change it to something brighter...

@Step: I can also repro the CP issue, where yad processes are maxing out the CPU and one or more tabs become blank.
It takes some rapid resizing, though, and it seems to be much easier to reproduce in VBox with limited CPU resources, than on real HW.

Greetings!

[dr. Dan]

@SFR: I've had the control panel issues on both the new higher dpi unit, and the previous ~1200x800 which was my first 800 series install. I don't typically resize windows like in the screen capture, but I had similar results with both, after just resizing once, and both with i5 processors and sufficient (8Gb, 4Gb) RAM. I've yet to try a VM of anything, but perhaps soon.

Thanks for isolating the 23oz issue. I do use it, somewhat modified, so I'll make changes and offer suggestions.

@step: I'm still on 721 on this computer, so I'll test later and let you know.

Thanks to you both.

Dan

[rufwoof]

initrd version of dropbear doesn't seem to support encryption of your private ssh key.

I opted to install ccrypt http://ccrypt.sourceforge.net/ as a wrapper to accessing my ~/.ssh db_id_rsa key (that I converted from openssh using drobbearconvert openssh dropbear id_rsa db_id_rsa)

Code: Select all

cd /root/.ssh
ccrypt -c db_id_rsa.cpt >/root/.ssh/db_id_rsa
echo "#!/bin/sh" >/tmp/cleanup
echo "sleep 20" >>/tmp/cleanup
echo "rm /root/.ssh/db_id_rsa" >>/tmp/cleanup
echo "rm /tmp/cleanup" >>/tmp/cleanuup
chmod +x /tmp/cleanup
/tmp/cleanup &
chmod 0600 /root/.ssh/db_id_rsa
ssh -i /root/.ssh/db_id_rsa <userid>@ny1.hashbang.sh

I've set it to leave the ssh key open for 20 seconds to allow for ssh connection (once connected the key is no longer required).

Would be nice/useful if there were a encryption tool such as ccrypt available within initrd by default, as in the absence of anything else (such as dropbear encryption of keys), its good practice to not leave your private keys 'open'.

PS the pre-built ccrypt binary worked for me, just dropped it into initrd's /usr/bin folder and done.

[rufwoof]

Fatdog initrd minus fd64.sfs, which instead is stored on a remote ssh server.

Dropped sshfs (along with ssh, scp) into initrd.

Booted with net kernel boot parameter to early net connect (I'm using wireless), exited to shell before fd64.sfs is processed in init and sshfs mounted the remote ssh server to local /hb folder

sym linked /hb/fd64.sfs to /fd64.sfs i.e. remote (via sshfs) ssh servers copy of fd64.sfs sym linked to the root folder of the initrd ... so somewhat like it having already existed in the initrd.

Resumed initrd boot (to load fd64.sfs/fatdog gui desktop) ... and it loads/runs fine. Quite slow initially to get going, as its pulling stuff over ssh (in my case from the US to the UK as my ssh server is located in the US whilst I'm in the UK). But not uncomfortably slow - not much different to booting a full blow fatdog DVD). Once up and running, i.e. more things cached, then no general operational speed differences.

It would seem that switch-root doesn't free processes that are running, so the sshfs mount made within initrd persists after the switch-root. Its just inaccessible from the command line.

Booted from usb that way and its around a 80MB initrd.gz filesize (largely due to kernel modules sfs within initrd). And once booted the boot usb can be unplugged so its all running in ram, with a remote sshfs mount (fd64.sfs).

I have quite a large savefile (multi-session saves that are stored on usb) as I have google chrome included in that (around 140MB of compressed save file space), without that loaded uses around 250MB of ram, with my saves loaded into ram however that rises to around 750MB



Having ssh/sshfs as in openssh rather than the default dropbear ssh within initrd is nice as its more functional. Along with mc and tmux also dropped into initrd and initrd expands to 92MB - but reduces to 80MB if you gzip it. Around a 8MB expansion (initrd without fd64.sfs is around 72MB).

I guess one negative is that once booted to desktop, you can't easily resume the sshfs remote mounted fd64.sfs if the ssh link drops. So this is more a observation than a proposed actual boot choice. It would be simpler/better to just scp the remote fd64.sfs into ram and use that to boot from.

[jake29]

I've searched this forum, but cannot find an answer. Is it possible to get 'vainfo' functional in Fatdog64 or is there an alternative I can use?

EDIT: I note that 'vainfo' is included in libva-utils. I need this utility to troubleshoot an app that is causing me trouble. Developer has requested 'vainfo.'

EDIT2: I've now done a temporary install of full Ubuntu 19.04 and my problem app is working without issue. I have established libva plays a key role in the functionality of the app, can anyone comment on libva-utils and why it is so far considered optional in Fatdog64?

EDIT3: I finally noticed that libva and libva-intel-driver are both outdated, and I have now compiled newer versions (along with libva-utils). Hopefully this will resolve my issue.

[rufwoof]

Fatdog 8.2 kernel-modules.sfs is 65MB. Compiling the kernel with no fd64.sfs inside initrd and kernel-modules.sfs extracted within (unsquashfs'd into) initrd (and kernel-modules.sfs removed from the initrd) ... and that initrd inserted into the kernel (vmlinuz) resulted in a 59MB vmlinuz filesize. i.e. boots with no need to include a initrd entry in menu.lst (just specify the vmlinuz as the initrd is contained within that).

Booting with just a vmlinuz (and external fd64.sfs) is nice in some respects - more secure as initrd is 'hidden' within vmlinuz, the smaller size (6MB saving) is also welcome. But does make accessing/changing things that more difficult (kernel recompilation needed after changing initrd). Built once however (slow first compilation) and subsequent kernel recompiles do run through relatively quickly (few minutes).

The method I used to insert initrd into vmlinuz was to copy initrd to usr/initramfs_data.cpio (/mnt/sdb1/fatdog-kernel/src/linux-4.19.44/usr/initramfs_data.cpio in my case) prior to running make (in /mnt/sdb1/fatdog-kernel/src).

Currently to swap a kernel typically you replace the kernel-modules.sfs and vmlinuz to the new/alternative version. Given that initrd content excluding kernel-modules and fd64.sfs is relatively small having both initrd and kernel-modules combined into a single vmlinuz is a nice alternatively, more so in that the initrd files/scripts could be better matched with the kernel-modules (kernel version) that was being selected.

[rufwoof]

/etc/rc.d/rc-cleanup ... is

$BB swapoff -a

strictly necessary? If swap has been heavily used then swapoff -a can take ages. I would have thought that instead a simple 'sync' might suffice?

[jake29]

Hi all. I do not like to make a request like this, however I am having difficulty doing a pkgbuild of LLVM. I need v7.0.0 or newer to compile some other stuff. The laptop (and only machine I have) is locking up for hours during compiling.

Currently only v5.0.2 is available in Gslapt, which is quite dated - latest version is 8.0.1.

[Flash]

I downloaded the Fatdog64-802 .iso and burned it to a DVD using Burniso2cd. The DVD booted to a desktop without my intervention, but neither the mouse nor the keyboard worked. (Both are USB.)

When I chose the multisession boot option, it wouldn't boot at all.

[jamesbond]

Sorry guys long time no post, I have been very busy lately.

Just a few comments.

rufwoof - love to see your interesting experiments. Yes, the possibilities are endless and that's what we had in mind when we started Fatdog - something that works out of the box but can be extended in many various ways.

A note about Fatdog's initrd. Fatdog is actually two operating system in one. Fatdog's initrd is actually its own independent CLI-only busybox-based mini operating system called "Bulldog Linux", with the proper Fatdog tacked on top of it. It is not an accident that the busybox in Fatdog's initrd is a complete busybox with almost all applets included; it is by design. In the proper Fatdog environment its only purpose is to setup Fatdog's layered filesystem and as a recovery console; but you can choose not to do that and run Bulldog by itself as a CLI operating system (run with basesfs=none and you'll see what I mean).

About loading fd64.sfs from NBD: it works but the busybox in 800 requires a patch to make this properly (thanks to bitrot). I've an updated ISO which does this - it's the 800tip, contact one of the Fatdog team to get the location.

<Might I propose adding Terminus bold 32 console> ==> link?

<ccrypt> ==> It's 815KB. If it can be made smaller then I'll bring it in. Otherwise I'm sure there are similar others which is in the order of 50K or less (perhaps bcrypt?). Otherwise cryptsetup is already included in initrd, you can have a "secure vault" containing your keys, then open/mount it, start ssh, and close the vault again.

As for LLVM update: LLVM is tightly integrated with the Mesa radeon driver. Updating it requires extensive re-testing to make sure that 3D with radeon/amdgpu still works properly. If you don't use 3D using radeon/amdgpu then it should be okay.

Flash I'll have to defer your problems to others for the moment.

I will disappear again, see you next year maybe
SFR and step, thank you for holding the fort.