Stripping Down a Puppy

Message

mikeslr · #21 Post by **mikeslr** » Tue 26 Mar 2019, 14:57

nic007 wrote:You may want to consider loading your additional (or stripped) applications as extra sfs's and not as part of an adrv or ydrv. The reason for this is that the adrv and ydrv are normally automaticly loaded into RAM at bootup if you have enough RAM (unless you specify pfix=nocopy, I think) whereas extra sfs's are not loaded into RAM...

That was also the conclusion I reached. I didn't find any saving of RAM on bootup if an adrv/ydrv was employed rather than including applications in the puppy_xxx.sfs. And adrv/ydrv can not be UNloaded-on-the-fly. About the only advantage these have is a way to package additional applications NOT to be included in the ISO but easy to download for those who want them. An adrv/ydrv can be renamed so it will be managed as an ordinary SFS.

nic007 · #22 Post by **nic007** » Wed 27 Mar 2019, 06:10

Further to this RAM discussion - If you have "enough" RAM, Puppy will load your base sfs, zdrv, adrv and ydrv (all the contents of it) into RAM whether you like it or not. But not only that, it will also copy these sfs files (the actual files) to RAM to just lie there for no other reason but to occupy RAM (remember the contents are already loaded into RAM) UNLESS you use pfix=nocopy in which case this unnecessary copying of the sfs's will not take place. This can be significant. Take the example where you only have 2 Gig RAM and your base sfs, zdrv, adrv, ydrv are say 300MB size in total. You can stop this 300MB from being copied to RAM by just using pfix=nocopy.

rufwoof · #23 Post by **rufwoof** » Wed 27 Mar 2019, 12:26

I always opt to not have the sfs copied to ram and just let the system decide what to cache or not. Quicker to boot, slower to initially load a program (but then typically no different) ... but does mean having the partition mounted (when ram copied all partitions can remain unmounted).

My mental reasoning is along the lines of 'why have the likes of gparted wastefully loaded into ram at each boot when I use it rarely'.

Argolance · #24 Post by **Argolance** » Wed 27 Mar 2019, 13:27

Hello,
If I cut my finger, bleed like a calf and urgently need the plaster at the bottom of the first aid kit that I have to empty completely before I find it, it is not good in my opinion.

Having everything at hand quickly/immediately is one of the strengths and the uniqueness of a system that is fully loaded into RAM. This does not prevent me, if the opportunity arises, from running Puppy in another way with options like "nocopy" (or any other) made available to the user.
In any case, I think that the advantages of such a system far outweigh the possible disadvantages.
Cheers!

Cordialement.

musher0 · #25 Post by **musher0** » Wed 27 Mar 2019, 13:51

Argolance wrote:Hello,
If I cut my finger, bleed like a calf and urgently need the plaster at the bottom of the first aid kit that I have to empty completely before I find it, it is not good in my opinion.
Having everything at hand quickly/immediately is one of the strengths and the uniqueness of a system that is fully loaded into RAM. This does not prevent me, if the opportunity arises, from running Puppy in another way with options like "nocopy" (or any other) made available to the user.
In any case, I think that the advantages of such a system far outweigh the possible disadvantages.
Cheers!

Cordialement.

The Word of the Wise... (IMO)

Please consider also that a lot of useful GNU utilities, Puppy is NOT offering
OOTB. Puppy is already a "no-fat" distro. Beware of making it "anorexic".

As for me, I am now looking in the other direction. For example, the ex-factory,
woof-CE, Puppy does NOT have enough themes and fonts anymore to offer the
user something that can be called a "choice". The user has to get the themes and
fonts (s)he likes through one or many additional downloads; some users may find
this annoying.

BFN.

nic007 · #26 Post by **nic007** » Wed 27 Mar 2019, 16:06

What is the point of copying the actual base, adrv, ydrv, zdrv sfs files (ie. not extracting and not loading them) to ram when their contents are loaded into RAM automatically anyway if you have enough ram. The pfix=copy or nocopy does not have anything to do with actually loading the contents of those mentioned sfs's into ram it seems to me.

s243a · #27 Post by **s243a** » Wed 27 Mar 2019, 16:12

nic007 wrote:What is the point of copying the actual base, adrv, ydrv, zdrv sfs files (ie. not extracting and not loading them) to ram when their contents are loaded into RAM automatically anyway if you have enough ram. The pfix=copy or nocopy does not have anything to do with actually loading the contents of those mentioned sfs's into ram it seems to me.

Maybe we can add other boot options like anocopy, ynocopy, znocopy, etc. So that the user can control exactly what they want in ram.

s243a · #28 Post by **s243a** » Wed 27 Mar 2019, 16:14

rufwoof wrote:I always opt to not have the sfs copied to ram and just let the system decide what to cache or not. Quicker to boot, slower to initially load a program (but then typically no different) ... but does mean having the partition mounted (when ram copied all partitions can remain unmounted).

My mental reasoning is along the lines of 'why have the likes of gparted wastefully loaded into ram at each boot when I use it rarely'.

I noticed when looking at gdmap the other day that gparted was quite large! I think that it can be purged from the base sfs.

Argolance · #29 Post by **Argolance** » Wed 27 Mar 2019, 16:32

What is the point of copying the actual base, adrv, ydrv, zdrv sfs files (ie. not extracting and not loading them) to ram when their contents are loaded into RAM automatically anyway if you have enough ram

Sorry, I thought the SFS files were uncompressed/loaded directly in RAM and that this was a single operation. This is why I also thought it was useful to copy the SFS files to the disk because their extraction from the CD/DVD may take time (much less from an USB stick).

nic007 · #30 Post by **nic007** » Wed 27 Mar 2019, 16:56

Argolance wrote:
What is the point of copying the actual base, adrv, ydrv, zdrv sfs files (ie. not extracting and not loading them) to ram when their contents are loaded into RAM automatically anyway if you have enough ram
Sorry, I thought the SFS files were uncompressed/loaded directly in RAM and that this was a single operation. This is why I also thought it was useful to copy the SFS files to the disk because their extraction from the CD/DVD may take time (much less from an USB stick).

Note the difference between decompressing and loading on the one hand (which is beneficial) and the mere copying of files without doing anything with the contents. The system will decompress and load the contents of the base sfs, zdrv, adrv, ydrv automatically into RAM and ALSO automatically copy the actual sfs files from wherever they are situated into RAM if you have enough RAM. I don't get this automatic copying (not the decompression and loading) into RAM. What purpose does it serve? Just make pfix=nocopy to avoid this (should be the default behaviour.

s243a · #31 Post by **s243a** » Wed 27 Mar 2019, 18:42

musher0 wrote:
Argolance wrote:Hello,
If I cut my finger, bleed like a calf and urgently need the plaster at the bottom of the first aid kit that I have to empty completely before I find it, it is not good in my opinion.
Having everything at hand quickly/immediately is one of the strengths and the uniqueness of a system that is fully loaded into RAM. This does not prevent me, if the opportunity arises, from running Puppy in another way with options like "nocopy" (or any other) made available to the user.
In any case, I think that the advantages of such a system far outweigh the possible disadvantages.
Cheers!

Cordialement.
The Word of the Wise... (IMO)

Please consider also that a lot of useful GNU utilities, Puppy is NOT offering
OOTB. Puppy is already a "no-fat" distro. Beware of making it "anorexic".

As for me, I am now looking in the other direction. For example, the ex-factory,
woof-CE, Puppy does NOT have enough themes and fonts anymore to offer the
user something that can be called a "choice". The user has to get the themes and
fonts (s)he likes through one or many additional downloads; some users may find
this annoying.

BFN.

The intent isn't to replace a typical puppy. Instead we strip down a puppy to achieve a specific application. For example:
1. We want a minimal environment to run the package manager (e.g. to download packages)
2. We want a minimal chroot environment, in which case we don't need a desktop
3. We want to run a server in a command line environment (in which case we don't even need Xorg, or sound or bluetooth etc...)
4. We want a base system from which to build upon.

Let's take case #2 above. Here are some core desktop packages that we might be able to remove from x-slacko slim:

Code: Select all

#Dektop related stuff
A_DesktopCore=( "desk_icon_theme_blue_moon_Slacko" "desk_icon_theme_stardust" \
"desk_icon_theme_zabuton" "desktop-file-utils" "gnome-menus" \
 "libgnomecanvas" "libxfce4ui" "libxfce4util" "libxfcegui4-4.10.0" \
"puppy_icon_theme" "pup-volume-monitor" "redshiftGUI-0.2.1" "retrovol" \
"pupx" "shared-mime-info" "soothe-theme" "xfce4-cpugraph-plugin-1.0.5" \
"xfce4-mixer-4.11.0" "xfce4-notifyd" "xfce4-panel" "xfce4-screenshooter-1.8.1" \
"xfce4-session" "xfce4-settings" "xfce4-terminal" "xfce4-whiskermenu-plugin" \
"xfconf" "xfdesktop" "xfwm4" \
 "garcon" "hicolor-icon-theme" "libexo_lib-0.3.2" "libgtkhtml" "lxtask" \
"netmon_wce" "mkwallpaper"  )
ALL+=( "${A_DesktopCore[@]}" )

From: https://pastebin.com/Zb45fM3C

Now by remove perhaps I mean "put into an sfs that can be loaded when needed"

I understand that a lot of the stuff is small and probably not worth removing. However, by removing clutter we can better see what the culprits are in terms of library or package size. For example I noticed that bash is quite large.

Perhaps bash can be compiled statically against a smaller C library than GNU C. For example in pupngo many applications are statically compiled against uclibc. Perhaps in the absolute minimal system we can use the bash binary from pupngo. Maybe there are other static binaries that we can use from pupngo and if someone needs a more recent version of bash they can put it in one of the higher layers (e.g. the adrv) or sfs or maybe install it into the save file.

Anway, I disagree with the notion of a "right size pup". Sure as a standard offering a typical puppy has made a lot of good choices. However, if someone feels that something is missing (e.g. more themes or gnu utils) they can always install it on their own.

Finally once we get rid of the big size culprits the smaller packages will make a greater marginal size percentage difference once removed.

wiak · #32 Post by **wiak** » Wed 27 Mar 2019, 19:36

s243a wrote:Perhaps in the absolute minimal system we can use the bash binary from pupngo. Maybe there are other static binaries that we can use from pupngo and if someone needs a more recent version of bash they can put it in one of the higher layers (e.g. the adrv) or sfs or maybe install it into the save file.
...
Finally once we get rid of the big size culprits the smaller packages will make a greater marginal size percentage difference once removed.

Good idea - use older smaller components when not critical most of the time. Old bash fine for most things. Also, components removed may be small, but if you can remove enough of them the difference in size soon adds up (a bit like cleaning out your hard disk - first a few huge items makes major difference, but then removing hundreds of little unused items also ooes.

Also, it's not just the size each little program takes up on disk; what may be more important can be the amount of RAM they occupy/use-up - looking for biggest saving there really.

wiak

rufwoof · #33 Post by **rufwoof** » Wed 27 Mar 2019, 21:01

Only if you want to load and run totally from ram is size a issue. I for instance have a tiny EasyOS that's less than 20MB, cli only, just the modules needed for net connection ...etc. For my main easyOS desktop I've merged the devx into the easy.sfs and use relatively quick lzo decompressor, not loading that into ram at bootup (frugal HDD installed). That weighs in at 1GB easy.sfs filesize, so approximately (without checking) around 2GB of non compressed size. In addition to that I have a changes sfs (compressed save folder content) that's 130MB (mostly firefox), and another similar changes sfs for the main container (that has its own separate save folder).

Two extremes you might say (20MB vs 1GB), and the one I boot the most is that larger choice. No it doesn't run in ram, but what it does use likely gets loaded and cached into ram relatively quickly whilst unused stuff just remains on disk. Load libre office writer for instance once and yes its (acceptably) slower to start the first time, but often remains cached so - subsequent starts are just as fast as had the whole system been loaded into ram.

musher0 · #34 Post by **musher0** » Wed 27 Mar 2019, 22:36

@s43a
You're thinking like a dev who likes challenges, and that's fine. But the
typical user.will like to have something between the skin and the bone!

BFN.

rufwoof · #35 Post by **rufwoof** » Wed 27 Mar 2019, 23:13

Kernel + busybox and small initrd, cli with no network connectivity would fit 2)

Add a sfs of modules/firmware and you can aufs (or overlayfs) mount that in order to add network modules etc. for whatever hardware you might be using. With net connected then you can aufs mount whatever, or use a command line package tool.

Stripping out is more difficult than adding/building.

rufwoof · #36 Post by **rufwoof** » Wed 27 Mar 2019, 23:18

If the idea is to support low spec hardware, poor comms links then cli and dialup to something like sdf could suffice. Less than 20MB size with low hardware requirements ... that provides access to the rest of the world, albeit mainly textual only. Low bandwidth text is also more ideal for poor/non sighted.

wiak · #37 Post by **wiak** » Wed 27 Mar 2019, 23:52

Aside from mainly to keep using xenialdog because of all its great utilities, and Void Linux because its package manager is fantastic, like wanderer I'm planning a new tinycorelinux build. Despite some comments I've read to the contrary, tinycore is easy to set up and fine-tune and understand. I used to make some packages for it, and I remember that was relatively straightforward too. However, alternative mechanisms are always good - all distros tend to have strong points and weak points. Puppy is certainly easier to set up than tiny core - especially since persistence simply works with Puppy but needs a bit of setting up with tiny core. That being said, I really like tiny core persistence mechanism - tends to just need a tiny and simple save file, which can be achieved with Puppy (even no save file/folder), but not so much fine-grained control as in tiny core. Aside from the configuration work to set up Tiny core optimally you also do need to take time to read, and really understand how tiny core works, and have a good memory of that...

http://www.murga-linux.com/puppy/viewto ... 130#990130

Also, above linked post also has link at the bottom to my notes re: Slitaz install. I'll probably put the links into my signature later (to save myself forgetting where they are...).

EDIT: Posting from latest 64bit Tiny Core Linux version 10.x right now. I used 'Apps' browser program to first search for fastest mirror, and then used Apps -> Cloud (Remote) ->Browse to fetch (OnBoot selected): firefox-Getlatest.tcz (followed by running the resultant installed shell script firefox-Getlatest.sh from terminal). So using Firefox Quantum 66.0.2. Normally, I'd actually use a separate firefox simply downloaded from Mozilla and uncompressed in its own folder for sharing between my distributions, but this was just a test install on TC. With this latest firefox installed, TC installation is taking up 144 MiB on disk. EDIT: I noticed wayland.tcz installed with firefox but that seems to just handle some 'wayland protocol' so I guess still just Xvesa framebuffer being used. Of course, default Xvesa framebuffer 64bit TC install, without firefox, is very small indeed: just 22 MiB disk space! Now installing bash, core-utils, mtpaint, geany, and mpv (since has Xorg as dependency - this will all swell the install size): Now 206 MiB, so approaching Puppy size as expected... However, best is, I feel, to use the tiny 22MiB default install (albeit maybe inflated with Xorg) and load the likes of firefox ondemand only or better still as portable external apps (usable also by other distros installed). Note that the main tiny core files always install into RAM and the user can decide whether to simply mount the extra stuff or load it into RAM too, can be different for each squash file tcz (usual is not into RAM). Also, it is indeed very easy to add and subtract pieces from tiny core base - that's part of its beauty.

It's easy, with TC to use 22 MiB core and arrange different onboot.lst files to load entirely different programs (for example, single program into RAM) via different grub4dos entries (and suitable symlinks).

Anyway, these results from TC give a comparison that is something to aim for - some kind of core Pup (a few tens of MiB in size) plus portable external apps or sfs addons for everything else.

Summary disk install sizes:

Core64 (commandline) boot 15 MiB
TC64 XVesa default 22MiB
TC64 default + latest Firefox with gtk3 libs 144 MiB total
TC64 default + bash, core-utils, mtpaint (gtk1), geany, mpv (with Xorg) and latest Firefox (with gtk3 libs and pulse-audio) 206 MiB.
But compare this with Slitaz (ok, midori, not firefox, and some other tricks like midori for video too, maybe just Xvesa by default [I'd have to check], etc: Slitaz 52 MiB installed to disk - but tiny core much more flexible anyway - like Lego set...)!
TazPup (32bit) pretty good: 98 MiB installed to disk. EDIT - has same packages etc as underlying SLitaz but probably more drivers etc and Puppy save persistence functionality - otherwise Slitaz though.

wiak

wiak · #38 Post by **wiak** » Thu 28 Mar 2019, 04:29

Forgot I'd written this (just stumbled upon it) - about using chroot with new woof-CE build; may come in handy for someone:

http://www.murga-linux.com/puppy/viewto ... 512#966512

wiak

wiak · #39 Post by **wiak** » Thu 28 Mar 2019, 10:55

nic007 wrote:The system will decompress and load the contents of the base sfs, zdrv, adrv, ydrv automatically into RAM and ALSO automatically copy the actual sfs files from wherever they are situated into RAM if you have enough RAM.

Are you sure about this - sounds very inefficient.

nic007 · #40 Post by **nic007** » Thu 28 Mar 2019, 11:40

wiak wrote:
nic007 wrote:The system will decompress and load the contents of the base sfs, zdrv, adrv, ydrv automatically into RAM and ALSO automatically copy the actual sfs files from wherever they are situated into RAM if you have enough RAM.
Are you sure about this - sounds very inefficient.

Yes, if you have "sufficient" RAM according to Puppy, Puppy will do that automatically. You can disallow the senseless copy part by using pfix=nocopy but contents will be decompressed and loaded automatically if there is sufficient RAM. I use the adrv just for configuration purposes so it's very small and load additional applications as extra sfs's which are not loaded into RAM at bootup.

(old)Puppy Linux Discussion Forum

(old)Puppy Linux Discussion Forum

Stripping Down a Puppy