How to list files contained in .sfs and pupsave.2fs files?

[MochiMoppel]

Is it possible to list the files contained in a SFS file from the command line?

[Edit] Edited title to reflect the later discussion of of pupsave files



[tallboy]

less will produce a lot of output from an .sfs, but I am not familiar with what format it has, and how to decode it to a readable format.

[musher0]

Code: Select all

unsquashfs4 -l some.sfs 

(That's a lower-case "L", not a "1".)

[/usr/bin]>unsquashfs4 --help
SYNTAX: unsquashfs4 [options] filesystem [directories or files to extract]
-v[ersion] print version, licence and copyright information
-d[est] <pathname> unsquash to <pathname>, default "squashfs-root"
-o[ffset] <bytes> skip <bytes> at start of input file, default "0"
-n[o-progress] don't display the progress bar
-no[-xattrs] don't extract xattrs in file system
-x[attrs] extract xattrs in file system (default)
-u[ser-xattrs] only extract user xattrs in file system.
Enables extracting xattrs
-p[rocessors] <number> use <number> processors. By default will use
number of processors available
-i[nfo] print files as they are unsquashed
-li[nfo] print files as they are unsquashed with file
attributes (like ls -l output)
-l[s] list filesystem, but don't unsquash
-ll[s] list filesystem with file attributes (like
ls -l output), but don't unsquash
-nl[s] list filesystem with file attributes (like
ls -n output), but don't unsquash
-f[orce] if file already exists then overwrite
-s[tat] display filesystem superblock information
-e[f] <extract file> list of directories or files to extract.
One per line
-da[ta-queue] <size> Set data queue to <size> Mbytes. Default 256
Mbytes
-fr[ag-queue] <size> Set fragment queue to <size> Mbytes. Default
256 Mbytes
-r[egex] treat extract names as POSIX regular expressions
rather than use the default shell wildcard
expansion (globbing)

Decompressors available:
gzip
lzma
lzo
lz4
xz

[MochiMoppel]

@tallboy: I don't see how less can help
@musher0: That works. Thanks!
I now came up with following conmmand:

Code: Select all

unsquashfs4 -d "" -ll /path/to/whatever.sfs | awk '/^(-|l)/ {printf "%10d %s %s\n",$3,$4,substr($0, index($0,$6))}' | sed 's/ /\xe2\x80\x87/g'

This will produce a 3 columns file list with filesize, modification date and filepath. Only files and symlinks are displayed since I don't care for (empty) directories. The only ugly part is the trailing sed command, which replaces all ASCII spaces with wider Unicode U+2007 ("FIGURE SPACE") spaces. I tried to integrate this simple replacement in the awk statement but failed.

[Edit] Corrected hex code for FIGURE SPACE (should be \xe2\x80\x87 , not \xe2\x80\x88)

[musher0]

less will produce a lot of output from an .sfs, but I am not familiar with what format it has, and how to decode it to a readable format.

Hello, tallboy.

Less will read and display the sfs file as a code file, unless some link to unsquashfs4
is included in less' companion script lesspipe.sh. (Untried; only a suspicion at this time.)

BFN.

[s243a]

It should be more efficient to mount the sfs rather than unsquash it. Mistfire has some code in her buildkit which mounts a sfs file.

[musher0]

@MochiMoppel: The display produced by your formula looks fine to me!

[musher0]

It should be more efficient to mount the sfs rather than unsquash it. Mistfire has some code in her buildkit which mounts a sfs file.

Hi s243a.

Euh... mounting an sfs file just to use ls on the mount would be a loss of disk space and
time, IMO. Especially since there are a couple of parms for listing in utility unsquashfs4.

BFN.

[MochiMoppel]

@musher0: With "ugly" I mean that the code looks ugly, not the result

BTW: The code contained an error in the hex value (see edit)

I now figured out a way to do the substitution within awk and remove the sed statement. I can't say that the code looks nicer
My hope was that it would be faster, but it isn't, Tends to be even slightly slower:

Code: Select all

unsquashfs4 -d "" -ll /path/to/whatever.sfs | awk '/^(-|l)/ {$3=sprintf("%10d",$3);gsub(" ","\xe2\x80\x87",$3); printf "%s %s %s\n",$3,$4,substr($0, index($0,$6))}' 

[MochiMoppel]

Sorry to revive this solved topic. I now noticed that the solution does not apply to pupsave files which may have extensions like .2fs. (MIME type application/x-ext2-image).

I assume that such image files are a completely different ball game and may have nothing to do with .sfs files. Still it would be nice to know what's inside. I tried the tools dumpe2fs and e2image without success and hardly knowing what I was doing. At least I did't break anything.

For encrypted pupsave files I may have to mount them, using SunBurnt's filemnt script, but for unencrypted files there must be an easier solution. Any ideas?

[musher0]

Hi MochiMoppel.

Indeed, Nfs files are not squashed.

To view their contents, it is enough to go to /initrd/pup_rw and list
the individual sub-directories
or
use on it something like the tree utility with appropriate level depth
(setting -L 10 is usually deep enough).

IHTH

[stemsee]

Hi MochiMoppel.

Indeed, Nfs files are not squashed.

To view their contents, it is enough to go to /initrd/pup_rw and list
the individual sub-directories
or
use on it something like the tree utility with appropriate level depth
(setting -L 10 is usually deep enough).

IHTH

Probably Mochi wants to list contained files in a *.2/3/4fs savefile image which is not in use, without mounting it, probably for mmview . So /initrd/pup_rw wont be relevant if the savefile is not the one used by the running system.

[musher0]

MochiMoppel? Is stemsee right?

[rufwoof]

less will produce a lot of output from an .sfs, but I am not familiar with what format it has, and how to decode it to a readable format.

OT to more recent discussion (around save files), but in answer to sfs's, the structure is relatively straight forward - but non trivial. The two bytes at byte offset 18 for instance indicate the compression method used by the sfs (1 for example (01 00) for gzip compression). 8 bytes at byte offset 60 points to where the inode table starts, and the 8 bytes at byte offset 68 point to where the Directory table starts. You could walk that to extract out the detail, but its non-trivial to do so. When I was looking at how easy it might be for a cracker to crack a sfs - for instance replace or extend a actual file to do something else in addition to its normal function, it was quicker/easier to target a particular known file contained within the sfs, search for that files signature (gzip'd (or whatever compression was being used by the sfs) data sequence) and replace that as the cracker desired. Which from a hackers perspective (someone trying to prevent crackers) suggests that a sfs should have both a checksum and filesize being validated as part of integrity checking. The filesize check is important otherwise crackers code can simply be appended to the end of a sfs, along with additional bytes in order to yield the same md5sum as the original files md5sum.

Returning to listing a sfs's content, its quicker/easier to simply mount and list the content of a fs than tree walking its binary/compressed content to determine its file/folder tree.

[rufwoof]

Save files are completely different to sfs's.

sfs's use their own format, whilst save files are file filesystems - created using the likes of

dd if=/dev/zero of=savefile bs=4096 count=1024 ... or whatever
then
mkfs.ext2 savefile

... that has a entirely different structure to that of sfs files created using something like

mksquashfs somefolder some.sfs

[MochiMoppel]

Save files are completely different to sfs's.

Unfortunately.

MochiMoppel? Is stemsee right?

Yes. For use in MMview I'm looking for an easy way to read the contents of such image files in a similar way as I can read .sfs, .iso or other archive files.
/initrd/pup_rw wouldn't be relevant in any case since it doesn't contain the contents of a pupsave file. You probably confused it with /initrd/pup_ro1

[rufwoof]

For file filesystems you can just mount them

mkdir mountpoint
mount some.2fs mountpoint

and then view the contents as per any other mounted filesystem

ls mountpoint/

Or use tree -d to display the folders (if you have tree installed), or otherwise something like

Code: Select all

ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/   /' -e 's/-/|/'

[MochiMoppel]

For file filesystems you can just mount them

s243a proposed the same for .sfs files.
I would prefer to avoid this, simply because I have not enough experience with mounting. I fear that somehow umount can fail, which will leave the user with a mounted file system. Pretty messy.

If indeed there is no tool similar to unsquashfs4 I might have to follow this road. The only problem: I have no experience with pupsave files either. I don't use them and I have only an an old unencrypted .2fs created in Puppy 4.2. which I can use for testing. Do .3sf and 4.sf flavors exist? What would happen if mount tries to mount an encrypted image? Would it gracefully fail?

Hi s243a.

Euh... mounting an sfs file just to use ls on the mount would be a loss of disk space and
time, IMO. Especially since there are a couple of parms for listing in utility unsquashfs4.

Thinking of it s243a's proposal might be not as bad as it appeared to you and me. I don't see how mounting would create "loss of disk space" and the speed of unsquashfs4 vs. mount+filetool+unmount is practically the same. The complexity is greater and more things can go wrong, but the biggest advantage is that code is needed only once. It applies to .sfs and .2fs alike.:

Code: Select all

SFSFILE=/path/to/whatever.sfs #works also with whatever.2fs pupsave
MOUNDIR=/tmp/mountpoint
mkdir  $MOUNDIR
mount "$SFSFILE" $MOUNDIR
find $MOUNDIR ! -type d  -printf '%9s %CY-%Cm-%Cd %y %p\n' | sed "s|$MOUNDIR||;s/ /\xe2\x80\x87/g" | leafpad
umount $MOUNDIR
rm -d  $MOUNDIR

First lines of output for my pupsave file:

Code: Select all

     1158 2016-09-14 f /usr/etc/cups/cupsd.conf
       29 2016-09-14 f /usr/local/bin/defaultconnect
     3255 2016-09-15 f /usr/local/ptrayDISABLED/ptray
     6476 2016-09-15 f /usr/local/ptrayDISABLED/func
      485 2016-09-15 f /usr/local/ptrayDISABLED/box_splash
  1031604 2016-09-14 f /usr/sbin/slmodemd
   514746 2016-09-14 f /usr/share/backgrounds/Pwidgets_background
     6961 2016-09-14 f /usr/share/doc/slmodem.txt
    42987 2016-09-14 f /usr/share/doc/index.html
   303296 2016-09-14 f /usr/share/fonts/default/TTF/cour.ttf
   480428 2016-09-14 f /usr/share/fonts/default/TTF/Bandal_korean.ttf
  1977316 2016-09-14 f /usr/share/fonts/default/TTF/M+1P+IPAG_japanese.ttf
       32 2016-09-15 f /usr/var/run/cups/certs/0
    16664 2016-09-15 f /usr/var/cache/fontconfig/02ee123a0860abc6ceda00dd4ed59b79-x86.cache-2
    16664 2016-09-15 f /usr/var/cache/fontconfig/bc06c1eea3e636f72101cafc3fb39508-x86.cache-2
       84 2016-09-14 f /usr/var/cache/cups/job.cache
     3664 2016-09-15 f /usr/var/log/cups/error_log
        4 2016-09-15 l /usr/X11R7/bin/X
        5 2016-09-14 l /usr/X11R6
       26 2016-09-15 l /root/bin
     1268 2016-09-15 f /root/Choices/ROX-Filer/PuppyPin