How ppm finds missing libs&Potential Arch32Pup Improvements

[s243a]

For better dependency resolution on Arch32Pup, I'm considering using the %PROVIDES% field that is provided by the arch linux repos in order to find the packages which contain the missing libs. From there, there are three options. These options are:
1. only install the missing libs on their system (i.e. pupngo style),
2. install the whole package or,
3. don't install the missing runtimes and load them as SFSs when needed.

These packages that have the %PROVIDES% filed in their metadata likely have lib in their package name and are likely the runtime for a given package which is listed as a "%MAKEDEPENDS% package for one of the packages installed by the ppm.

Puppy actually already does some of the work here because the puppy ppm, identifies which libs are missing after it finishes installing a package. The first thing I would like to do is trace some of this process. This will give me an idea of how we might incorporate this suggested feature into the ppm.

[s243a]

As promised, here is a rough trace of the process which leads the ppm to identify the missing libs.

The first script that is called when the ppm (puppy package manager) starts is called pkg_chooser.sh. This script defines the first gui that you see when the ppm starts. At some point when you tell this gui to install one or more packages the function do_install() is called. In this function the script installmodes.sh is called:

Code: Select all

	if ! [ -f /tmp/petget_proc/force_install -a -f /tmp/petget_proc/install_pets_quietly ]; then
		#/usr/local/petget/installed_size_preview.sh "$NEWPACKAGE" ADD
		/usr/local/petget/installmodes.sh "$INSTALL_MODE"
	fi

/rootfs-skeleton/usr/local/petget/pkg_chooser.sh#L161

installmodes.sh contains the gui that says which libs are missing:

Code: Select all

MISSING_LIBS=$(cat /tmp/petget_proc/overall_missing_libs.txt 2>/dev/null | tr ' ' '\n' | sort | uniq )

/rootfs-skeleton/usr/local/petget/installmodes.sh#L89

Code: Select all

# Info window/dialogue (display and option to save "missing" info)
 if [ "$MISSING_LIBS" ];then
  MISSINGMSG1="<i><b>$(gettext 'These libraries are missing:')
${MISSING_LIBS}</b></i>"
  LM='  <hbox space-expand="true" space-fill="true">
    <hbox scrollable="true" hscrollbar-policy="2" vscrollbar-policy="2" space-expand="true" space-fill="true">
      <hbox space-expand="false" space-fill="false">
        <eventbox name="bg_report" space-expand="true" space-fill="true">
          <vbox margin="5" hscrollbar-policy="2" vscrollbar-policy="2" space-expand="true" space-fill="true">
            '"`/usr/lib/gtkdialog/xml_pixmap building_block.svg 32`"'
            <text angle="90" wrap="false" yalign="0" use-markup="true" space-expand="true" space-fill="true"><label>"<big><b><span color='"'#bbb'"'>'$(gettext 'Libs')'</span></b></big> "</label></text>
          </vbox>
        </eventbox>
      </hbox>
      <vbox scrollable="true" shadow-type="0" hscrollbar-policy="1" vscrollbar-policy="1" space-expand="true" space-fill="true">
        <text ypad="5" xpad="5" yalign="0" xalign="0" use-markup="true" space-expand="true" space-fill="true"><label>"'${MISSINGMSG1}'"</label></text>
      </vbox>
    </hbox>
  </hbox>'
 fi

/rootfs-skeleton/usr/local/petget/installmodes.sh#L105

The missing libs are identified in check_deps.sh

Code: Select all

   MISSINGLIBS="`ldd "${ONEFILE}" | grep "not found"`"
   if [ ! "$MISSINGLIBS" = "" ];then
    MISSINGLIBS="`echo "$MISSINGLIBS" | cut -f 2 | cut -f 1 -d " " | tr "\n" " "`"
    echo "$(gettext 'File') $ONEFILE $(gettext 'has these missing library files:')" >> /tmp/petget_proc/missinglibs_details.txt #100718
    echo " $MISSINGLIBS" >> /tmp/petget_proc/missinglibs_details.txt #100718
    echo " $MISSINGLIBS" >> /tmp/petget_proc/missinglibs.txt #100718
   fi ;;

/rootfs-skeleton/usr/local/petget/check_deps.sh#L112

which is called from installpreview.sh

Code: Select all

PKGS="`cat /tmp/petget_proc/petget_missing_dbentries-* | cut -f 1 -d '|' | tr '\n' '|'`"
/usr/local/petget/check_deps.sh $PKGS

/rootfs-skeleton/usr/local/petget/installpreview.sh#L447

installpreview.sh is called from the install_package() function of installmodes.sh

Code: Select all

   if [ -f /tmp/petget_proc/install_quietly ];then
    if [  "$(grep $TREE1 /root/.packages/user-installed-packages 2>/dev/null)" = "" \
     -a -f /tmp/petget_proc/install_pets_quietly ]; then
     if [ "$(cat /var/local/petget/nt_category 2>/dev/null)" = "true" ]; then
      /usr/local/petget/installpreview.sh
     else
	  rxvt -title "$VTTITLE... $(gettext 'Do NOT close')" \
	  -fn -misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-*-* -bg black \
      -fg grey -geometry 80x5+50+50 -e /usr/local/petget/installpreview.sh
     fi
    else
     if [ "$(cat /var/local/petget/nt_category 2>/dev/null)" = "true" ]; then
      /usr/local/petget/installpreview.sh
     else
	  rxvt -title "$VTTITLE... $(gettext 'Do NOT close')" \
	  -fn -misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-*-* -bg black \
      -fg grey -geometry 80x5+50+50 -e /usr/local/petget/installpreview.sh
     fi
    fi

/woof-code/rootfs-skeleton/usr/local/petget/installmodes.sh#L419

One point of interest of this section of code is that it is where the rxvt window pops up that says "do not close" and gives some indication of what is installing. This is an important part of the code to know about if one is trying to trace what is going on.

Anyway, that is a quick walk-through of the parts of the code I can find which lead up to the indication of what shared libraries are missing. As I said in the first post, the arch linux repos have metadata to help find the package that one needs to isntall in order to rectify a given missing lib.

[s243a]

Time for some new code now. Here is a link to a slightly modified version of /usr/local/petget/0setup, which was found in arch32pup20.02+9

https://pastebin.com/7LSHJzB2

The original code also generates files of the form:

/var/packages/Packages-*

the slightly modified code generates packages of the form:

/var/packages/Provides-*

Here is an example record in this new table:

boost-libs-1.71.0|boost-libs|1.71.0|4.1|+libboost_atomic.so=1.71.0-32,+libboost_chrono.so=1.71.0-32,+libboost_container.so=1.71.0-32,
+libboost_context.so=1.71.0-32,+libboost_contract.so=1.71.0-32,+libboost_coroutine.so=1.71.0-32,
+libboost_date_time.so=1.71.0-32,+libboost_fiber.so=1.71.0-32,+libboost_filesystem.so=1.71.0-32,+libboost_graph.so=1.71.0-32,
+libboost_graph_parallel.so=1.71.0-32,+libboost_iostreams.so=1.71.0-32,+libboost_locale.so=1.71.0-32,+libboost_log.so=1.71.0-32,
+libboost_log_setup.so=1.71.0-32,+libboost_math_c99.so=1.71.0-32,+libboost_math_c99f.so=1.71.0-32,
+libboost_math_c99l.so=1.71.0-32,+libboost_math_tr1.so=1.71.0-32,+libboost_math_tr1f.so=1.71.0-32,
+libboost_math_tr1l.so=1.71.0-32,+libboost_mpi.so=1.71.0-32,+libboost_numpy27.so=1.71.0-32,+libboost_numpy38.so,
+libboost_prg_exec_monitor.so=1.71.0-32,+libboost_program_options.so=1.71.0-32,+libboost_python27.so=1.71.0-32,
+libboost_python38.so=1.71.0-32,+libboost_random.so=1.71.0-32,+libboost_regex.so=1.71.0-32,+libboost_serialization.so=1.71.0-32,
+libboost_stacktrace_addr2line.so=1.71.0-32,+libboost_stacktrace_basic.so=1.71.0-32,+libboost_stacktrace_noop.so=1.71.0-32,
+libboost_system.so=1.71.0-32,+libboost_thread.so=1.71.0-32,+libboost_timer.so=1.71.0-32,+libboost_type_erasure.so=1.71.0-32,
+libboost_unit_test_framework.so=1.71.0-32,+libboost_wave.so=1.71.0-32,+libboost_wserialization.so=1.71.0-32|

+icu,+python,+python2,
+python-numpy,+python2-numpy,+bzip2,+zlib,+openmpi,+zstd,+findutils|

+openmpi: for mpi support|

**Extra line breaks added for readability, which aren't part of the actual output.

The fields are as follows:

Fullname|pkgname|pkgversion|build#|Provides|make depends|Optional Depends|

These fields are printed as follows (awk code):

Code: Select all

   printf("%s-%s|%s|%s|%s|%s|%s|%s|\n", pkgname, pkgverarr[1], pkgname, pkgverarr[1], pkgverarr[2],provides,mkdeps,optdeps) >> "sandbox0/archtemp2"

and then the file is moved (bash/ash code):

Code: Select all

mv sandbox0/archtemp2 ${RUNNINGPUP}Provides-arch-32-${arepo}

At this point, I"m mainly interested in the "provides" filed because it will be useful for finding which missing packages to install. I envision a gui like the one in puppy with check-boxes that lets you decide which packages that you want to install in order to resolve these missing libs.