jhalfs and the book xml

Peter Palmer peter at galama.org
Sat Feb 25 10:54:59 PST 2012

I'm just moving this to a new thread.
On 25 February 2012 14:11, Pierre Labastie <pierre.labastie at neuf.fr> wrote:
> Le 24/02/2012 20:20, Peter Palmer a écrit :
>> On 24 February 2012 16:45, Pierre Labastie<pierre.labastie at neuf.fr>  wrote:
>>> Le 24/02/2012 15:46, Peter Palmer a écrit :
>>>> On 24 February 2012 13:13, Pierre Labastie<pierre.labastie at neuf.fr>    wrote:
>>>>> I have seen that CLFS is now on git. Have you tried
>>>>> to replace svn instructions by "equivalent" git ones
>>>>> in jhalfs? I do not know much about git, but I know
>>>>> rather well jhalfs, so if I may be of some help...
>>>> First thanks to Thomas for giving me an early pointer to the problem I
>>>> introduced.
>>>> I was just working on another patch but Pierre beat me to it. I don't
>>>> bother posting as the only difference is the name for the new variable
>>>> :)
>>> Sorry for that. I always think it is easier to explain with code...
>>>> I never worked with svn as vcs so I use git locally to work with
>>>> jhalfs. I've got one branch that I keep in sync with the svn trunk.
>>>> Branching is very easy in git so I create a branch for whatever
>>>> project I work on. To get the diff I just run:
>>> Just to make sure we understood each other (I am not a native
>>> English speaker), I meant that jhalfs had some instructions to
>>> download on the fly the LFS book from SVN repo. I was not talking
>>> about managing jhalfs mods with GIT. Seeing how the philosophy
>>> of GIT is to have a local "working copy" (+ repo), those download
>>> instructions are maybe not necessary for CLFS.
>>> Since I am also busy with the blfs tools, I think I won't add GIT
>>> download instructions now.
>> Pierre, I was on a different planet there. I am here now:
>> ./jhalfs:
>> case $PROGNAME in
>> # TODO: clfs is now on git
>> #  clfs* ) declare -r SVN="http://svn.cross-lfs.org/svn/repos" ;;
>>        * ) declare -r SVN="svn://svn.linuxfromscratch.org" ;;
>> esac
>> ~~~~~and here:~~~~~
>> common/libs/func_book_parser:48:
>> svn co $SVN/${svn_root}/${TREE} ${PROGNAME}-$LFSVRS>>$LOGDIR/$LOG 2>&1
>> IMHO for a casual user there is little difference between svn, git or
>> a tarball and the git philosophy doesn't matter.  Personally, even
>> when working on a development branch I would always want to be able to
>> fix a particular version of a book as this makes error hunting a lot
>> easier. However I can see how the svn/git option is a useful feature
>> for some; both for LFS and CLFS (and BLFS of course).
>> I would propose to add something like this though:
>> if [[ -d $SVN ]] ; then
>>    echo --n "You seem to have a working copy of the book already. Do
>> you want to update this to the newest version? (no)"
>>    read ANSWER
>> etc...
>> What do you think?
>> In the mean time I'll have a look at the CLFS/git download.
> Peter,
> There is no need to test $SVN, I think, because in the menuconfig
> interface, the user is given the choice of using either local book sources,
> or sources downloaded from a remote repository.
Long text below covers that :o)
> My remark about "philosophy" was just that with GIT, it does not seem
> to make much difference whether the repository is local or remote, once
> you have 'git clone'd one.
Git is like a database that contains the whole development history of
a project. You can tell git to go back to earlier commits of other
developers etc. because you've got all that in your local repository.
If the online copy gets lost you can use your local copy to restore
that repository up to the state where you did your last fetch. It does
not stay updated automatically though.
SVN (so I believe) just syncs your current directory with the
repository without pulling all that metadata.
In both cases you can:
svn co / git clone
to get the initial download
svn up /git remote fetch
to bring your data up to date.
if you can ignore all the metadata, don't modify in the directory you
work on, the result is effectively the same.
My point is in reference to the SVN feature in jhalfs. Most people
want the snapshot of the data, they don't plan to do development work.
If you look from that point of view svn and git will look to you the
> As you (and other recently) said, it is not very common to need to
> download a book source on the fly. What I do with LFS is to have a local
> working copy of the current development book, and I usually don't need it
> either. Only when Bruce asked to test the 7.1-rc1 version did I downloaded
> it, since it was not supposed to be modified (of course) for tests...
> AFAICT, you pointed all the places where the code should be modified.
> By browsing the source with Trac, I see that there are a lot of branches.
> So a user might want to download a specific version (say 1.2 or 1.1.0..),
> to try it. I do not know how to do that with git. Looks like you need to
> somehow convert those version numbers into some hexadecimal
> string. With SVN, the encoding is done in $svn_root/$TREE.

May I propose two solutions (a short and a long one) to that problem.

Short: In menuconfig lets take all the options under 'release' out and
only leave the string field for the path to the book sources. Maybe
get some instructions into the help file where sources can be obtained
and get on with other things ;)

Long: On the other hand it would be nice having all the other options
working again. I had a look around anyway when I still thought 'set
git up' was a quick job. This is what I wrote earlier:

I am trying to remember the fist time I used jhalfs. I spent a lot of
time messing around with the download options because I assumed that
way I had the best chance to get a version of the book that works well
with jhalfs. To get to the point that I got the book myself and used
'working copy' took quite a while. I think this can and should be made
easier and more intuitive.

The SVN option... you say yourself you switched after the first
download to 'working copy'. I think having to do this is a workaround
to a bug though that is not obvious to a new user.
When working with the SVN (or any other) option we should tell the
user in the help what jhalfs is going to do in regard to updating
before starting a build that may take ... on a very slow computer as
long as days (CLFS still takes 12 hours here when run on one processor

Moreover there is a funny bug:
The SVN download works fine on the first run.
Every subsequent run it does a 'svn up' but does not extract a new set
of scripts. So it does stay consistent with the initial download
_except_ if you delete the lfs build dir or... svn tips over - not an
unlikely thing if your network connection is not stable. I even tested
that because I couldn't believe I read the script right:

      if LC_ALL=C svn up | grep -q At && \
         test -d $JHALFSDIR/${PROGNAME}-commands && \
         test -f $JHALFSDIR/pkg_tarball_list ; then
        # Set the canonical book version
        echo -ne "done\n"
        cd $JHALFSDIR
        case $PROGNAME in
          clfs | clfs2 | clfs3 )
            VERSION=$(xmllint --noent
$BOOK/prologue/$ARCH/bookinfo.xml 2>/dev/null | grep subtitle | sed -e
's/^.*ion //'  -e 's/<\/.*//') ;;
            VERSION=$(xmllint --noent $BOOK/prologue/bookinfo.xml
2>/dev/null | grep subtitle | sed -e 's/^.*ion //'  -e 's/<\/.*//')
        echo -ne "done\n"

The example above is by the way the only path where extract_commands
is not run by get_book. IMHO this should only run the first time or
when requested by selecting the option 'rebuild makefile' in the menu.
This needs to be implemented in a different way. But before that can
be done it needs to be clear what it's supposed to do. Please bear
with me and allow me to start from the top.

This is how it currently works: if you look at menuconfig there are
three choices in the release section (they are the same for everything
except BLFS and set the same vars in the configuration file by the

2.Working Copy
3.Branch or stable

of these only #2 is working correctly. #1 as I already pointed out is
working correctly on the first run but does weird things if run
repeatedly. #3 is wired up in a way that it basically does the same as
#2 (and is thus currently redundant).

I think #2 is the main option for experienced users anyway and it's
working. Fine.

About #3... has anyone read the help for that and followed the link?
It is not clear what to enter. People will spend ages and get
frustrated trying to figure out what to do. Then when you read the
script this is actually all it does:
a. unnecessarily bomb out if you leave the default
b. exactly the same as #1

Here is my understanding how it should work:

1. establish we've got a book by
1.1 SVN - (should be renamed to 'development' to avoid confusion with
svn and git):
if !$local_copy_present
  svn co (or equivalent)
# New option on the menu 'update svn' that becomes available if we
select 'rebuild the makefile'.
  if $update_svn
    svn up
1.2 Working copy
if !$local_copy_present

1.3 Branch or stable:
This needs to change on the menu.
a.) rename to 'stable' only. I expect only experienced users to work
with branches and they would want to use 'working copy' anyway.
b.) get rid of the string. Instead use radio buttons that point to
actual hardcoded stable releases.
c.) for the time being though take this option off the menu until we
have the code behind that in place.
And then:
if !$local_copy_present

2. Extract the version (use as sanity check of the book)
3. Only run extract_commands if we are (re-)building the whole makefile as well.
N. Do the rest

Pierre, you are probably aware that the code around get_book looks
very ... grown. I've got a fair understanding now of how the cogs and
wheels fit together so I started here pruning things a bit. I am quite
sure this is not in conflict with any of the blfs code so I hope there
are no issues with your work.

I am currently working on these files:


Thanks for your time,


More information about the alfs-discuss mailing list