iovation / rpmcpan Goto Github PK

I had trouble getting B::C to pass t/perldoc.t, I was seeing the same problem that was reported on cpantesters.

It seems to be due to confusion between the version of Cwd in the 5.18.4 base system and the newer version that some module pulled in during the build.

#   Failed test 'same result'
#   at t/perldoc.t line 102.
#          got: 'Cwd object version 3.40 does not match $Cwd::VERSION 3.47 at (eval 11) line 68.
# '
#     expected: 'No documentation found for "-T".

I did not see the failure in t/issue24.t that cpantesters saw.

I fixed it with a big hammer:

--- a/t/perldoc.t   2015-05-22 21:34:22.943582144 +0000
+++ b/t/perldoc.t   2015-05-22 21:34:33.579898843 +0000
@@ -1,6 +1,6 @@
 #! /usr/bin/env perl
 # brian d foy: "Compiled perlpod should be faster then uncompiled"
-use Test::More;
+use Test::More skip_all => 'Skip failing tests, conflict w/ dual-lived Cwd modules';
 use strict;
 BEGIN {
   unshift @INC, 't';

Not sure if there's a similar problem with a more current perl release and/or if this patch is worth conditionalizing somehow or.....

Getting rpmcpan up into the air has been a bit of an awkward dance. I wonder if there's a better way, or if not then if a bit of docs would help.

I started with a simple virtualbox VM based on NREL's Centos 6.6 box.

After a bit of trial and error, I've learned to install these (some are things I need for my trimmed etc/dists.json):

sudo yum install perl-devel libxml2-devel expat-devel git rpm-build gcc=c++

Then I cloned my fork of the iovation rpmcpan repository.

When I tried to run bin/rpmcpan I was missing a double handful of prerequisites.

So I built a copy of cpanm for the system perl and used that to install the prereqs.

That let things run up until the point that rpmcpan relaunches itself using the perl that it built. That perl trips over the prereqs I built for the system perl and seems to need its own copy of them.

So I ended up building a cpanm and a set of prereqs for the new perl and that has let me proceed to assembling a set of rpms for the various distros I need.

In short, it looks like this:

sudo yum -y install perl-devel libxml2-devel expat-devel git rpm-build gcc-c++
git clone ...
curl https://raw.githubusercontent.com/miyagawa/cpanminus/master/cpanm | perl - -l local.system App::cpanminus
env PERL5LIB=`pwd`/local.system/lib/perl5 PATH=`pwd`/local.system/bin:$PATH cpanm -l local.system < rpmcpan-prereqs
sudo env PERL5LIB=`pwd`/local.system/lib/perl5 PATH=`pwd`/local.system/bin:$PATH ./bin/rpmcpan --version 5.18.4 --prefix=/opt/perl518 -v
#dies here
curl https://raw.githubusercontent.com/miyagawa/cpanminus/master/cpanm | /opt/perl518/bin/perl - -l local.opt App::cpanminus
env PERL5LIB=`pwd`/local.opt/lib/perl5 PATH=`pwd`/local.opt/bin:/opt/perl518/bin:$PATH cpanm -l local.opt < rpmcpan-prereqs
sudo env PERL5LIB=`pwd`/local.opt/lib/perl5 PATH=`pwd`/local.opt/bin:/opt/perl518/bin:$PATH ./bin/rpmcpan --version 5.18.4 --prefix=/opt/perl518 -v

1. How did you get the initial build (using the system perl to run rpmcpan) to work? Did you load a bunch of system perl module rpms?
2. How did it happen that the system was able to restart with the freshly built perl w/out freaking out?

Thanks!

etc/perl.spec includes these three lines after the %description paragraph.

Provides: iov-%{sname}
Provides: %{sname}%{plv}(:WITH_ITHREADS)
Provides: %{sname}%{plv}(:WITH_PERLIO)

I tried to use one of these symbols in a Requires statement in another RPM I'm building and it didn't work. Some digging showed that they weren't getting included in rpm file, e.g the output of:

rpm -qp --provides repo/perl518-5.18.4-4.1432570286.el6.x86_64.rpm

did not include them.

However, they are included as the last three lines of the rpm's description, e.g. in the output of running:

rpm -qp --info repo/perl518-5.18.4-4.1432570286.el6.x86_64.rpm

It appears that the %description block swallows everything up until the next rpm section (e.g. %prep), or something like that.

I've placed them up after the Requires(postun): line and now they no longer appear in the %description section (phew) and have an effect.

Unfortunately, now I'm seeing a hard to reproduce bug in which rpath in incorrectly defined, resulting a failure to find libperl.so at runtime. I'll file another issue describing what I've narrowed down about that problem.

The command line that I'm running is

sudo env PERL5LIB=`pwd`/local.system/lib/perl5 PATH=`pwd`/local.system/bin:$PATH ./bin/rpmcpan --version 5.18.4 --prefix=/opt/perl518 -v -v -v [email protected]

Issue #5 describes missing "provides" information in the perl rpm that rpmcpan builds.

This Issue describes a heisenbug that I'm seeing in which the perl binary is built with an incorrect rpath and fails to find libperl.so.

Feel free to read through it if you'd like, but after wrestling with in the background all weekend, I think that it's either an OS/tools problem or an Openstack/nebula problem. I'm raising it anyway in case it comes up again, but I think you might want to just close it. Executive summary at the end.

I've been working on an Openstack based cluster (from Nebula) with a Centos 6 image at work and I've been seeing variations of the problem described below all weekend.

I moved the Provides lines in etc/perl.spec up underneath the Requires(postun) line (issue #5) and was able to build an rpm that included that information, so I patted myself on the back and moved on.

While doing more builds (localizing other things in the spec) I suddenly ended up with a perl that was unable to run because it couldn't find libperl.so. Using LD_DEBUG I can see that the rpath is incorrect (this output is from a recent failed experiment, not the original failure, but it's representative):

[...]
- Collecting and installing perl RPMs
Preparing...                ########################################### [100%]
   1:perl518                ########################################### [100%]
Perl 5.18.4 built; restarting with that version

/opt/perl518/bin/perl: error while loading shared libraries: libperl.so: cannot open shared object file: No such file or directory
[centos@perl-builder rpmcpan.gne]$ LD_DEBUG=libs /opt/perl518/bin/perl
      5924: find library=libperl.so [0]; searching
      5924:  search path=/opt/perl518/lib/5.18.4/x86_64-linux-thread-multi/CORE/tls/x86_64:/opt/perl518/lib/5.18.4/x86_64-linux-thread-multi/CORE/tls:/opt/perl518/lib/5.18.4/x86_64-linux-thread-multi/CORE/x86_64:/opt/perl518/lib/5.18.4/x86_64-linux-thread-multi/CORE      (RPATH from file /opt/perl518/bin/perl)
      5924:   trying file=/opt/perl518/lib/5.18.4/x86_64-linux-thread-multi/CORE/tls/x86_64/libperl.so
      5924:   trying file=/opt/perl518/lib/5.18.4/x86_64-linux-thread-multi/CORE/tls/libperl.so
      5924:   trying file=/opt/perl518/lib/5.18.4/x86_64-linux-thread-multi/CORE/x86_64/libperl.so
      5924:   trying file=/opt/perl518/lib/5.18.4/x86_64-linux-thread-multi/CORE/libperl.so
      5924:  search cache=/etc/ld.so.cache
      5924:  search path=/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib64        (system search path)
      5924:   trying file=/lib64/tls/x86_64/libperl.so
      5924:   trying file=/lib64/tls/libperl.so
      5924:   trying file=/lib64/x86_64/libperl.so
      5924:   trying file=/lib64/libperl.so
      5924:   trying file=/usr/lib64/tls/x86_64/libperl.so
      5924:   trying file=/usr/lib64/tls/libperl.so
      5924:   trying file=/usr/lib64/x86_64/libperl.so
      5924:   trying file=/usr/lib64/libperl.so
      5924:
/opt/perl518/bin/perl: error while loading shared libraries: libperl.so: cannot open shared object file: No such file or directory

and if I look back through my terminal window's history (having run with -v -v -v) I can find that perl was indeed linked with that rpath.

cc -o libperl.so -shared -O2 -L/usr/local/lib -fstack-protector op.o perl.o   gv.o toke.o perly.o pad.o regcomp.o dump.o util.o mg.o reentr.o mro.o keywords.o hv.o av.o run.o pp_hot.o sv.o pp.o scope.o pp_ctl.o pp_sys.o doop.o doio.o regexec.o utf8.o taint.o deb.o universal.o globals.o perlio.o perlapi.o numeric.o mathoms.o locale.o pp_pack.o pp_sort.o   DynaLoader.o -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
LD_LIBRARY_PATH=/home/centos/rpmcpan.gne/BUILD/perl\-5.18.4 cc -o perl  -fstack-protector -L/usr/local/lib -Wl,-E -Wl,-rpath,/opt/perl518/lib/5.18.4/x86_64-linux-thread-multi/CORE perlmain.o  libperl.so `cat ext.libs` -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
Processing extracted/DLineBreak.txt
LD_LIBRARY_PATH=/home/centos/rpmcpan.gne/BUILD/perl\-5.18.4  ./miniperl -Ilib make_ext.pl lib/auto/B/B.so MAKE=make LIBPERL_A=libperl.so LINKTYPE=dynamic

What's crazy is that in this situation, the only file in the BUILD dir that contains the string "/opt/perl518/lib/5.18.4/x86_64-linux-thread-multi/CORE'" is "myconfig"

[centos@perl-builder rpmcpan.gne]$ find . -type f | xargs grep '/opt/perl518/lib/5.18.4/x86_64-linux-thread-multi/CORE'
Binary file ./BUILD/perl-5.18.4/perl matches
./BUILD/perl-5.18.4/myconfig:    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/opt/perl518/lib/5.18.4/x86_64-linux-thread-multi/CORE'
grep: ./BUILD/perl-5.18.4/t/pleaseletthisfilenotexist.pag: Permission denied
grep: ./BUILD/perl-5.18.4/t/pleaseletthisfilenotexist.dir: Permission denied

which I believe is not actually used for anything in the build. Everything else seems to have use the path that's edited in the %build step that edits config.sh in place before calling make.

I've thrashed around for a while testing various hypothesis and trying to isolate/reproduce what I'm seeing. My general setup looks like this (see Issue #3 for details):

• In order to get rpmcpan to work, the system perl seems to need a handful of things, which I've provided by populating and using a local library like this:

  curl https://raw.githubusercontent.com/miyagawa/cpanminus/master/cpanm | perl - -l local.system App::cpanminus
  env PERL5LIB=`pwd`/local.system/lib/perl5 PATH=`pwd`/local.system/bin:$PATH cpanm -l local.system < rpmcpan-prereqs
  

• then I build like this:

  sudo env PERL5LIB=`pwd`/local.system/lib/perl5 PATH=`pwd`/local.system/bin:$PATH ./bin/rpmcpan --version 5.18.4 --prefix=/opt/perl518 -v [email protected]
  

  which (this heisenbug not withstanding) will build an rpm and populate /opt/perl518 and then crash when restarting with the newly built perl because there are things in local.system that are /usr/bin/perl specific. I've solved that by buidling a local.opt for the newly built perl and running rpmcpan again using the local.opt content (see Isssue #3 for details).

While chasing the heisenbug I clean up between builds by running this little script:

rm -rf BUILD/*
rm -rf BUILDROOT/*
rm -rf RPMS/*
rm -rf SOURCES/*.tar.gz
rm -rf SOURCES/*.bz2
rm -rf SOURCES/*.tgz
rm -rf SOURCES/*.zip
rm -rf SRPMS/*
rm -rf SPECS/*
rm -rf repo/*

and also

sudo rm -rf /opt/perl518

I started off thinking that maybe the Provides lines needed to be somewhere else in the file, which led to heisenbug behavior. Then I thought maybe one of them was causing a problem (some weird interaction with some C macro somewhere or ...) and tried commenting out various combos, still more heisenbug behavior. At one point I commented out the actual execution of the tests to speed things up, but still heisenbug behavior.

I'm trying to work at this systematically, here's an odd thing that I've noticed.

1. If I have a build that has generated a perl with an incorrect rpath and I comment out all three Provides lines;
2. The next build fails, so I clean up and ...
3. The build after that fails, so I clean up and ...
4. The build after that fails, so I clean up and ...
5. The build after that succeeds.

even though I've made no changes (other than the "cleanup" described above). Before I started taking careful notes I believe that I saw the recovery on the 4th step, one iteration shorter.

And now that I've been taking notes, I've become convinced that this is some sort of platform bug. Two things persuade me of that:

• I can't reproduce it in a Centos vagrant/virtualbox vm running on my mac
• I've now seen failures/recoveries after 2 iterations, 3 iterations and 4 iterations.

For now, I'm going to go with rpm's built in a vagrant vm and maintain a bit of skepticism when using the nebula system.