FAQ for the RSPerl package

July 6, 06

  • When running Perl inside R, I create a new Perl interpreter via .PerlInit() and later I close it via .PerlTerminate(). However, I get a lot of messages of the form
    Attempt to free non-existent shared string 'ldlibpthname' during global destruction.
    
    What's the problem?
    I have two versions of Perl installed on one of my machines. These are 5.8.0 and 5.8.8 in /usr and /usr/local. When I use the 5.8.8 version, I get these error messages. When I use 5.8.0, everything is fine.

    I haven't yet had the time to determine what the underlying cause of the problem is. Thread is one possibility. Fortunately, it doesn't seem to be a problem as we can continue the R session and we can also subsequently create new Perl interpreters that work.

    July 5, 06

  • When running Perl inside R, I try to load a module via .PerlPackage("myPackage") or .PerlExpr("use myPackage;"). But I get an error something like
    Error in .PerlExpr("use GCC::TranslationUnit;") : 
    	Error evaluating use GCC::TranslationUnit;: Can't load '/usr/local/lib/perl5/5.8.8/i686-linux/auto/Fcntl/Fcntl.so' for module Fcntl: /usr/local/lib/perl5/5.8.8/i686-linux/auto/Fcntl/Fcntl.so: undefined symbol: PL_sv_undef at /usr/local/lib/perl5/5.8.8/i686-linux/XSLoader.pm line 70.
     at /usr/local/lib/perl5/5.8.8/i686-linux/Fcntl.pm line 234
    
    where Fcntl is replaced by the package you are trying to load. What's the problem? and how can I fix it?
    The problem relates to how Perl loads modules with native code. When the modules are compiled and installed, it is expected that they will be used within the standalone Perl. All the Perl code (symbols such as PL_sv_undef) that these modules use is found when they are dynamically loaded into the running Perl application. When R is the host and Perl is embedded, these symbols are not in the host application, but kept in the separate table that is created when Perl is loaded.

    How do you get around this? There are two ways:

    1. When installing RSPerl, tell it which modules you will want to load that have native code that will be dynamically loaded. Do this with the --with-modules argument to configre (using the --configure-args option for R CMD INSTALL).
    2. The other approach is to force the Perl symbols to be available centrally via the host application. You can do this on UNIX, at least, by loading the RSPerl.so into R with the local argument set to FALSE, i.e.
      dyn.load(system.file("libs", paste("RSPerl", .Platform$dynlib.ext, sep = "")), package = "RSPerl") ,
                local  = FALSE)
      
      Do this BEFORE you load the RSPerl package with library(RSPerl) or it is implicitly loaded.

    Jun 7, 06

  • When I install, I'd like to have the Perl module files (i.e R.pm and RReferences.pm and the documentation files) be installed in the usual places for Perl packages. How can I do this?
    The simplest answer is to use --with-perl-prefix when specifying the configuration arguments to R CMD INSTALL. Use
      R CMD INSTALL --configure-args='--with-perl-prefix=' RSPerl
    
    Note that there is no value given for the argument. This tells the configuration script that you want Perl's own setting. (Alternatively you can specify a directory of your own chosing and the files will be installed under that.

    That wasn't so hard now, was it? Well the only problem is that when the configuration script asked Perl to install these files it probably failed as you did not have permission to write to those system-level directories. (You may have if you were running as root, but that is far from ideal. Do you really trust this configuration script in all its complexity not to do something dangerous. I know I don't and I wrote the script!).

    To fix this problem, assuming you didn't use the --clean argument for R CMD INSTALL, we can do that part of the installation manually. Either use sudo or explicitly login as root to perform the following task:

    • Change directory to RSPerl/src
    • give the command
      make -f Makefile.perl install
    There you go!
    Thanks to Michael Dondrup for motivating me to do this.
  • When running Perl within an R session, I am trying to load a Perl module. However, I just get a segmentation fault and a crash.
    I'm trying to find a good way to prohibit this from happening and catch error messages about the reason for the failure. Before I get to that however, the following may be helpful. The issue is probably to do with dynamically loaded modules in Perl running inside R. The way Perl loads modules that themselves have compiled native code, e.g. C, C++ or Fortran, is a little akward and this manifests itself when Perl is embedded in another application. When compiling the embedded Perl, we must tell it what modules we expect to load that have native code. And we must also get the dependencies right. For example, I wanted to load the IO module. But that depends on the Fcntl and Socket modules which have native, compiled code. So I need to tell the embedded Perl about those too when compiling it to go into R. To do this, use
           R CMD INSTALL --configure-args='--with-modules="IO Fcntl Socket"' RSPerl
    
    If you just specify IO and not the Fcntl and Socket, you'll see a segmentation fault after
    library(RSPerl)
    .PerlExpr("use IO;")
    
    When I try to install, the final link step in building RSPerl.so fails saying
    /usr/bin/ld: cannot find -lperl
    collect2: ld returned 1 exit status
    make: *** [RSPerl.so] Error 1
    ERROR: compilation failed for package `RSPerl'      
    
    This has occurred on Debian. Dirk Eddelbuettel (edd@debian.org), points out
    you need to install the matching -dev package which creates the link for you -- in this case the package is called "libperl-dev".

    The simple perl script

     perl -e 'use ExtUtils::Embed; print ldopts(), "\n"' ;  
    
    should return the correct options. However, for some reason it is not giving the correct values on some machines.
  • When I use RSPerl on the Mac, I get messages about not being able to find different Perl modules.
    Set the environment variable PERL5LIB, not PERLLIB.
  • When calling R from Perl to draw a plot everything works fine when I call the script without any redirection. When I redirect a data stream into the script via a pipe or from a file such as
       perl plot.pl < fileName
       bar | perl plot.pl
    
    the plot does not appear! What's the problem?
    In fact the plot is created but it is not displayed in the usual window. Instead, it is stored in the file Rplot.ps since the Postscript graphics device is the default when the R session is considered non-interactive. And it is considered non-interactive since the standard input stream is the redirected input. Essentially there is no terminal associated with the Perl process. This is an interaction between R and Perl when embedded and makes sense but may need to be overridable in the future. Connecting "connections" in Perl to R is also needed.

    To fix this "problem", just explicitly open a graphics device before calling plot. See plot1.pl in the tests directory of the package.

    When dealing with numbers in Perl and using them in R, I get errors about the wrong mode, etc. What's the problem?
    Perl tries to be helpful by working with strings and numbers alike without the user having to worry about the particular form. So $a = "1" can be used as a string or a number. However, when such a value is passed to R, we pass it is a string to R. When used in an R expression that expects a number, R raises an error.

    This arises when we use R to read numbers from a file or stream and then pass them directly to R. Instead, we must ensure that they are numbers in Perl before passing them to R. This can be done easily by merely adding 0 to the value as it is read. Or we can explicitly use R to convert the values to numbers (e.g. by calling as.numeric) and then use the result. This invilves multople transfers of the data. See stringNums.pl in the tests/ directory.

    I get errors saying Non-code reference passed via .Perl. Need a sub-routine name or Code object.
    More than likely, you are specifying both the name of a Perl routine and its package name, e.g.
     .Perl("myRoutine",  pkg = "MyPackage")
    
    One thing to do is to load the Perl module/package (MyPackage) before using the call
    .PerlPackage("MyPackage")
    
    Then the call
     .Perl("myRoutine",  pkg = "MyPackage")
    
    works fine.

    Alternatively, we need to fix the code to identify the Code object that gets resolved.

  • How do I find out where Perl is looking for modules.
    .PerlGet("INC", TRUE)
          
    You may want to do this if you have two versions of perl installed and want to verify that it is using the right one.
  • When I try to install the RSPerl package, I get an error message something like the following:
    ld: /System/Library/Perl/5.8.1/darwin-thread-multi-2level/auto/Apache/Leak/Leak.bundle is input for the dynamic link editor, is not relocatable by the static link editor again
    ld: /System/Library/Perl/5.8.1/darwin-thread-multi-2level/auto/Apache/Symbol/Symbol.bundle is input for the dynamic link editor, is not relocatable by the static link editor again 
    
  • When Perl is run inside R, and one wants to load a Perl module which uses C code, special handling is needed which the RSPerl package tries to do automatically. To exclude some C-based modules from this treatment, you can specify which ones to include, or none.

    To run R inside Perl (only), you do not need this support for loading Perl modules. Accordingly, the default for PERL_MODULES is no. So

        
       R CMD INSTALL --configure-args='--with-in-perl' RSPerl
    
    is the same as
        
       R CMD INSTALL --configure-args='--with-in-perl --with-modules=no' RSPerl
    
  • When running Perl from within R, I sometimes attempt load a Perl module and I get a horrific crash that terminates my R session. Can you help?
    This relates to the question above. If that module uses dynamically loaded C code in Perl, then you have to explicitly tell the RSPerl package to include it in the PERL_MODULES during the installation.
    Of course, the R session shouldn't crash. This is related to the presence of threads in the Perl installation. One can determine all of the configuration settings used to build the particular installation of Perl using
    perl -V
    and the command line arguments to Configure with perl -V:config_args If I build Perl with the command line
          ./Configure -de -Doptimize='-fPIC -g' ; make
          
    then the resulting version of Perl in /usr/local/bin behaves properly. Adding -Dusemultiplicity causes an error to terminate the process, e.g. .PerlPackage("Storable") just exits.

    July 6, 06

  • So you tell me I need to have specified the Perl modules that use compiled/native code when RSPerl is installed if I want to make use of them from within an R session. But how do I know which ones I need to specify.
    Well one "brute force" way is on Linux is to use strace to find all relevant operations on a .so file. We can do this with the following command where we load the Mail::Box::Mbox module
          
    strace -f -F -s 300 perl -e 'use Mail::Box::Mbox;' |& grep '/.*\.so"'  | grep -v '\(No such file or directory\)' | grep -v CORE | sed -e 's|.*/auto/\(.*\)/.*\.so".*|\1|' | sort | uniq
    
    All this does is run the Perl command to load the module and trace the system calls. We look only at the lines containing the string ".so" and exclude lines where the attempt to access the file was unsuccessful (i.e. there is a message "No such file or directory"). Then we exclude libperl.so itself which is in CORE and then get to work to extract the directory names after the /auto directory. We can then process the list further to convert / to :: to get the Perl module name when they are nested.

  • PePe
    Duncan Temple Lang <duncan@wald.ucdavis.edu>
    Last modified: Tue Jul 31 06:32:40 PDT 2007