Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Perl: porters

[perl #113022] Configure misdetects strlcpy et al. with gcc -flto

 

 

Perl porters RSS feed   Index | Next | Previous | View Threaded


perlbug-followup at perl

May 21, 2012, 3:07 AM

Post #1 of 3 (244 views)
Permalink
[perl #113022] Configure misdetects strlcpy et al. with gcc -flto

# New Ticket Created by l.mai [at] web
# Please include the string: [perl #113022]
# in the subject line of all future correspondence about this issue.
# <URL: https://rt.perl.org:443/rt3/Ticket/Display.html?id=113022 >



This is a bug report for perl from l.mai [at] web,
generated with the help of perlbug 1.39 running under perl 5.16.0.


-----------------------------------------------------------------
[Please describe your issue here]

Hi,

while trying to build perl-5.16.0 I ran into this problem:

...
cgcc -fstack-protector -L/usr/local/lib -O2 -flto -o miniperl \
perlmini.o opmini.o miniperlmain.o gv.o toke.o perly.o pad.o regcomp.o dump.o util.o mg.o reentr.o mro.o keywords.o hv.o av.o run.o pp_hot.o sv.o pp.o scope.o pp_ctl.o pp_sys.o doop.o doio.o regexec.o utf8.o taint.o deb.o universal.o globals.o perlio.o perlapi.o numeric.o mathoms.o locale.o pp_pack.o pp_sort.o -lnsl -ldl -lm -lcrypt -lutil -lc
/tmp/ccn4nr6G.ltrans6.ltrans.o: In function `Perl_sv_usepvn_flags':
ccn4nr6G.ltrans6.o:(.text+0x19c5): undefined reference to `malloc_size'
ccn4nr6G.ltrans6.o:(.text+0x1a8b): undefined reference to `malloc_size'
/tmp/ccn4nr6G.ltrans9.ltrans.o: In function `Perl_find_script':
ccn4nr6G.ltrans9.o:(.text+0x453f): undefined reference to `strlcpy'
/tmp/ccn4nr6G.ltrans9.ltrans.o: In function `Perl_more_bodies':
ccn4nr6G.ltrans9.o:(.text+0x73b7): undefined reference to `malloc_good_size'
/tmp/ccn4nr6G.ltrans25.ltrans.o: In function `Perl_sv_grow':
ccn4nr6G.ltrans25.o:(.text+0x172b): undefined reference to `malloc_size'
ccn4nr6G.ltrans25.o:(.text+0x1760): undefined reference to `malloc_size'
/tmp/ccn4nr6G.ltrans4.ltrans.o: In function `Perl_av_extend':
ccn4nr6G.ltrans4.o:(.text+0x721): undefined reference to `malloc_size'
/tmp/ccn4nr6G.ltrans28.ltrans.o: In function `Perl_magic_set':
ccn4nr6G.ltrans28.o:(.text+0x1ca3): undefined reference to `setproctitle'
ccn4nr6G.ltrans28.o:(.text+0x2659): undefined reference to `setrgid'
ccn4nr6G.ltrans28.o:(.text+0x26d8): undefined reference to `setruid'
collect2: ld returned 1 exit status
make: *** [miniperl] Error 1

This is caused by Configure thinking I have malloc_size, strlcpy, etc. when I
do not.

I was able to fix this problem with the following patch:

--- perl-5.16.0-orig/Configure 2012-04-25 02:18:30.000000000 +0200
+++ perl-5.16.0/Configure 2012-05-15 08:11:20.000000000 +0200
@@ -7782,13 +7782,13 @@
if $contains $tlook $tf >/dev/null 2>&1; then
tval=true;
elif $test "$mistrustnm" = compile -o "$mistrustnm" = run; then
- echo "$extern_C void *$1$tdc; void *(*(p()))$tdc { return &$1; } int main() { if(p()) return(0); else return(1); }"> try.c;
+ echo "$extern_C void *$1$tdc; void *(*(p()))$tdc { return &$1; } int main() { if(p() && p() != (void *)main) return(0); else return(1); }"> try.c;
$cc -o try $optimize $ccflags $ldflags try.c >/dev/null 2>&1 $libs && tval=true;
$test "$mistrustnm" = run -a -x try && { $run ./try$_exe >/dev/null 2>&1 || tval=false; };
$rm_try;
fi;
else
- echo "$extern_C void *$1$tdc; void *(*(p()))$tdc { return &$1; } int main() { if(p()) return(0); else return(1); }"> try.c;
+ echo "$extern_C void *$1$tdc; void *(*(p()))$tdc { return &$1; } int main() { if(p() && p() != (void *)main) return(0); else return(1); }"> try.c;
$cc -o try $optimize $ccflags $ldflags try.c $libs >/dev/null 2>&1 && tval=true;
$rm_try;
fi;


And this is what happened (I think):

Configure thinks I have these symbols because I'm building with -O2 -flto.
"lto" stands for "link-time optimization" and lets gcc apply optimizations in
the linking phase.
The configure test effectively checks whether &foo is NULL. gcc knows that the
address of a symbol can't be NULL, so the test is optimized out.
But that was the only reference to 'p' in the program, so dead code elimination
removes it (and with it the only reference to 'foo').
The program is now effectively 'int main() { return 0; }', so linking succeeds.
End result: Configure thinks the symbols strlcpy, malloc_size, etc exist.

My patch makes it actually check the address so gcc can't get rid of it
completely, thus Configure gets the linker errors it expects.

Andy Dougherty reports that a workaround is to call

./Configure -Doptimize=-O2 [...]

but then call make as

make OPTIMIZE='-O2 -flto'


[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
category=install
severity=medium
---
Site configuration information for perl 5.16.0:

Configured by mauke at Mon May 21 11:48:30 CEST 2012.

Summary of my perl5 (revision 5 version 16 subversion 0) configuration:

Platform:
osname=linux, osvers=2.6.38-gentoo-r6, archname=i686-linux
uname='linux nora 2.6.38-gentoo-r6 #1 preempt sat aug 6 03:05:34 cest 2011 i686 amd athlon(tm) 64 processor 3200+ authenticamd gnulinux '
config_args='-de -Dprefix=/home/mauke/usr/perlbrew/perls/perl-5.16.0'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=undef, use64bitall=undef, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2',
cppflags='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.6.3', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib
libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
libc=/lib/libc-2.14.1.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.14.1'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

Locally applied patches:


---
@INC for perl 5.16.0:
/home/mauke/usr/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/i686-linux
/home/mauke/usr/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0
/home/mauke/usr/perlbrew/perls/perl-5.16.0/lib/5.16.0/i686-linux
/home/mauke/usr/perlbrew/perls/perl-5.16.0/lib/5.16.0
.

---
Environment for perl 5.16.0:
HOME=/home/mauke
LANG=en_US.UTF-8
LANGUAGE (unset)
LC_COLLATE=POSIX
LD_LIBRARY_PATH=/home/mauke/usr/local/lib
LOGDIR (unset)
PATH=/home/mauke/usr/perlbrew/bin:/home/mauke/usr/perlbrew/perls/perl-5.16.0/bin:/home/mauke/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/4.4.5:/opt/sun-jdk-1.4.2.13/bin:/opt/sun-jdk-1.4.2.13/jre/bin:/opt/sun-jdk-1.4.2.13/jre/javaws:/opt/dmd/bin:/usr/games/bin
PERLBREW_HOME=/home/mauke/.perlbrew
PERLBREW_PATH=/home/mauke/usr/perlbrew/bin:/home/mauke/usr/perlbrew/perls/perl-5.16.0/bin
PERLBREW_PERL=perl-5.16.0
PERLBREW_ROOT=/home/mauke/usr/perlbrew
PERLBREW_VERSION=0.27
PERL_BADLANG (unset)
PERL_UNICODE=SAL
SHELL=/bin/bash


nick at ccl4

Jul 4, 2012, 3:05 AM

Post #2 of 3 (216 views)
Permalink
Re: [perl #113022] Configure misdetects strlcpy et al. with gcc -flto [In reply to]

On Tue, Jul 03, 2012 at 06:56:20PM -0700, Ricardo SIGNES via RT wrote:
> I would very much appreciate some opinions on the applicability of this change.

It looks sane, but it is an area that caused great fun before.

I think we push it to a smoke-me branch, but also try to ensure we test
Configure with and without generates the same config.sh (no need to actually
build) on as many compilers as we collectively have access to, at high
optimisation levels.

(ie I think that's the vendor C and C++ compilers on Solaris, HP-UX and AIX.
Test gcc and clang, but I don't think that they'll be the problem. icc, if
anyone has access to it.)

Nicholas Clark


nick at ccl4

Mar 6, 2014, 9:11 AM

Post #3 of 3 (82 views)
Permalink
Re: [perl #113022] Configure misdetects strlcpy et al. with gcc -flto [In reply to]

On Thu, Mar 06, 2014 at 08:38:21AM -0800, Shlomi Fish via RT wrote:

> Tested now on a FreeBSD 9.2 VM with gcc46 (gcc-4.6). This command:
>
> sh Configure -de -Dprefix=$HOME/apps/perl/bleadperl -Dusedevel -Doptimize='-O2 -flto' -Dcc=gcc46
>
> Succeeds there with and without the patch. make test succeeds without the patch, and with the patch I am getting this output with a failure in Benchmark (which I recall something that it could fail some time):

Cool.

Do we have any non-gcc non-clang tests yet?

Does anyone have a gcc 4.9 development build of gcc to test this?

I seem to remember that Sun's compiler was a particular problem with this
probe, and caused previous code to get reverted. The problem being that
getting this wrong *breaks* the entire platform, whereas not changing it
merely breaks an optimisation, that IIRC is not even the default.


So demonstrating that Sun's compiler, IBM's compiler and HP's were happy
(at least C, hopefully C++ also available on the same machine) feels like
the minimal comfortable check.

Is this patch in a smoke-me branch yet?

That gets some automated testing, and makes it a lot* easier to manually
build on annoying platforms.

Nicholas Clark

* "Yes, we're right with it. We can even handle context diffs"

Perl porters RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.