Mailing List Archive: Perl: porters

[perl #113406] perldoc in 5.16.0 required groff upgrade but now misdisplays asterisks

Index | Next | Previous | View Threaded

perlbug-followup at perl

May 27, 2012, 8:42 PM

Post #1 of 10 (133 views)
Permalink

[perl #113406] perldoc in 5.16.0 required groff upgrade but now misdisplays asterisks

# New Ticket Created by James E Keenan
# Please include the string: [perl #113406]
# in the subject line of all future correspondence about this issue.
# <URL: https://rt.perl.org:443/rt3/Ticket/Display.html?id=113406 >

This is a bug report for perl from jkeenan [at] cpan,
generated with the help of perlbug 1.39 running under perl 5.16.0.

-----------------------------------------------------------------
[Please describe your issue here]

I upgraded from Perl 5.12 or Perl 5.14 to Perl 5.16.0 on three machines
in the
last week: two Linux and one Darwin. As I began to use 'perldoc' on these
machines, I noticed that warnings were being generated before the POD was
formatted. These warnings reported that my version of 'groff' was
out-of-date. Accordingly, I got the 'grotty' package from gnu.org and
compiled, built and installed the code from source.

##########
$ groff --version
GNU groff version 1.21
Copyright (C) 2009 Free Software Foundation, Inc.
GNU groff comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of groff and its subprograms
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.

called subprograms:

GNU grops (groff) version 1.21
GNU troff (groff) version 1.21
#########

Once I completed this upgrade, 'perldoc' worked without generating warnings.
However, I now observe a regression in perldoc's performance. I (and
others)
have often written POD like this:

##########
=head2 Comparing Two Lists Held in Arrays

=over 4

=item *

Given two lists:

@Llist = qw(abel abel baker camera delta edward fargo golfer);
@Rlist = qw(baker camera delta delta edward fargo golfer hilton);

=item *

##########

That is, POD where the '=item' directive is followed by a wordspace and an
asterisk. Prior to 5.16.0 and the groff upgrade, perldoc always
rendered this
simply as an asterisk.

Now, however, it's representing this as follows:

##########
Comparing Two Lists Held in Arrays
<C2><B7> Given two lists:

@Llist = qw(abel abel baker camera delta edward fargo golfer);
@Rlist = qw(baker camera delta delta edward fargo golfer
hilton);

<C2><B7> Get those items which appear at least once in both lists
(their intersection).
##########

That is, the '*' is being represented by 'perldoc' as '<C2><B7>'. To
me, this
is a regression. What is wrong and how can it be fixed?
##########

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
category=docs
severity=medium
---
Site configuration information for perl 5.16.0:

Configured by jimk at Sun May 20 20:01:26 EDT 2012.

Summary of my perl5 (revision 5 version 16 subversion 0) configuration:

Platform:
osname=darwin, osvers=8.11.0, archname=darwin-2level
uname='darwin macintosh-8.local 8.11.0 darwin kernel version
8.11.0: wed oct 10 18:26:00 pdt 2007; root:xnu-792.24.17~1release_ppc
power macintosh powerpc '
config_args='-des'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=undef, use64bitall=undef, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing
-pipe -I/usr/local/include -I/opt/local/include',
optimize='-O3',
cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe
-I/usr/local/include -I/opt/local/include'
ccversion='', gccversion='4.0.1 (Apple Computer, Inc. build 5250)',
gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags ='
-L/usr/local/lib -L/opt/local/lib'
libpth=/usr/local/lib /opt/local/lib /usr/lib
libs=-ldbm -ldl -lm -lc
perllibs=-ldl -lm -lc
libc=, so=dylib, useshrplib=false, libperl=libperl.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup
-L/usr/local/lib -L/opt/local/lib'

Locally applied patches:

---
@INC for perl 5.16.0:
/usr/local/lib/perl5/site_perl/5.16.0/darwin-2level
/usr/local/lib/perl5/site_perl/5.16.0
/usr/local/lib/perl5/5.16.0/darwin-2level
/usr/local/lib/perl5/5.16.0
/usr/local/lib/perl5/site_perl/5.14.2
/usr/local/lib/perl5/site_perl/5.14.0
/usr/local/lib/perl5/site_perl/5.12.0
/usr/local/lib/perl5/site_perl/5.10.1
/usr/local/lib/perl5/site_perl/5.10.0
/usr/local/lib/perl5/site_perl
.

---
Environment for perl 5.16.0:

DYLD_LIBRARY_PATH=/Users/jimk/work/pseudoinstall/lib:/Users/jimk/gitwork/parrot/blib/lib
HOME=/Users/jimk
LANG (unset)
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)

PATH=/usr/local/bin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/opt/local/bin:/opt/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/Users/jimk/bin:/Users/jimk/bin/perl:/Users/jimk/bin/c:/Users/jimk/bin/shell:/sw/lib:/sw/bin:/Users/jimk/bin:/Users/jimk/bin/perl:/Users/jimk/bin/c:/Users/jimk/bin/shell:/sw/lib:/sw/bin
PERL_BADLANG (unset)
SHELL=/bin/bash

tchrist at perl

May 27, 2012, 9:14 PM

Post #2 of 10 (133 views)
Permalink

Re: [perl #113406] perldoc in 5.16.0 required groff upgrade but now misdisplays asterisks [In reply to]

James E Keenan (via RT) <perlbug-followup [at] perl> wrote
on Sun, 27 May 2012 20:42:21 PDT:

> Now, however, it's representing this as follows:

> Comparing Two Lists Held in Arrays
> <C2><B7> Given two lists:
>
> @Llist = qw(abel abel baker camera delta edward fargo golfer);
> @Rlist = qw(baker camera delta delta edward fargo golfer hilton);
>
> <C2><B7> Get those items which appear at least once in both lists
> (their intersection).
>
> That is, the '*' is being represented by 'perldoc' as '<C2><B7>'. To
> me, this is a regression. What is wrong and how can it be fixed?

"\xC2\xB7" is simply the UTF-8 representation of U+00B7 MIDDLE DOT, which
means it is just fine.

I suggest you have something else wrong.

But I don't know. I wouldn't use perldoc if you paid me. It works
just fine using the regular toolchain, meaning either pod2text
or pod2man|nroff -man.

--tom

perlbug-followup at perl

May 27, 2012, 10:56 PM

Post #3 of 10 (127 views)
Permalink

[perl #113406] perldoc in 5.16.0 required groff upgrade but now misdisplays asterisks [In reply to]

On Sun May 27 20:42:21 2012, jkeen [at] verizon wrote:
> I upgraded from Perl 5.12 or Perl 5.14 to Perl 5.16.0 on three
> machines
> in the
> last week: two Linux and one Darwin. As I began to use 'perldoc' on
> these
> machines, I noticed that warnings were being generated before the POD
> was
> formatted. These warnings reported that my version of 'groff' was
> out-of-date. Accordingly, I got the 'grotty' package from gnu.org and
> compiled, built and installed the code from source.
>
> ##########
> $ groff --version
> GNU groff version 1.21
> Copyright (C) 2009 Free Software Foundation, Inc.
> GNU groff comes with ABSOLUTELY NO WARRANTY.
> You may redistribute copies of groff and its subprograms
> under the terms of the GNU General Public License.
> For more information about these matters, see the file named COPYING.
>
> called subprograms:
>
> GNU grops (groff) version 1.21
> GNU troff (groff) version 1.21
> #########
>
> Once I completed this upgrade, 'perldoc' worked without generating
> warnings.
> However, I now observe a regression in perldoc's performance. I (and
> others)
> have often written POD like this:
>
> ##########
> =head2 Comparing Two Lists Held in Arrays
>
> =over 4
>
> =item *
>
> Given two lists:
>
> @Llist = qw(abel abel baker camera delta edward fargo golfer);
> @Rlist = qw(baker camera delta delta edward fargo golfer hilton);
>
> =item *
>
> ##########
>
> That is, POD where the '=item' directive is followed by a wordspace
> and an
> asterisk. Prior to 5.16.0 and the groff upgrade, perldoc always
> rendered this
> simply as an asterisk.
>
> Now, however, it's representing this as follows:
>
> ##########
> Comparing Two Lists Held in Arrays
> <C2><B7> Given two lists:
>
> @Llist = qw(abel abel baker camera delta edward fargo
> golfer);
> @Rlist = qw(baker camera delta delta edward fargo golfer
> hilton);
>
> <C2><B7> Get those items which appear at least once in both
> lists
> (their intersection).
> ##########
>
> That is, the '*' is being represented by 'perldoc' as '<C2><B7>'. To
> me, this
> is a regression. What is wrong and how can it be fixed?
> ##########

It sounds to me as though perldoc is assuming you have a UTF-8
environment when you don’t. So UTF-8 gets fed to less, and less sees it
as just bytes, because it hasn’t been told via the LANG env var to treat
its input otherwise. That sounds like a perldoc bug.

--

Father Chrysostomos

---
via perlbug: queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=113406

perlbug-followup at perl

May 28, 2012, 6:02 AM

Post #4 of 10 (124 views)
Permalink

[perl #113406] perldoc in 5.16.0 required groff upgrade but now misdisplays asterisks [In reply to]

On Sun May 27 21:15:14 2012, tom christiansen wrote:
>
> "\xC2\xB7" is simply the UTF-8 representation of U+00B7 MIDDLE DOT,
> which
> means it is just fine.
>
> I suggest you have something else wrong.
>
> But I don't know. I wouldn't use perldoc if you paid me. It works
> just fine using the regular toolchain, meaning either pod2text
> or pod2man|nroff -man.
>

Neither of those solutions are really satisfactory.

pod2text displays the asterisks as asterisks, but, of course, at the
expense of (a) losing all the man-like formatting; and (b) requiring
that I pipe the content to 'less':

pod2text List/Compare/Functional.pm |less

######### (excerpt)
Comparing Two Lists Held in Arrays
* Given two lists:

@Llist = qw(abel abel baker camera delta edward fargo golfer);
@Rlist = qw(baker camera delta delta edward fargo golfer hilton);

* Get those items which appear at least once in both lists (their
intersection).

@intersection = get_intersection( [ \@Llist, \@Rlist ] );
##########

pod2man piped to nroff -man displays the asterisks as open circles,
which is acceptable. But now it gets =head2 wrong (on top of being many
more keystrokes):

pod2man List/Compare/Functional.pm |nroff -man |less

########## (excerpt)
ESC[.1mComparing Two Lists Held in ArraysESC[.0m
o Given two lists:

@Llist = qw(abel abel baker camera delta edward fargo golfer);
@Rlist = qw(baker camera delta delta edward fargo golfer hilton);

o Get those items which appear at least once in both lists (their
intersection).

@intersection = get_intersection( [ \@Llist, \@Rlist ] );
##########

Thank you very much.
Jim Keenan

---
via perlbug: queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=113406

perlbug-followup at perl

May 28, 2012, 6:04 AM

Post #5 of 10 (125 views)
Permalink

[perl #113406] perldoc in 5.16.0 required groff upgrade but now misdisplays asterisks [In reply to]

On Sun May 27 22:56:22 2012, sprout wrote:
>
> It sounds to me as though perldoc is assuming you have a UTF-8
> environment when you don’t. So UTF-8 gets fed to less, and less sees it
> as just bytes, because it hasn’t been told via the LANG env var to treat
> its input otherwise. That sounds like a perldoc bug.
>

I have not made any recent changes to my 'locale' settings on these
machines. For instance, on Darwin I have:

$ locale
LANG=
LC_COLLATE="C"
LC_CTYPE="C"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL="C"

... and on Linux I have:

$ locale
LANG=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

Thank you very much.
Jim Keenan

---
via perlbug: queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=113406

fawaka at gmail

May 28, 2012, 6:24 AM

Post #6 of 10 (126 views)
Permalink

Re: [perl #113406] perldoc in 5.16.0 required groff upgrade but now misdisplays asterisks [In reply to]

On Mon, May 28, 2012 at 3:04 PM, James E Keenan via RT
<perlbug-followup [at] perl> wrote:
> I have not made any recent changes to my 'locale' settings on these
> machines. For instance, on Darwin I have:
>
> $ locale
> LANG=
> LC_COLLATE="C"
> LC_CTYPE="C"
> LC_MESSAGES="C"
> LC_MONETARY="C"
> LC_NUMERIC="C"
> LC_TIME="C"
> LC_ALL="C"
>
> ... and on Linux I have:
>
> $ locale
> LANG=
> LC_CTYPE="POSIX"
> LC_NUMERIC="POSIX"
> LC_TIME="POSIX"
> LC_COLLATE="POSIX"
> LC_MONETARY="POSIX"
> LC_MESSAGES="POSIX"
> LC_PAPER="POSIX"
> LC_NAME="POSIX"
> LC_ADDRESS="POSIX"
> LC_TELEPHONE="POSIX"
> LC_MEASUREMENT="POSIX"
> LC_IDENTIFICATION="POSIX"
> LC_ALL=

Both C and POSIX are ASCII locales, AFAIK, so it seems Father
Chrysostomos' diagnosis is right.

Leon

perlbug-followup at perl

May 28, 2012, 6:39 AM

Post #7 of 10 (125 views)
Permalink

[perl #113406] perldoc in 5.16.0 required groff upgrade but now misdisplays asterisks [In reply to]

On Mon May 28 06:25:41 2012, LeonT wrote:
>
> Both C and POSIX are ASCII locales, AFAIK, so it seems Father
> Chrysostomos' diagnosis is right.
>

Leon, thanks for your quick response.

It's safe to assume that there will be many other people who, when they
try out 'perldoc' in 5.16, obey the recommendation to upgrade groff and
who subsequently get unexpected, less attractive 'perldoc' output than
they did with 5.14 and earlier.

Since I both upgraded 'perldoc' (via upgrading Perl) *and* upgraded
groff, it seems that the first question to ask is, "What output would I
have gotten if I had ignored the warnings and *not* upgraded groff?"
If, indeed, an assumption of UTF-8 input is being made, where is it
being made: in perldoc, groff or someplace else?

Thank you very much.
Jim Keenan

---
via perlbug: queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=113406

tchrist at perl

May 28, 2012, 6:54 AM

Post #8 of 10 (125 views)
Permalink

Re: [perl #113406] perldoc in 5.16.0 required groff upgrade but now misdisplays asterisks [In reply to]

> pod2man List/Compare/Functional.pm |nroff -man |less

Which is what the man program effectively does. Which is what I use.

--tom

perlbug-followup at perl

May 28, 2012, 7:05 AM

Post #9 of 10 (127 views)
Permalink

[perl #113406] perldoc in 5.16.0 required groff upgrade but now misdisplays asterisks [In reply to]

On Mon May 28 06:02:08 2012, jkeenan wrote:
>
> pod2man piped to nroff -man displays the asterisks as open circles,
> which is acceptable. But now it gets =head2 wrong (on top of being many
> more keystrokes):
>
> pod2man List/Compare/Functional.pm |nroff -man |less
>
> ########## (excerpt)
> ESC[.1mComparing Two Lists Held in ArraysESC[.0m
> o Given two lists:
>
> @Llist = qw(abel abel baker camera delta edward fargo golfer);
> @Rlist = qw(baker camera delta delta edward fargo golfer
hilton);
>
> o Get those items which appear at least once in both lists (their
> intersection).
>
> @intersection = get_intersection( [ \@Llist, \@Rlist ] );
> ##########
>

This is also the output I get if I use the older, system 'perldoc',
which is presumably interacting with the upgrade 'groff':

##########
$ /usr/bin/perldoc -V
Perldoc v3.13, under perl v5.008006 for darwin

$ perldoc -V
Perldoc v3.17, under perl v5.016000 for darwin

$ perldoc -l List::Compare::Functional
/usr/local/lib/perl5/site_perl/5.10.0/List/Compare/Functional.pm

$ /usr/bin/perldoc `perldoc -l List::Compare::Functional`
##########
ESC[.1mComparing Two Lists Held in ArraysESC[.0m

o Given two lists:

@Llist = qw(abel abel baker camera delta edward fargo golfer);
@Rlist = qw(baker camera delta delta edward fargo golfer hilton);

o Get those items which appear at least once in both lists (their
intersection).

@intersection = get_intersection( [ \@Llist, \@Rlist ] )
##########

---
via perlbug: queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=113406

mrallen1 at yahoo

May 28, 2012, 6:47 PM

Post #10 of 10 (127 views)
Permalink

Re: [perl #113406] perldoc in 5.16.0 required groff upgrade but now misdisplays asterisks [In reply to]

> From: Father Chrysostomos via RT <perlbug-followup [at] perl>

>>On Sun May 27 20:42:21 2012, jkeen [at] verizon wrote:
>> That is, the '*' is being represented by 'perldoc' as '<C2><B7>'. To
>> me, this
>> is a regression. What is wrong and how can it be fixed?
> It sounds to me as though perldoc is assuming you have a UTF-8
> environment when you don’t. So UTF-8 gets fed to less, and less sees it
> as just bytes, because it hasn’t been told via the LANG env var to treat
> its input otherwise. That sounds like a perldoc bug.

Yes perldoc makes an implicit assumption (since about November or so) that it is

going to process UTF8 data in and send UTF8 data out.

Like most assumptions it gets things wrong sometimes. There's already a ticket open in the
Pod-Perldoc RT queue for this issue. I am planning to release a new version of Pod-Perldoc
to address it.

Hopefully it can go into 5.16.1.

Thanks.

--Mark

---
via perlbug: queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=113406

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads