This Week on perl5-porters - 27 February - 5 March 2006

This Week on perl5-porters - 27 February - 5 March 2006

Module::Build achieves core status, VMS improvements, and the usual bunch of bug fixes and enhancements.

Topics of Interest

Function prototyping and op/utftaint.t failures in blead in Win32

Yves Orton first starting looking at smoke failures on Win32, and wondered why he was seeing redefined subroutines in Getopt::Long, but Nicholas Clark stepped up and said that Yves had been caught out by a glitch that had since been fixed in the source code.

Steve Hay and Nicholas worked through the failure, which was a problem of taintedness when . (dot) appears in the PATH environment variable. A fine example of how hard it is to get cross-platform platform-dependent code correct.

Term::ReadLine, CON: and Term::ReadKey on Win32 (cpan #17773)

Johnathon Stowe mentioned the issues he had uncovered with CPAN bug reports filed against Term::ReadKey. He had traced the problem to Term::ReadLine using the special device CON: (as in console) on Win32. One may open such a file directly, and things will work correctly, but open the file and pass it as a handle to Term::ReadLine and things start to fall apart, which is in fact what happens at the moment. Johnathon thought that the best approach would be to fix Term::ReadLine.

  Somebody else's bug

DBD::Oracle 1.17 and 64-bit perl on AIX 5.1

John L. Allen was failing miserably at getting Oracle and 64-bit perl on AIX to talk to each other, and asked for help. H.Merijn Brand thought that the combination was "like voluntarily having someone tie you down on a bed of nails, and then being whipped with a bunch of rusty barbed wire." Dave Mitchell considered it highly doubtful that one would be able to connect a 32-bit Oracle library to a 64-bit perl.

After trying out different long integer and long double Perl compilations, John succeeded in building a perl with 32-bit integers, long doubles (128 bits) and Oracle all playing nicely.

Alan Olsen, speaking from experience, recommended that John go either full 32-bit, or full 64-bit. Choosing the middle ground just leads to pain and unhappiness.

  It can be done

pow failures on AIX with uselongdouble

After having done battle with Oracle on AIX, John L. Allen turned his attention to failures in the test suite, and noticed some misbehaviour with the pow function. Dominic Dunlop confirmed that something strange was indeed occurring.

After a bit of detective work, John realised that the problem was not with perl, but AIX's underlying math libraries. He wrote a small C program to demonstrate the error, which should allow someone with the necessary Configure-fu to write the probe to work around the breakage.

  The library and the damage done

Detecting @ISA cache invalidation

Caching in which package a method is found ensures decent run-time performance. That is, if an object calls a method unknown to the current package, it consults @ISA to look for other packages that may have a suitable method. Once it succeeds in finding it the first time, it caches the information so that subsequent method calls can avoid paying the cost of the search. That works fine until @ISA is modified, in which case the cache has to be invalidated, which in turn means that objects need to take a fresh look through the @ISA chain to locate their methods again.

Usually you don't have to care about this at all, it just works. But some modules extremely tricky things, and can get thrown by changes to @ISA, and worse, up until now there was no easy way of determining when it happens.

So Joshua ben Jore wrote B::sub_generation, which provides a handy technique for letting user code know when cache invalidation occurs. When asked to explain what this all meant, Joshua provided a short snippet of code, showing how things can get derailed if code doesn't realise that @ISA changed. Rafael thought that Joshua's exposé should be saved somewhere in perlguts.

  Talkin' about my B::sub_generation

Small bug fixed in Module::Build passthrough Makefile.PL

Ken Williams made a change to Module::Build::Compat to deal with the fact that the method call CPAN::Shell->install() doesn't appear to return anything useful.

Yitzchak Scott-Thoennes wanted to know how one could reliably test to see whether the install did in fact succeed. Andreas Koenig recommended using the $module->uptodate method as a suitable work-around.

Ken planned to use that approach soon.

  Look for a new version of Module::Build soon

Tracking down leaks in XS SVs

Tracking down leaking scalars in XS code is hard.

Andy Armstrong has started to work on Devel::LeakTrace, which uses a custom run loop, and was having trouble determining the relationship between where SVs are created, and where perl reports that they were created.

Dave Mitchell pointed out that since 5.8.1 it has been possible to compile a perl executable with -DDEBUG_LEAKING_SCALARS, which adds extra information to SVs, which in turn can be picked up by something like Devel::Peek. On the other hand, Andy's approach has the benefit of not requiring a specially prepared perl.

Understanding ext and testing

Adam Kennedy is working on the Vanilla Perl 5.8.8 distribution, and was having trouble with some of the tests in the IO:: packages. He didn't understand how the tests could succeed at all, and wondered how long they had been failing.

Steve Hay had a look, and saw that the behaviour is dependent on whether the fork emulation code is used or not. And that some of the tests to see whether fork is defined or not are quite bizarre.

Nick Ing-Simmons came to the rescue, giving an explanation for why some of the things are the way they are. All in all, it's an area that Needs Work.

Making File::Find Iterative

The venerable File::Find module uses a callback scheme, something beginners to Perl may find a bit unusual. Shlomi Fish looked at extending it to provide an iterative interface similar to Shlomi's own File::FTS module.

Randal Schwartz pointed to an article he had written that dealt with the same issue. Steve Peters noted that the main criterion for judging patches to File::Find is that they should incur no speed penalty, thus, for a patch to be accepted, it would need solid benchmarking.

Nick Ing-Simmons explained that the question of speed was one of system calls. Make one unnecessary call, and the difference would show up for people scanning deep and wide directory trees.

In the end, Shlomi decided to concentrate on File::Find::Object, as it appeared to be closer to suiting his needs. David Nicol wondered whether File::Find was thread-safe, and Tels made a plug for his own File::Walker



Will someone please fix Devel::DProf?

Jarkko Hietaniemi made a plea for someone with too much spare time on their hands to please take a look at Devel::DProf and fix some its worst bugs. Andy Lester wanted some test cases.

Jarkko countered with:

  $ perl -d:DProf -e 'sub foo { next } for (1) { foo }'
  $ perl -d:DProf -e 'use autouse qw(Pod::Usage pod2usage); pod2usage'

Tels searched the RT queue:

  The bug stops here

Module::Build 0.27_08 added to the standard distribution

A first attempt was made to add Module::Build to the core. This was perhaps the largest thread of the week. It appears the graft has succeeded, but more work is required.

The continuing saga of *a=$a=*b;${"a"}=*a;

Nicholas returned to the *a=$a=*b;${"a"}=*a;-perl-go-boom bug and couldn't see a easy way out, short of cheating and getting the optimiser to spot the sequence and thereby emit a different op-tree to make the problem go away.

David Nicol wondered whether it was a case of worrying too much about a construct that doesn't come up in practice. chromatic thought that a tool simply should not crash on invalid input.

Some more VMS goodness

Peter Prymmer sent in a patch for some VMS-specific issues, which Craig Berry applied partially. Peter was lacking a bit of context to understand the reason why Craig did not apply the rest of it, and John E. Malmberg summarised the issue nicely for the vmsperl readership who don't follow p5p, and thus also saved the p5p summariser from having to do so:

  Peter's patch

  John's summary

Matching dynamic Unicode properties

Dan "Mr. Encode" Kogai related something he'd heard about non-existent Unicode properties (like \p{IsBogus}) in pattern matches. Sometimes perl spits out an error... but not always. Yitzchak was quick to that the results depended on whether the match had failed or not before the scanner reached the location of \p property in the target string.

Hugo van der Sanden wasn't even sure one could call it a bug (which Dan did not say either), but thought that perhaps the utf8 module could offer something to check for this situation, since, as Sadahiro-san pointed out, it could be hard to check for user-defined properties sufficiently early in the general case.

What Andy Lester did this week

  Removed a redundant o->op_type

  Non-null optimizations for SvREFCNT_inc

  Initialising all of mgvtbl

Patches of Interest

PERL_TRACK_MEMPOOL cripples environment after exit()

Marcus Holland-Moritz uses blead for his daily work, which leads to some interesting discoveries. He had wanted to run gcov coverage analysis of one of his XS modules, and became distracted by strange failures, probably best attributed to the environment variable code clean out that took place a few months ago. Evidently there are still demons lurking in that code. Marcus supplied a patch to nail one of them, applied by Rafael Garcia-Suarez.

Silence useless errors in B::Lint for grep

Joshua ben Jore offered a patch to silent what he perceived to be a useless warning emitted by B::Lint, concerning implicit matches with grep. Since grep{ /pattern/ } is such a frequent idiom (and insofar as $_ is topicalised (the New way of referring to what used to be known as localised) within the grep code block), it's a rather stupid warning. Rafael agreed with Joshua and applied the patch, saying that he'd be happy to accept similar patches for postfix for and map.

Long pathnames on VMS and sundry fixes

John E. Malmberg sent in a patch for VMS builds and long pathname handling. Abe Timmerman reported that results looked good.

  The patch

  The smoke results

Watching the smoke signals

Smoke [5.9.4] 27358 FAIL(F) hp-ux 11.11/64 (PA-2.0/64/2 cpu)

H.Merijn Brand poked around in to try and cope with change #27345 and problems with tainting. He made the appropriate changes to get things to work on HP-UX but wondered whether the patch would work on other systems like VMS or Win32. Steve Hay found problems with the latter, and proposed an alternative.

New and old bugs from RT

Lexical variables in regexp ?{} assertions (#38639)

Richard Clayton identified a problem with lexicals defined within a for and being set within a ?{} assertion of a pattern match. It gets set the first time through the loop, but not on subsequent iterations. The work-around is to declare the lexical in a scope further out from the for block. No takers.

Configure -des stops waiting for <CR> (#38642)

Anton Koinov ran into problems configuring perl on Gentoo. Andy Dougherty was intrigued, and tried to guess what was going wrong, and suggested Anton run the following command

  sh -x ./Configure -S 2>&1 | tee Configure.out

... in order to capture all of the text generated during the run (as -des discards certain output deemed to be mostly harmless). No news back from Anton as of yet.

MULTICALL causes segfaults with very large return stacks (#38644)

Tassilo von Parseval has been working with the new MULTICALL interface and noticed problems when the return stack is very large. This shows up as segfaults in XS code, such as List::Util.

Long file names on Stratus VOS (#38645 and #38646)

Paul Green uses Perl on Stratus VOS, whose file system limits file names to no more than 32 characters. This poses a problem when he tries to run the test suite of a module like Pod::Simple, which contains a test name search_25_glob_squaa_coloncolon_kleene.t. At 40 characters long, that's more than Stratus can bear.

Similarly, there's another file, _white_with_navy_blue_on_black.css that is also a bit too long. Nicholas Clark wondered whether 32 was some sort of POSIX minimum, and recalled that some tests had already been renamed to deal with characters in file names that VMS had trouble processing. Joshua Juran noted that traditional Mac-based platforms (MacOS and Lamp) are limited to 31 characters. (MacOS X does not have this limitation).

(The Summariser wishes to point out the existence of Sébastien Aperghis-Tramoni's Test::Portablility::Files to module authors).

  One file

  And then another

  No more excuses

64-bit (non-)builds on Solaris 8 (#38664)

James Overly had problems building a 64-bit perl on Solaris 8.

Debugger Loses Names for Anonymous Subroutines (#38673)

chromatic showed how the trick of assigning to the __ANON__ typeglob to name an anonymous subroutine doesn't work in the debugger.

O::Deparse not working as expected (#38684)

"harleypig" was having trouble with split. I tried to pay attention, but Stephen McCamant supplied a patch to fix the behaviour (which was apparently broken), which was applied by Rafael. The end.

Perl5 Bug Summary

  1548 open items

  in all their glory

New Core Modules

In Brief

Seung-Ho Han wanted to know how make sense of what B::Xref::compile produces. Joshua ben Jore explained what was going on, and that more clues can be found by studying

Jerry Hedden started to extract from the core in order to make it available on CPAN. This would permit enhancements to be made to the threads implementation faster than the current blead to maint to release cycle.

Jim Cromie continued to refine his self-described hare-brained op-next/op-siblingoptimisation scheme, defending it against the critics, but ran into trouble somewhere deep down in the guts. In doing so, he came up with a new presentation scheme for B::Concise.

Anton Berezin caught and eliminated an extraneous #endif in fakesdio.h.

Jarkko Hietaniemi discovered that -d:Foo=bar no longer works in the current Perl release (bug report #38657). Rafael traced the problem down to the fact that \0 (NUL) was no longer allowed in environment variables, and fixed things up as change #27359. Jarkko also supplied another patch for the same thing, which Rafael also applied.

Getting back to last week's thread on unwanted auto-vivification with foreach, David Nicol showed a snippet of code that offers a way of working around the problem, and wondered whether the documentation should record it somewhere. Graham Barr noted that using reverse also suppresses the auto-vivification behaviour. And that if you didn't want a slice reversed, you needed to reverse reverse.

  There's at least one way to do it

Jarkko added a test to Data::Dumper's test suite to ensure that the bug #38612 that was fixed in 5.8.7 never returns.

H.Merijn added Configure support for the GCC compiler options __builtin_expect and __builtin_choose_expr.

Paul Marquess delivered a patch for Compress::Zlib, following on from his recent work to add compression plugins.

Joshua start to have a look at the new smart match operator ~~ and found a discrepancy between its behaviour and that of Pugs/Perl6.

Abe Timmerman patched Porting/ to pick up and plunged unwittingly into the wonderful and frightening world of VMS symbols. John E. Malmberg and Craig A. Berry were sent in to rescue him.

Abe then thought it would be a good idea to see what happens when one builds a threaded perl on VMS. In the process he uncovered a couple of problems that John and Craig sorted out.

Dave Rolsky wanted to know if nested closures still leak memory. Dave Mitchell thought that no closure leakage bugs remained in blead, but pointed out that not all of the new code had made it back into maint.

  Until proven otherwise

Randy W. Sims want to install an older perl on his Ubuntu linux distribution, but the install began to create an infinitely deep directory tree.

  On and on and on

Linda W gave a report on the effects of configuration options on the resulting perl binary.

Perl can no longer be compiled with a K&R compiler.

David Nicol posted his thought on what he would like to see in the way of plug-in optimisers. A optimistic optimiser could optimise a section of code, and then, if some assumption failed to hold true, it could de-optimise the code back to the initial state. I think some proof-of-concept code will be required.

About this summary

This summary was written by David Landgren.

Information concerning bugs referenced in this summary (as #nnnnn) may be viewed at

Information concerning patches to maint or blead referenced in this summary (as #nnnnn) may be viewed at

If you want a bookmarklet approach to viewing bugs and change reports, there are a couple of bookmarklets that you might find useful on my page of Perl stuff:

Weekly summaries are published on and posted on a mailing list, (subscription: The archive is at Corrections and comments are welcome.

If you found this summary useful or enjoyable, please consider contributing to the Perl Foundation to help support the development of Perl.