This Week on perl5-porters - 27 November-3 December 2006

This Week on perl5-porters - 27 November-3 December 2006

"So I don't really have an answer for you. When faced with a similar situation, I punted and did a PerlIO_fdopen(fileno(stdio)) instead of a PerlIO_importFILE(stdio). This way you know you are creating a unixio layer which gets ref counted properly." -- Craig A. Berry, somewhere in the sprawling PerlIO layers saga.

Topics of Interest

PerlIO::Scalar buffering

H.Merijn Brand wondered if he had uncovered a problem with performing IO to a scalar, since at the end of a series of operations where the scalar should have been emptied, a lone newline character remained.

Whither 5.8.9

Jerry D. Hedden wondered if there was going to be a 5.8.9 release, and if so would arrive before 5.10.0. H.Merijn Brand confirmed that a 5.8.9 release is going to happen some time, and it's likely to escape from the lab before 5.10. No dates have been set for either event as yet.

  when it's ready

Old dmakes for building perl on Win32...

Steve Hay noticed that recent changes to the Makefiles for Windows are no longer understood by the older make programs floating around in the Windows world. He felt that the best approach was to update the README to explain the fix, in the unlikely event that someone encounters it, and to move on.

  time to go

... and archaic Visual C++ compilers

Steve also noted that the compiler writers at Microsoft have played the compiler command line switch fandango in recent versions, and now warnings are emitted when certain deprecated switches are used. To get things into shape means drawing the line at VC++ 5.0. This equates to roughly a 10 year old compiler. The Makefile states that things should be compatible all the way back to 2.0, but Steve has his doubts.

Nicholas Clark pointed out that if people really need to use a pre-5.0 compiler, all they have to do is change the new-name switch back to old-name, and things should work. Steve updated README.win32 to note this point.

Trouble with PERLIO_INIT/PERLIO_TERM changes

Steve Hay and Jarkko Hietaniemi tried to make sense of the ongoing PerlIO problems, in that reference counts were going too far negative and bad things were happening as a result. And the code was a bit too twisty for people to understand exactly what was happening. My eyes sort of glazed over on this thread.

It eventually wound up being filed as a bug, see below.

Also, in another thread, Steve also wondered whether PerlIO_importFILE() needed to do a refcnt_inc. Craig A. Berry showed him how to test the hypothesis. One thing that came out of it is that Steve noticed that the code was leaking file descriptors.

Similarly, Yves Orton found a segfault on Windows due to one of the many changes that were attempted this week to come to grips with these issues. Rafaël was pretty sure about which patch was the problem, and when Yves reverted it, his build succeeded in starting the test suite.

PathTools now at

Ken Williams announced that he'd thrown up File::Spec and Cwd, on a Subversion repository, and asked if there were any volunteers who wanted to poke around at them. Philippe Bruhat bravely stepped forward.

Change #26513 breaks Devel::SmallProf

Nicholas Clark found that the addition of an assertion to ensure an AV is always non-null made Devel::SmallProf curl up and die, and didn't what to do about it.

In the new month, the thread continued: Gisle Aas observed that the problem went away of the build was configured with usethreads. Rafaël Garcia-Suarez realised that the heart of the problem lay in toke.c, and made the appropriate change there.

Minor patch for

Christophe Grosjean patched what he thought was a minor problem in Benchmark, but was really quite a significant bug when you think about it: cmpthese used to ignore child process times, which led to the fascinating outcome that a process that spawned an external command appeared to be faster than something done in native Perl.

  Highly illogical

Patches of Interest

Abstract mempool header testing

Jarkko did a bit of work on memory pools, (that approximates the conditions of Win32 on Unix), where memory allocation (especially in threads) is a much trickier issue. Nicholas Clark gave a clear explanation of how the mechanisms present in the source code fitted together, and Jan Dubois completed the picture.

Based especially upon Jan's information, Jarkko produced a patch to attempt to clear up some PerlIO problems. Jan, however, thought that the code in question traded one set of problems (the mempool allocations) for another (leaking stuff from the opcode tree instead), and explained how to fix that problem

As the month rolled over into December, Yves Orton took a stab at the matter, pointing to a couple of spots in the code that he felt relied on flawed assumptions. The patches went back and forth, and I lost track of which patch undid what, and which ones were reverted.

Jan Dubois suggested one strategy for getting things straightened out, and Jarkko coded it up as a patch. This seemed to fix things sufficiently well that Steve Hay was able to unrevert a previously reverted patch with everything still behaving itself.

Yves was most impressed, but also confused, and asked if there was some documentation that explained The Big Picture. Steve Hay did his best to illuminate the part of the canvas that he understood, hoping that Jan Dubois would chip in and complete as necessary. And so he did. Jan gave a thorough best exposé on the issue of mempools here:

Now all we need is for someone to draw a diagram of the myriad ways in which one may allocate memory in perl. The thread in the December, from the top:

Building 5.8.8 on OS/2

Ilya Zakharevich delivered a patch to get 5.8.8 compiling again on OS/2. There are still some tests that fail, and Ilya promised to address them in a future patch.

perl5db on miniperl

Similarly, Ilya noticed that no longer worked with miniperl, due to a reliance on IO::Handle, which was not yet fully operational at that stage of the build process. So he tweaked things to handle this situation gracefully.

Test::Class and blead

A test in Test::Class tested for a bogus regular expression, and was supposed to die. The funny part was that thanks to Yves Orton's tireless efforts, what used to be a meaningless string of characters is now a legal pattern, so the pattern matches and the test fails.

So Nicholas Clark made the pattern even more bogus, to force the pattern to fail again. Which judging by Yves's prodigious output, should see the test safe for a least three weeks.


MacOSX debugger fork support received some more patching goodness from Bo Lindbergh to allow the debugger to be fired up in a separate terminal window on Mac OS/X.

Make B::Lint use Module::Pluggable

Joshua ben Jore was one of the first people to take advantage of Module::Pluggable's new promotion to core, and rewrote B::Lint to make use of it. He was immensely pleased, since it meant that he was able to rip out the previous clumsy efforts.

  The street finds its own uses for things

No code specified for -.

Jarkko discovered that an 'e' was dropped in the message No code specified for -e if in fact there was no code.

  dash before e except after p

Perl regexp structures bifurcation

Yves Orton continued to deliver a stream of changes to introduce a higher level of abstraction to the regular expression engine. This is part of his ongoing work to permit pluggable engines.

\R means something else in POSIX

Yves discovered that not only does the Unicode consortium care about regular expressions, they even published a specification on the matter. Yves read the spec, and learnt that they recommend using \R as a best-practice sequence for specifying a generic line-break marker.

This unfortunately clashes with \R as a relative back-reference assertion in Perl's regular expression, but since we are still not out of beta, Yves was quite happy to cast around for a different unused character. After looking around at what was left of the alphabet, he settled in \g (as in group). He implemented a generalised syntax with curly braces, so that now \g1 or \g{1} is the same as the existing \1, and conversely, that \g-1 and \g{-1} as a replacement for the now-defunct \R1 syntax. (Yves, if you're reading this, doesn't that mean you could drop \k<foo> and subsume it into \g<foo>? You have 15 minutes to write the patch).

Yves also thought that implement \R shouldn't be too difficult either, noting in passing that Philip Hazel will be delivering it in the next release of PCRE.

  14 minutes and counting

New and old bugs from RT

Segfault due to a semicolon inside an array reference (#40995)

Shlomi Fish discovered that a semi-colon, such as in [ $i->func2(); ] gets the tokeniser in a tizz. Rafaël noted that this misbehaviour was fixed in blead.

Setting $0 invalidates environment shown by ps (#41008)

Alexander Bluhm noted that assigning to $0 causes unwanted side-effects on OpenBSD, and suggested a patch to fix things up. Rafaël took it and applied it to blead.

(?(COND)) in pattern matching not working properly (#41010)

Karl Williamson discovered a bug where an alternation ((a|b)) did not match the same thing as a character class ([ab]). What was worse was that the original program, when it didn't crash, the values of other assorted variables in the program mysteriously changed. Yves Orton figured out what the problem was, and developed a patch to fix things up.


"backslash-G" does not appear to work in a while loop (#41011)

Jim Woodworth filed what he thought was a bug, but Dr. Ruud suspected that he was a victim of crypto-context.

Changing $0 on darwin leads to excessive padding in ps (#38868)

The week continued in a $0 vein, with Darwin also causing trouble. Chris Dolan sorted things out with the help of the C preprocessor.

  You were expecting \0?

Regexp optimizer loses its hopes too soon (#39096)

Yves Orton explored the reasons why the optimiser was ignoring obvious shortcuts and pointed out why a naive solution wouldn't work, and the problems that a generalised solution would have to consider.

  It used to be simple

LD_LIBRARY_PATH changes not exported to Dynaloader,Perl 5.8.8 On Solaris 8 (#40655)

A mysterious person reported that this bug was resolved.

  Private and Confidential

SIGSEGV with pos() in regexp (#40989)

Yves Orton noted that this bug no longer existed in blead. This is no doubt a pleasing side-effect of all the work that the engine has received in recent times, although he was hard pressed to say which change would have been responsible. By extension, he didn't expect the fix to be backported to maint.

  One more reason

Yves noted that bug #39893 was probably another facet of the same problem.

File::Glob braces with character classes erratic behaviour (#41027)

Kolano discovered some inconsistencies with globs, curly braces and character classes.

Nested closures result in scoping related segmentation fault (#41028)

... and they are fixed in blead.

PerlIOUnix_refcnt_dec() croaks in bleadperl since #29065 (#41030)

Steve Hay reported that since change #29065, his Win32-SharedFileOpen croaked with a strange error from somewhere within PerlIO. He traced the problem to a couple of patches, thinking that while they were not incorrect, perhaps there was an underlying bug which had only now been exposed as a result.

MSWin32: can't disable crlf trans. on STDxxx using :raw or binmode (#41034)

Vaclav Ovsik reported a problem with end-of-line markers on Windows. He was not able to disable end-of-line translations on files opened by default (such as STDOUT and STDERR, but files opened afterwards were just fine.

Perl5 Bug Summary

We almost crept under the 1500 barrier this week.

  Once more into the breech

New Core Modules

In Brief

Nicholas Clark taught the Aho-Corasick regexp patch how to behave in the presence of threads.

H.Merijn Brand discovered that PerlIO::Scalar ignores format and write, preferring instead to send the output to STDOUT, instead of the associated variable.

  The last user of formats in the world

Time::Piece is back from the future.

David Landgren discovered that a literal before \k<foo> (a named back reference) can really ruin a conference talk. Yves Orton resolved the problem in 20 minutes, give or take a few.

The 5.8.3 sprintf patch made its way into the Mac OS X Security Update 2006-007.

Paul Marquess independently rediscovered the joy of testing filehandles with threaded 5.6.x perls, and decided to give it a wide berth.

Bo Lindbergh proposed adding lexical scope to SelfLoader, complete with patch, but elicited no comments.

Gabor Szabo and Abe Timmerman talked about the broken pages on

  In November

  and in December

Yves Orton had some unhappy results with Time::HiRes. After an incredibly long thread, everything was sorted out.

Johan Vromans was worried that adding Module::Load::Conditional to the core would be like unleashing Pandora's box.

Yves Orton wanted to know whether 'twas better to merge bugs or set up dependencies.

  Whatever lowers the count most

Jarkko tried in vain to sort out the UTF-8 -related failures that have been occurring in the smokes since August. Nicholas Clark explained the difficulties.

Jarkko added some things to think about in perltodo.

  Nick's movie

Jim Cromie noticed that the install target misses Compress/IO/{Base,Zlib}. The ideal solution involves using a more orthodox directory layout for the modules: Configure gets too confused otherwise.

Rob "Sisyphus" puzzled over integer exponentiation under -Duse64bitint on Cygwin, wondering why the same result could be stored as an NV (a float) or an IV (an integer). Nicholas Clark could no longer remember why, only that it was a conscious design decision not to do integer exponentiation.

About this summary

This summary was written by David Landgren.

Weekly summaries are published on and posted on a mailing list, (subscription: The archive is at Corrections and comments are welcome.

If you found this summary useful, please consider contributing to the Perl Foundation to help support the development of Perl.