This Fortnight on perl5-porters - 23 October-5 November 2006

This Fortnight on perl5-porters - 23 October-5 November 2006

Yes, I complained to Corel, and they said "thank you for your insight as a valued customer, we forwarded this to our manager of customer support". So maybe someday they will fix the problem, or maybe they will not bother. -- Glenn Linderman, being realistic.

Topics of Interest

New version diagnostic breaks a bunch of modules

The discussion of trapping error messages, and future-proofing the code that traps error messages continued on its merry way this week, with lots of attempts and trying to figure out what the code would look like from the client side. If that was sorted out, the implementation would be trivial. The biggest issue was minimising the amount of makework code required on the client side, otherwise people will never remember the incantation and will get it wrong and/or waste time.

I would say Glenn Linderman made the best proposal, way down at the end of the thread, with his errorcodes module. But then I'm not sure I'm qualified to judge, since my code never has errors.

And the thread trickled on into November.

Can't compile blead on SuSE 10.1 (x64_64)

Andy Dougherty and Cosimo Streppone waded through the difficulty of getting Perl compiled on a 64-bit SuSE linux host. But by the end of the fortnight, Tels, who raised the original problem was still not getting complete satisfaction.

Patches for dual-lived modules

Jerry D. Hedden wanted to know if patches could be forwarded to him first, instead of blead, as it would result in fewer patches all round. There was some reluctance, since there are many cases where the putative maintainer is busy with Other Things in Real Life and are not always as responsive as Jerry. Thus, people like to be able to make changes to the codebase and worry about CPAN catching up afterwards.

In terms of the threads module, Dave Mitchell stated that he still considered the core as being the primary "home" of the source, the CPAN modules just being a way for people to get new thread goodness now, without having to wait for 5.10.

Configure patch for 5.\d\d.\d+

Andy Dougherty thanks H.Merijn Brand for taking the time to prepare this patch. H.Merijn pointed out that it was unlikely to be exercised by the current test infrastructure, and if people cared about older perls they would have to pay special attention.

  Listen up

A call for File::Spec/Cwd maintainers

Ken Williams announced that his copious spare time is no longer sufficient to devote the time required to keep these two modules happy and bug free, and asked for volunteers interested in becoming co-maintainers.

Johan Vromans kicked off the idea of a small working group with access to a wide variety of platforms and perls to sit down and and figure out how to cross-platform file-system manipulations the Right Way. Ken Williams thought that Path::Class was the right way, but admitted to being slightly biased, since he wrote it.

Profiling and dprofpp

Nicholas Clark bemoaned dprofpp's sluggishness at processing a 150MB tmon.out file, and wondered whether anyone who had said that they would take a look at the problem had ever made any headway. He wondered if the suggestion to rewrite it in terms of Simon Cozens's Devel::DProfPP module was feasible. For starters, he wasn't sure of how to go about setting up regression tests.

Data::Bind compilation failure with blead

This failure was caused by the fact that no longer loads Carp at compile time, only if needed, and this masked what was, strictly speaking, a syntax error in Data::Bind.

If the change that gave rise to this were to be backported to maint, a number of CPAN modules would break, which would mean that 5.8.9 would not be a drop-in replacement for the 5.8.x series. Yves wondered whether it was sufficient to flag it as an important main upgrade issue, and document a work-around.

Closures with named subs

Christian Winter was puzzled by how closures behave when the closed-over routine is named, and the documentation didn't help him understand what was happening.

After it was explained to him, he cooked up a patch for perlref.pod, which Rafaël Garcia-Suarez applied.

Valgrind findings

Rafaël ran valgrind over blead, and noted that the memory was leaking like a sieve. Nicholas made a couple of suggestions on leaks that may be caused by optrees being half-destructed. Yves cleaned up the leaks from the regular expression engine.

Rafaël ran a second round, and 14 previous test files came up clean.

He found two leaks with One, he fixed, the other, he did not know how to proceed.

Perl source code protection and the demise of B::Bytecode

Nicholas Clark received out-of-band feedback on the announcement that B::Bytecode has been dropped for 5.10. Turns out there are people out there who are using it to deliver shrouded Perl code.

Nicholas pointed out that it was dropped because no-one who cared sufficiently about the matter had neither the time nor inclination to look after it. If someone starts to maintain it again then it's another story. In the meantime, he wondered what other solutions were available to "protect" Perl source code.

Steve Hay's Filter::Crypto probably comes the closest these days. Nicholas, imitating Bruce Schneier, noted that "As the attacker, you only have to find the weakest link. As the defender, you have to ensure that no leaks are weak. The defender can only delay the inevitable."

Adam Kennedy thought that munging the code through Perl::Squish and then possibly creating random xyzzy variable names would go a long way to making the code sufficiently difficult to understand. You could also check that the test suite still succeeds after shrouding.

Andy Dougherty mentioned that it might be worthwhile hanging out a sign in the usual places saying "Orphaned modules! The following modules will be removed from 5.10 unless someone offers to maintain them". Not that he expects anyone to reply, but at least that way we don't have to feel guilty about it.

Scott Walters admitted to liking Logo and SNOBOL, and that it might be fun some day to revive the Bytecode material in order to allow those languages to emit Perl bytecodes, which could then run Perl (and access CPAN). Some day.

Last words on the matter from Tels, in November:

UTF-8 Failures in smoke ($PERL_UNICODE)

An old bug from several months ago raised its head again. With the restructuring of some part of the internals, we achieved the possibility of have two right answers to a test, and Nicholas Clark couldn't find an easy way to hack the test suite to cope with the fact that (to paraphrase) foo() could return "bar" or "rat", and either would be okay. He was then thoroughly swamped by hordes of volunteers beating a path to his Inbox offering to help out. (I am being sarcastic).

The problem surfaced again, this time managing to snag Dave Mitchell's attention (who lamented that his p5p spool currently stands at over 4000 messages), and so he missed it the first time around. (And what are my summaries? Chopped liver?)

As the warnings code was mainly Dave's handiwork, he was able to craft a fix to improve the situation.

Unclear version error message

Glenn Linderman was playing around with PAR, and didn't realise that WordPerfect Mail had discretely slipped a perl58.dll into c:\windows\system when he wasn't looking. So the PAR stuff spat out a strange message about a version of Perl he didn't have, and it took him a certain amount of time to figure out why things were misbehaving.

To cut a long story short, it would have helped if canonical file names appeared somewhere in the error message. The fact that Glenn had been using PAR meant that $^X and $0 didn't point anywhere near the library.

Yves pondered whether it would make more sense for Perl to use a little more of the Windows infrastructure, and stashing version information of installed modules into the registry. Andy Dougherty wasn't too sure about this, feeling that this was more of a vendor issue than one for the porters.

Vadim Konovalov pointed out that the registry was not a good idea for a number of reasons. A big problem is that locked-down business installations may not even let the user account write to the registry.

Glenn thought the whole registry business was annoying, because if you copy an application from an old machine to a new one, it doesn't work, because of all the missing registry entries. At this stage, the thread started to discuss the finer aspects of how to get multiple Perl installations playing nicely on the same machine, and strategies for deal with moving the Perl directory around and still have it work.

Andy Dougherty provided a useful executive summary here:

INC handlers and __FILE__

A selfless implementation

Chia-liang Kao asked what the porters thought of his module, that does a my $self = shift. The reactions varied from "cute" to "sick".

Patches of Interest

Deparse bug redux

Stephen McCamant followed up on Juerd's incorrectly deparsed monster reference {{{{{[[[[[[[[[[]]]]]]]]]]}}}}} and produced a real program puts it in context and produces bad behaviour. He also bundled in a patch, to correct the problem in B/

A stab at UNITCHECK blocks

Now available as a module on CPAN. fixed wrong treatment of PATH entries

Alexey Tourbin had a problem with FindBin getting mixed up with executable files appearing in directories on DOS-ish platforms. In a nutshell, he wanted to skip any irrelevant files that may be encountered.

Adam Kennedy suggested that Alexey's one-line fix would be complete with a test.

Fix a problem with jump-tries, add (?FAIL) pattern

Yves Orton discovered a flaw in his previous work on adding jump tries (/(foo[ab]|bar[cd])/ worked, but /(foo[ab]|bar[cd])+/ did not). In the process of fixing it, he invented a new regop FAIL, which deals with things in a more elegant manner. And some tests, too.

This lets you specify deep in the middle of a pattern that, no you really don't want to match anything at all. Sometimes, this comes in handy.

Supporting 'threads' with multiple embedded Perls

Jerry D. Hedden landed a patch to allow multiple Perl interpreters and threads play nicely together, following an issue raised by Jan Dubois.

  the big issue

Jan had a look at the code and made a couple of suggestions on how to improve it.

Add a few useful developers maketargets to win32/Makefile

Yves Orton tweaked the Win32 Makefile to add a couple of targets, regnodes, which comes in handy when hacking on the regular expression engine (one can see what itch this scratches) and regen which does the whole lot. Making only regnotes avoids a considerable amount of useless recompilations.

Add a commit verb to regex engine to allow fine tuning of backtracking control

Joshua ben Jore appears to understand some of the more esoteric improvements that Yves has been making in the regular expression engine. He asked how (?COMMIT) behaves in terms of (?>...) and was most impressed with the answers Yves gave. Since I've never taken the time to understand the usefulness of (?>...) in the first place, it all went slightly over my head.

Removing the beta status from the IO::Compress modules

Paul Marquess expressed the belief that the IO::Compress modules were sufficiently stable to be moved out of went out of beta status. He made one final tweak.

This triggered some problems with threads and Readonly, and so he fixed that up.

And then bumped the version numbers and called it a day.

Add more backtracking control verbs to regex engine (?CUT), (?ERROR)

More backtracking goodness escaped from our favourite mad scientist of the regular engine. I'll have to play around with all this stuff to see what sense I can make of it, but it certainly looks nifty.

Monitoring wedged tests on Win32

Steve Hway was having problems with his smoke machine. Sometimes a test will hang, and the test run is ruined. It used to be worse, but it's still not perfect.

So a method is needed to monitor the tests, and detect when they get wedged. And, more importantly, killing them when they do get wedged. Jerry D. Hedden offered a patch for t/TEST that sets up a monitor thread to keep an eye out for tests that get stuck (and thus completely ruin a smoker's evening).

Unfortunately, it didn't work out as expected, since it appeared that when things go wrong and the monitor thread tries to kill the blocked thread, it is the parent of the blocked thread that winds up taking the bullet.

Jan Dubois said that a robust solution to this would be to use Win32::Job, a module that ActiveState wrote to help keep work with building ppm distributions, and not having one bad package in the batch blocking the show. But alas, Win32::Job is not in the core.

Nicholas Clark did not see this as an obstacle. If the module was needed, in order to test core modules, then there was no reason it should not become core.

Another thing Jan wanted to do was to emulate Unix's kill -9, $pid, to take out a whole process group in one fell swoop. He was strongly encouraged to proceed with this.

New and old bugs from RT

Special var @- becomes arbitrarily large (#36046)

Yves Orton figured that this was fixed in blead, and added a couple of tests to make sure it stays that way.

Use of study() with utf8 enabled breaks regexps (#37646)

Similarly, Rafaël Garcia-Suarez noted that this bug was also fixed in blead.

sprintf "%#04X" also uppercases the 0x-prefix (#40583)

Dr. Ruud noticed that, for instance, sprintf "%#04x", 15 produces the slightly odd looking 0X0F, and contended that it should produce 0x0F, as per the C spec, or the Perl documentation, or something.

This kicked off a large thread where it was noted that Perl lets sprintf produce binary 0b001011 strings, but not 0B001011, that perl doesn't know how to eval "0X0F" and a few other things besides. By the end of the thread, blead had been patched in various different ways, tests added and documentation amended, in an attempt clean up most of the glaring inconsistencies.

Source code patch for win32/perlhost.h (#40591)

Kenneth L. noted a problem with building perl on Win32 with dmake and MinGW. Steve Hay noted that it was fixed in blead, and suggested what packages Ken should download in order to build a perl without such problems.

Out of memory message should not require memory allocation (#40595)

Schmorp (possibly Marc Lehmann in disguise) found that when perl runs out of memory, it needs to allocate memory in order to print the error message. But it runs out of memory, so it has to allocate some memory to... you get the picture. And Schmorp gets a segfault.

  there must be A Better Way To Do It

gnu/linux (glibc) likely needs USE_PERL_SBRK (#40603)

Marc Lehmann found that bad things happen when one uses modules that use pthreads with a perl built to use its internal malloc (via usemymalloc=y). It seems that both glibc and perl's mallocs wind up allocating overlapping areas, with predictably hilarious results.

It turns out that both perl and glibc call sbrk to acquire more memory, but sbrk is not thread-safe, so concurrent calls all get the same result.

Andy Dougherty wondered whether the hints for linux/glibc should prevent usemymalloc=y from being chosen, or whether usemymalloc=y provides such benefits that it may be worth spending some effort to get everything playing nicely.

Nicholas Clark remembered that people have been having trouble with threads on FreeBSD, and that it may well be for this very same reason, and usemymalloc=y is definitely worthwhile on that platform.

srand() behaviour with large numbers (#40605)

Brian Szymanski uncovered some nasty behaviour with srand() when the seed approaches the size of the CPU width.

Crash with unicode characters in regex comment (#40641)

Where yet again Sadahiro Tomoyuki does battle with UTF-8...

  and wins!

(Obscure) Bug in Dynaloader (#40651)

Why you should always assign to $VERSION in a BEGIN block.

Unclear version error message (#40652)

The bug report of the problem raised by Glenn Linderman (see Topics of interest).

Bleadperl crash in Perl_pp_entersub() (#40681)

Interesting thread, not enough time to summarise it, sorry.

Perl5 Bug Summary

  8 more, 5 fewer, leaves 1531

New Core Modules

In Brief

Andrew Savige wrote a new test for close-on-exec ($^F) in t/run/cloexec.t, as there were no tests for it. Since the test used a magic number, Steve Peters thought that it should be hidden behind a preprocessor macro in perl.h.

  This test brought to you by the number 3 and the letter F

The problems surrounding change #29050, a memory leak fix by Jarkko were all sorted out. There was still a problem with glob and threads on VMS that Craig A. Berry needed to look at.

  done and dusted

Craig reported that this was due to asymmetry in reference counting in PerlIO_tmpfile.

Vadim Konovalov wanted to know why DynaLoader's dl_win32.xs was located in ./win32 and not ./ext/DynaLoader along with everything else.

David Feldman began work on a patch to provide better Attribute::Handlers diagnostics.

Alexey Tourbin noted that the documentation and behaviour of sv_setref_pv() differ, and couldn't decide if this was a bug or a feature.

Andreas König still didn't know what do about change #24542 breaking Math::Pari.

Rafaël crossed off another perltodo TODO: readpipe and qx() (backticks) are now overridable. Saves having to pull in an IPC module.

Considerable progress was made on getting perl built with VC++ 2005 (VC8).

Andreas König noted that XML::Writer 0.601 throws the new Variable length character upgraded in print diagnostic, which means it probably needs to be tweaked.

  ooh, we found a bug

About this summary

This summary was written by David Landgren.

Weekly summaries are published on and posted on a mailing list, (subscription: The archive is at Corrections and comments are welcome.

If you found this summary useful, please consider contributing to the Perl Foundation to help support the development of Perl.