This Week on perl5-porters - 22 December 2007-29 December 2007

This Week on perl5-porters - 22 December 2007-29 December 2007

"Remember, everyone is a beginner, but only for a small time." -- Abigail, on why you can't please everyone, everywhere, all the time.

Topics of Interest

The summaries are back! I had meant to get this out earlier this week, but the season's festivities got the better of me. Enjoy! -- David

Consting goodness done to excess

Robin Barker, in a remarkable coincidence of increasing the summariser's burden, went about adding const goodness to XS modules in the core distribution.

  MIME::Base64 and Digest::MD5 (not applied)

  Compress::Raw::Zlib and Filter::Util::Call (not applied)

  Cwd and ExtUtils::ParseXS (applied, with slight turbulence)

  IO (not applied)

  Storable (applied)

  Digest::SHA (applied)

Not to mention the judicious application of a NUM2PTR macro (applied)

grep and smart match should warn in void context

Michael G. Schwern wondered why a grep operation performed in void context produced no warnings. The idea being that since a grep is a filtering operation over a list, you either want to get back some, all or none of the list, or possibly the number of elements that matched. But to want nothing at all was either stupid, an error, or some sort of obfuscation.

Nicholas Clark asked Michael to write some TODO tests to nail down the desired behaviour, but Michael went one better and produced a first cut at a patch to do what he wanted.

Yitzchak Scott-Thoennes wasn't against the idea, but wanted to make sure there was a simple way to disable the warning message, just in case. Rather than introduce a new void keyword, Michael pointed out that scalar grep foo(), list() works just as well.

  obfuscated map, anyone?

Big slowdown in 5.10 @_ parameter passing

Yves Orton forwarded a message from the DBIx-Class mailing list, where they had discovered with horror that

  my ($x, $y, $z) = @_;

has become unacceptably slow in 5.10 (whereas a semantically equivalent code block using shift ran as swiftly as ever). Dave Mitchell discovered that the optimiser had become confused, and was running the list assignment through the code path that spends the extra cycles that makes sure that

  ($x, $y) = ($y, $x)

works as expected. That is, extra care has to be taken when there are variables common to both sides of the assignment, otherwise things get clobbered.

  wish they had taken 5.10 for a spin earlier

Paul Johnson identified the patch responsible. No solution as yet.

  it was just a teeny bug fix

Should lexicals used only once warn?

Nicholas Clark asked innocently whether it would make sense to have the interpreter warn about lexical variables that were defined and then never used afterwards. A volley of reasons as to why this would not be a good idea followed, and ten days later the thread was still bubbling along nicely. Lots of nice idioms in there worth pondering; I liked Hugo van der Sanden's resource lock trick.

  don't call us, we'll call you

Extra warnings for beginners

Gabor Szabo kicked off a slightly shorter thread as to whether it would help to have extra warnings that would help beginners not fall prey to some types of simple errors. A concensus formed rather rapidly that this would not be a Good Idea.

  use brain;

Configure -Dunknown_config_var should exit with error

Jim Cromie wrote a simple patch to make Configure do sanity checking on its command line. Andy Dougherty explained just how hard it was to do such a thing correctly, as hints files complicate the picture.

H.Merijn Brand began to work on the scripts used to build the Configure file itself, and pulled his hair out over the bizarro coding style used. Jim Cromie was worried that if H.Merijn and Andy were simultaneously unable to work on shepherding the configuration infrastructure, The Perl project would be seriously stalled.

  bus fault in undisclosed location

Optimising opcodes

Jim Cromie had been wondering whether all the null op-codes left behind after the optimiser was done with an op-tree could cause cache misses due to bloat, as the CPU ran ahead of the data pipeline. Nicholas Clark wanted to find out whether adding another pass that threw away the dead ops, rearranging the ops into a more sensible order and storing them with a slab allocator would be a win.

Paul Johnson pointed out that one consequence would be that some error messages would no longer be able to relate to line number, as currently that information is stored in the very nullops that Nicholas was proposing to throw away.

  need some running code for concensus

Later on in the week, Jim returned with a patch to move op-sibling pointers out of OPs, the theory being that it reduces the optree's cache footprint by 20%. Unfortunately the MJD advice about getting the wrong answer as fast as possible applies, since the patch (which Jim admits is a work in progress) causes things to dump core.

  it's a start

Storable for bytecode?

Since the decision to jettison the Perl compiler for 5.10, perl no longer has a bytecode loader. Nicholas observed that Storable can serialise stuff, so, how much effort would it take before it could store an optree? And, at that point, would we have a byteloader? First up, Storable doesn't do regular expressions or typeglobs or... a whole lot of things it turns out, all of which are vital for an optree. Nicholas wondered whether it was feasible, easy or impossible.

Hugo said that the first step (this no doubt applies to the above thread of optimising opcodes as well) would be to overhaul the optimiser to separate the manditory fixups (that if absent, would cause the code to be unrunnable) from the true optimisations.

chromatic thought that a more promising avenue would be to remove the pessimisation in Perl 5 that makes the interpreter to poke at everything incessantly in case there's some sort of magic behaviour lying in ambush to do something completely different.

  the last frontier

APC now includes perl-5.10

Philippe M. Chiasson updated the APC (All Perl Changes repository) to include the 5.10 track. Since it has been over five years since the 5.8 track was created, there was considerable head-scratching on how to do a couple of things that had been lost in the mists of time.

Philippe was hoping that Sarathy could chip in with what he recalled of the process.

  write it down for 5.12

Array ~~ Any

Michael G. Schwern caught wind of a thread on Perlmonks discussing the behaviour in something like

  42 ~~ ['foo', 15]

warning, which some people find rather unsettling, about "foo" not being numeric. Michael wondered if something could be done about the matter.


Smart matching with objects

Ricardo Signes thought that smart matching and objects didn't mix very well as things stand, and suggested that objects participate in smart matching only if they overload ~~. This is a pretty reasonable suggestion, as it gives the object the power to decide how it smart matches, rather than the operator blindly assume that anything behind the implementation curtain is fair game.

Tels was surprised to learn that it was even possible to overload ~~ and began to wonder whether Math::BigInt objects and their relatives needed to do so. (Answer: yes, probably).

Michael thought that the path of least resistance would be to make a smart match against an object die, unless the class had overloaded ~~. Ricardo was a little squeamish at first, but realised that it solves future backward-compatibility concerns nicely: no need to suffer the result of a poor choice in what tricksy dwimmery ~~ the porters could invent on the spur of the moment to deal with a not-smart-match-aware object.

Larry Wall chipped in to point out that the smart match, as implemented in Perl 5 today, is now considered a misfeature in Perl 6, and there smart match behaves differently. That is, the match is determined solely by the nature of the RHS argument. Which means that if Perl 5 wants to borrow from the future and implement Perl 6 ideas, we need to track things more closely.

  electing to match

~~ changing behaviour after using ==?

In a similar vein, Gabor Szabo reported being tripped up by two scalars being smart matched, then tested for numeric equality, and then watching a subsequent smart match return a different result. This was in spite of the fact that the scalars had not changed value.

No-one had a really good answer, but Mark-Jason Dominus thought it reminded him of a heisenproblem with vec a few years back which in his recollection wound up labelled as, at best, a misfeature.

In the final thread of the week concerning smart matching, Jerry D. Hedden patched the documentation to indicate that ~~ is not a feature.

  it just is (unapplied)

Localising $@ in a BEGIN block

... prevents perl from noticing errors in the block. As related by Yves Orton, following a thread on Perlmonks. No-one ventured a reason as to why.

  looks like a bug

Always to use strict

Steffen Müller, whose post for some reason isn't showing up on Xray (and thus we point you to the first follow-up in the thread), suggested that it would be a really great idea to enable use strict by default for 5.11 (and hence 5.12).

While I was expecting a flamefest, surprisingly, everyone was more or less in agreement, although a push to enable use warnings as well had Abigail wringing her hands in dismay.

  there is no strict

Steffen cooked up a first cut at a patch to implement the desired behaviour, but Rafaël declined it, explaining that it felt a bit too hackish and suggested a better approach.

  the heat is on

Weird BEGIN block CV assignment behaviour

Peter Scott reported a question seen on c.l.p.m concerning typeglobs, BEGIN blocks and comments, for which he had no satisfactory answer. Nicholas and Dave Mitchell weren't able to do much better apart from some sympathetic waving of hands.

Regexps are people, too

Ricardo dreamt about the possibility letting an object knowing when it was being used in a regexp context, which would allow it to be used with ease in pattern matches and substitutions, and splits.

  my life is made of patterns

So Nicholas set about making regexps orange (no, don't ask). As part of the fallout, Jerry D. Hedden noticed that it caused threads tests to fail and proposed a patch to fix it.

Jerry then fixed the "orange" regexps in threads.

The shaking up of the code base in this way uncovered something else we weren't testing for, so Nicholas added a test:

And that brought us first class regexps (like ARRAY and HASH), rather than Regexp objects. Nicholas tried to change it to REGEXP but too many things broke. The underlying implementation structure is nothing more (for the moment) than a PVMG (a thing of magic) with a pointer to the regexp structure. The quest was on to see what could be hoisted out of the latter structure (such as reference counting) into the PVMG, since it had a slew of unused elements begging to be used.

Yves Orton liked the idea, and felt that it could go quite some way in reducing the complexity of the regexp engine. Nicholas continued to chip away at now-obsolete infrastructure code... until his monitor gave up the ghost.

There was idle chatter about renaming the REGEXP type to REGEX, RULE and PATTERN. ORANGE and PLUM also got a look in.

  he makes it look so easy

When strict 'refs' is not strict enough

Mark-Jason Dominus penned a thoughtful critique of why he thought strict 'refs' was not good enough. it was well-written enough to be summarised as: "stringifying a reference is probably an error". If you really need to do so, you should be explicit about it. Having it done implicitly is probably unwise.

Michael S. made a couple of pertinent remarks as to why things had to be the way they were.

Patches of Interest

Jan Dubois's 64-bit fix for Time::Local was applied.

And his patch to bring blead up to 5.11 on Win32 was also applied.

Vincent Pit found a two year old typo in op.c (using && instead of &) and offered a patch. Nicholas applied it.

Obsolete Nullxx macros

Jan Dubois spotted some now-obsolete Nullxx macros in x2p and took out the axe. Applied by Marcus.

Elsewhere in the tree, Jim Cromie found a couple of Nullstrs that had been missed in the initial cull. Also applied by Marcus.

One of these changes may have been the reason why Jerry D. Hedden had to fix cygwin.c again.

Loss of context with return do { my $x; 1} constructs

Some years back, Vincent Pit reported a bug (#38809) that reveals how return do {my $x; 1} returns undef, yet return do {1} returns 1 (except in taint mode).

It's all slightly confusing until you look at the output from B::Concise, which shows how the result from the do block gets discarded. Vincent Pit proposed a simple brute-force patch that does better, pointing out that a more subtle approach would be better, and requested comments (but received none) on the matter.

Replace FH by my $fh in open

Gabor thought it would be a good idea to use lexical filehandles in the documentation instead of globs. Michael G. Schwern thought so too.

He also suggesting listing where $_ is used. This was applied.

He followed up with another patch which received a lot more attention and came back with a revised version. Not yet applied.

Factor out duplicate code in struct xpv*

Marcus Holland-Moritz was puzzled, because with a short amount of work, he removed 377 lines of code and reduced the duplication of some hard-to-keep-in-sync code. He wondered if it was so blindingly obvious that no-one had spotted it before, or whether it was that way because of some impossibly subtle edge case.

On the downside, some struct members need to be wrapped in macros. On the other hand, some structures become identical, for instance, XPVIV and XPVUV are defined the same way.

Nicholas Clark was worried that the extra macroification might cause cranky compilers, such as the one on AIX, to have a fit. H.Merijn Brand took the patch for a spin on an AIX machine he had handy and gave it a clean bill of health.

Marcus discovered that there are 63 macros in the codebase that expand to more than 533 bytes, the grand-daddy of them all being REXEC_TRIE_READ_CHAR weighing in at 3697. In the end the patch was applied.

Watching the smoke signals

Smoke [5.11.0] 32712 FAIL(m) MSWin32 WinXP/.Net SP2 (x86/2 cpu)

Steve Peters looked at one of Steve Hay's smoke failures, and thought that it would be fixed by change #32713.

Smoke [5.11.0] 32718 FAIL(M) OSF1 V5.1 (EV6/4 cpu)

Jarkko Hietaniemi wondered if some recent change had broken the ability to compile the source with a C++ compiler (other than g++).

  and after all that effort

New and old bugs from RT

pod2html: Various markup errors with (nested) definition lists (#45211)

Steve Peters applied a patch that came from Debian.

CGI::Util::escape broken for iso-8859-1 data (#49055)

Slaven Rezic reported that this was broken in 5.10. Ævar Arnfjörð Bjarmason traced the problem down to the change in pack formats.

SIGTERM not terminating child process (#49073)

Jerry D. Hedden noted that a SIGTERM was not killing its child properly, resulting in a failure in Time::HiRes's test suite.

$_[0] seems to get reused inappropriately (#49115)

schmorp filed a report that boils down to being yet another Bizarre copy of ARRAY in sassign at Carp/ bug.

  need to get to the bottom of this

\R doesn't backtrack into \r\n (#49149)

Abigail discovered the above behaviour, but Yves was of the opinion that the Unicode specification pretty much imposes it.

$REGMARK not available in REPLACEMENT (#49190)

Abigail also reported that $REGMARK is not available on the right hand side of an s/// expression.

So naturally, Yves fixed it.

nested closures keep scalars alive (#49208)

schmorp reported a bug (on 5.8) regarding closures holding into scalars for too long, thus preventing them from being destructed before the end of the program.


Dave Mitchell confirmed that the correct, desired behaviour is seen on 5.10

  one reason to make the switch

Perl5 Bug Summary

307 new, 1465 open makes 1772 tickets (ouch). 28 created this week, 6 closed.

New Core Modules

constant 1.14

Sébastien Aperghis-Tramoni pushed out a new version of constant and then a second version followed hot on its heels (1.14), with the actual fix that 1.13 was supposed to contain.

  this is the one
SelfLoader 1.13_01

Steffen Müller made himself co-maintainer of SelfLoader, dual-lifed it, and pushed a development version out to CPAN.

  pull yourself in

Prior to this, Andreas König reported that the previous version was unhappy on 5.6.x

  what does '<&' on open do, anyway?

This is the BBC

Andreas König runs a smoke rig that checks as many CPAN modules in the shortest time possible against the most recent version of bleadperl, to see what breaks. Hence, Blead Breaks CPAN, or BBC.

  32013 broke Apache::DB (0.13)

  32707 broke Data::Alias, Devel::Declare anD autobox

  32734 broke Params::Validate and Clone

(The final item was reported by Steve Peters).

In Brief

Michael discovered that blockless greps run as fast as blocky greps, but both are about twice as slow as using a smart match.

  The color of surprise

Tels had some Warnocked Math::BigInt, Ubuntu and Module::Install woes

Marcus readied himself for the push to IPC::SysV 2.0

  if all goes according to plan

Nicholas responded to a May 2005 thread and replaced the C-level assert mechanism in the perl source by the standard C library assert mechanism.

  not dead yet

He also noticed with some dismay that there was no simple, elegant method for detecting the building of modules under the core. The PERL_CORE=1 signal was used in just about every way imaginable.

  Build a better core trap

Continuing, we find that he considers gcc -pedantic to be a mixed blessing, since it unfortunately disables a few important things like inlining macros, which kill performance.

  Bondage & discipline and speed: choose 1

Andy Lester reported on 5.10 coverage on the tech web sites

Sam Vilain, who has been working on getting the entire perl source history into a git repository reported that its tag for 5.004_05 returned the wrong set of files and in tracking down the reason, discovered that Module::CoreList was probably wrong and issued a patch to fix it (unapplied).

Jim Cromie, noticing that the Perforce repository was close to change 32768, or 2**15, tries to flog it off to someone willing to donate TPF, only to discover that H.Merijn Brand had carelessly burnt it several hours previously on a minor whitespace tweak.

  changes 33333 and 34567 still up for grabs

Ricardo proposed real exceptions for 5.12, picking up on some topics that were discussed in the run up to 5.10. None of the few people who answered were against the idea.

  something to be pursued

Gabor encountered a surprising test failure when building with a nonsensical path, as in sh Configure -Dprefix==/path/to/perl. (Note the ==). As the failure was somewhere in ExtUtils, Schwern sighed and said he'd have a look at it.

Marcus discovered a 8 year old bug (or at least a performance drain), and quietly committed a patch to fix it.

  UTF-8 file to UTF-8 scalar now 8% faster!

About this summary

This summary was written by David Landgren as part of the Winter of Code project. Thanks Vienna!

Weekly summaries are published on and posted on a mailing list, (subscription: The archive is at Corrections and comments are welcome.

If you found this summary useful, please consider contributing to the Perl Foundation to help support the development of Perl.