This Week on perl5-porters - 3-9 July 2006

This Week on perl5-porters - 3-9 July 2006

"CPAN is an unorganised anarchy. The checksum only guarantees that your downloaded file is identical to the one on CPAN, but there's no guarantee whatsoever that the file on CPAN is sane." -- Johan Vromans, reflecting upon the joys of open source software.

Topics of Interest

Is Perl an itch-free zone?

Following on from the thread following on from the unfinished summary of a couple of weeks ago, Nicholas Clark wanted to know if the Perl engineering process was too good for itself. With a dedicated crew of smokers, a fanatical attention to backwards compatibility, there is very little that an average author of CPAN modules has to worry about. If you have a module that works on 5.6 and 5.8, it is just about certain that it will work just fine on 5.10 as well.

This is not the case for other languages, like Java, PHP or Python, where things may change radically from release to release, and the hapless programmer just has to deal with it.

Scott Walters offered a certain amount of conjecture as to why things are the way the are (hint: it's got something to do with money). He then put forward a few items that he thinks about from time to time: fast bytecode loading, better coroutine support in the core and a bit of syntactic sugar for object-oriented programming.

Nicholas Clark thought that the lower bound of success for byte loading was to work with any Perl code you care to throw at it, and figuring out the upper bound of how long that would take seemed rather too high to be reasonable. Coroutine support in the core was reasonable: it requires adding more hooks in the interpreter that authors could grab hold of, and this can only be a good thing in terms of the innovation it might allow to spring forth.

Scott came back later on with some interesting information on how Python works. In a nutshell, it's still pretty much Guido's baby.

Jan Dubois chipped in to say that Tcl has a superb API to its internals, and you cannot sneak past it from extensions. This has permitted all Tcl extensions to be backwards and forwards compatible across the entire 8.x release series. This makes Jan very envious.

[background info: on #p5p a few weeks ago there was a discussion about the fact that most of the core developers know the guts of perl so well that they unconsciously play to its strengths and skirt around its weaknesses to the extent that there do not appear to be a whole lot of itches left, in their eyes.]

  The state of play

In a similar thread, Nicholas pointed to the absence of feedback from Perl users to the core developers. If CPAN authors out there have wishlists of things that would make their lives easier, it's not being channelled back to perl5-porters.

Similarly, the release candidates leading up to a new minor release get next to no testing, about from the usual band of smokers. This means that anything truly interesting that changes in the interpreter remains unnoticed up until after a release, which is what we want to avoid in the first place.

So, if you have an XS module, or an Acme module that does weird things, take a maintenance snapshot for a spin on your code and see how it fares. If you don't have any, try some of Damian's modules. Or those of the Phalanx.

  Nicholas Clark wants *you*

CPAN-ifying Math::Complex and Math::Trig

Jarkko Hietaniemi pondered dual-lifing Math::Complex and Math::Trig as CPAN packages, the idea being to be able to backport bug fixes and new functionality to previous Perl releases.

Dan Kogai was very enthusiastic about the idea, and hinted that it would be wonderful if Math::Complex were a little more bignum aware.

  Playing the numbers

CPAN and security

There was a rather long thread about CPAN. It's apparently broken, so people are discussing how CPAN-for-Perl6 will make it better. The evil-author-uploads-rootkit was mentioned. (Summariser's note: I do not mean to disparage this thread, it's just I feel that p5p is not the best place to discuss it).

  Read the source

Itanium warnings for Compress::Zlib

H.Merijn Brand sent in a list of warnings when Compress::Zlib is compiled on the Itanium platform. Paul Marquess initially hoped to steal ideas from the new 2.0 codebase, but H.Merijn quashed that idea.

So Paul had a closer look. One problem arose from the MEM_WRAP_CHECK macro, and Paul had next to no idea about what to do about that. Another warning was from the zlib source itself. The remaining warnings were from his own code, so Paul set about seeing what he do to silence them.

At the same time, he mentioned that this would be close to last ever release of the 1.x series, and that 2.x could start to move out of beta.

Paul then had a closer look at the MEM_WRAP_CHECK macro and isolated what he thought was the offending construct. It turns out that Yves Orton had already run into the problem last month:

  Compilers of very little brane

and it had prompted Nicholas Clark to opine that what really needs to be done is for compiler writers to add a -STFU switch to their wares. Andy Lester mentioned that he had given it his best shot some time ago at silencing the warnings. Paul debated the idea of generating an ideal macro at configure-time, but the more he thought about, he couldn't see how to avoid a picky compiler moaning about comparison always false.

  The ideal copy

Itanium warnings for Unicode::Normalize 1.01

And if the above wasn't enough, H.Merijn continued his quest to win friends and influence enemies by putting Unicode::Normalize through the Itanium wringer.

  It's all good

Escapes in heredocs

Curtis "Ovid" Poe wondered if he had misunderstood how backslashes behave in single-quoted heredocs, and wondered if he had misread the documentation, or whether the documentation was wrong, or unclear.

Sadahiro Tomoyoki cited perlop chapter and verse, where it says that in single-quoted heredocs, no interpolation is performed, so the snippet Ovid posted was working as advertised. Yves had a shot at improving the documentation, but Tomoyuki poked a hole in it.

  Heredoc, there a doc

Patches of Interest

Autogenerate all ppport.h files

Marcus Holland-Moritz produced a patch to generate all ppport.h files automatically during the build process. He asked for some feedback from Win32 and VMS porters. Steve Hay reported that it seemed fine on Win32. Craig Berry filed a small tweak for VMS, and explained what the problem was regarding, which does not show up on other platforms. This prompted Marc to change the way mkppport behaves, since if it poses a problem for VMS, it may pose similar problems on other, more esoteric, platforms.

  It's a job for Perl!

After having received the above feedback, Marcus announced that all ppport.h files were now being generated automatically by mkppport on Linux, VMS, Win32 and cygwin, and that any remaining bugs should get smoked out on other platforms soon. The end result is 160Kb saved in the bzip2 tarball, and/or 150Kb for the gzip tarball.

Nicholas Clark pondered what else would look nice shaved. After toying with the idea of autogenerating some tests, it was decided that there are no more easy savings to be had.

  use less 'disk';

Sys::Syslog also acquired its own ppport.h.

Dual life for

Adriano Ferreira discovered in horror that uses string evals to achieve its nefarious ends, and worse, is indented with tabs. So he proposed a new version that uses closures, and changed the tabs to spaces.

Rafael complained that the spaces/tabs change added a lot of noise to the diff, so Adriano redid the work as two separate patches.

  The modern world

consting, End of Part I

Andy Lester added some consting and localizing to universal.c

and toke.c.

Andy announced that he was going to scale back operations on Perl 5 and move to Parrot. Mainly, because most of the low-hanging fruit on the Perl 5 source tree has been harvested, and also because Parrot could do with some help.

  Moving on

state variables

Rafael Garcia-Suarez filed a status report on the current situation of state variables. He mentioned one problem remaining was what happens with declarations of state variables in list assignments.

Rick Delaney thought that about it and suggested what he thought the right thing to do was, but it turns out that Rafael spoke to Larry Wall, who suggested another alternative. David Nicol thought that Larry was wrong.

  A new state of affairs read full lines from remote socket

Brendan O'Dea sent in a patch to help out the EPIC (Eclipse/Perl Integration) crowd, who were trying to run the debugger over a remote socket, and sysread was giving them grief.

The new POSIX tests

Jarkko Hietaniemi discovered that the POSIX tests were spewing a lot of black smoke on the Tru64 platform, perhaps because Tru64 POSIXes you harder. He patched the test suite to make allowances for this, and this was applied by H.Merijn Brand.

  POSIX is the new black

Yves Orton tweaked other POSIX tests to be skipped if the function in question is not implemented on the host platform.

Latest cut-and-paste findings for blead

Jarkko also reported on the current state of play of his collaboration with the author of cpd (Cut-and-Paste-Detector) to get it to work with hairy C code, the sort which abounds in the perl codebase, due to the combination of heavily-nested macros and machine-generated source.

Sadahiro Tomoyuki put forth a couple of suggestions based on the results which Rafael applied.

Yves Orton found it too painful to try and deal with this stuff by hand, so he wrote a program in a popular text-manipulation language to do the job in his place. The result was some truly mammoth macros.

Dr. Ruud suggested parenthesising macro arguments to diminish the chance of precedence errors creeping in.

\N{...} in regular expression

Sadahiro Tomoyuki discovered that when use charnames ':full' is operative, /a\N{PLUS SIGN}b/ will actually match "aaaaaab", but not "a+b", which is possibly the wrong way around, and proposed a patch to fix it. It turns out that that \cX has similar problems, and Sadahiro-san located the culprit.

  Nice obfu potential

New and old bugs from RT

Literal Whitespace in RegExps not properly parsed (#5051)

Noted as fixed in 5.8.6 and blead by Yves Orton.

.pmc logic: keep, delete, deprecate? (#8860)

Steve Peters explained that the logic in question shall be neither deleted nor deprecated.

  Here to stay

Segfault with complicated regex inside map (#24898)

Steve also tried out a patch that Abigail offered to deal with a problem of the regexp state stack being reallocated during pp_match. Unfortunately, blead doesn't like it at all.

panic: magic_killbackrefs during global destruction. (#27630)

Steve thought that this was more likely due to the local patches that Red Hat added to their perl build at the time, since he was unable to produce it on more recent bleads.

Closing file seems to cause a seek operation (#29883)

Steve offered a work-around, by way of POSIX::_exit to solve this one.

tests fail in 5.8.8 if $TMP is not writable (#38947)

Steve added a check to bail out if the test being run was not able to write to the directory in question. Adriano Ferreira objected to the fact that no indication is given to the luckless programmer who may be watching the test results scroll past as to the reason why the test was skipped. Adriano thought a bit more explanation would be in order.

  Absence of evidence

Do not recommend in perlfaq (#39170)

Steve agreed that we shouldn't mention that the next release of Perl will have a switch statement, until we release the next release.

cygperl hangs on FIFO use (#39211)

Alexandr Ciornii indicated that this has been fixed in the latest version of the Cygwin library (version 1.5.20).

stat() doesn't work on dirhandles (#39261)

It does now, thanks to the efforts of Steve Peters. There is now a need to hoist out common code from Perl_pp_stat and Perl_my_stat, and some more tests would not go astray.

*** glibc detected *** free(): invalid pointer from Perl_mg_free (#39528)

Christoph Maser solved this problem by installing a more recent version of DBD::Mysql.

make test fails with getppid in a Solaris 10 zone (#39536)

Steve Peters took the change from blead to fix this issue and backported it to maint.

nl_langinfo ==> Installation problems (#39625)

Dave Mitchell explained that this problem (an undefined symbol) was more likely that the C run-time library or something else on the system dealing with shared libraries was messed up, and that it was unlikely that anything could be done from perlspace to fix it.

Segmentation faults with *@ (#39709)

David Serrano found a new way to do evil, by assigning to the *@ typeglob. Apparently, since nobody had thought of doing this before, it doesn't work as best it might.

$AUTOLOAD is never tainted (#39733)

Rick Delaney discovered this, and showed how a program could misbehave without giving an error. He then followed up with a patch to make it so. Hugo van der Sanden recommended that this be flagged as a significant change to keep in mind when upgrading.

Exporter::Heavy ignores custom $SIG{__WARN__} handlers (#39739)

"evanspa" reported that Exporter::Heavy pays no attention to warn handlers. This is unfortunate, as FastCGI needs this functionality, and so it was patched. Dave Mitchell commented that in blead, __WARN__ is no longer overridden (so the problem no longer arises).

crash in unpack (#39747)

Lukas Mai noticed a nasty problem with the Z specifier and null characters, but Yves Orton noted that it was already fixed in blead.

  Packed up

perl -d bug with 5.8.8 (#39758)

Shlomi Fish reported a problem with disappearing text in the debugger.

Perl5 Bug Summary

Gah! Back over the 1500 threshold.

  1501, to be precise

  All here

New Core Modules

In Brief

Craig A. Berry wanted to know how to interpret some threads 1.33 results on VMS.

Jerry D. Hedden asked Craig to take some code for a spin. This did the trick, and so Craig applied it to blead.

Dave Mitchell pointed out a certain category of the last remaining evals leaks, when Fatal upgrades warnings to death, may prove to be intractable in terms of leak proofability.

Shlomi Fish began a wikipedia entry about Tom Christiansen and asked for contributions.

Jarkko Hietaniemi provided a very useful addition to perlhack, listing some tricks and traps for C programmers to remember.

Gabor Szabo added some examples in the core documentation to explain some of the finer points of substr. Rafael applied the patch. It remains to be seen whether other functions lack similar, concise examples. Gabor mentioned he may send some more in.

Paul Johnson mentioned, in the Did you know this worked? thread that a variant of this trick is how he gets Devel::Cover to weave its magic.

About this summary

This summary was written by David Landgren.

Last week's thread:

received a comment from Jan Dubois clarifying the status of DBD::Oracle on ActiveState. Executive summary: it's good for 5.8 too. (And thus my evil plan for writing these summaries is unmasked: it's all an elaborate plot to get my own bugs and problems fixed -- and as bonus icing on the cake, Philippe M. Chiasson reported that a new version had been uploaded, which slices off about 50b of stuff that isn't required).

If you want a bookmarklet approach to viewing bugs and change reports, there are a couple of bookmarklets that you might find useful on my page of Perl stuff:

Weekly summaries are published on and posted on a mailing list, (subscription: The archive is at Corrections and comments are welcome.

If you found this summary useful, please consider contributing to the Perl Foundation to help support the development of Perl.