Modern Perl, the book, is now available in ePub format as well as A4- and letter-sized PDFs.

There is no DRM and there are no special licensing terms. Please redistribute this book far and wide and tell everyone that they too can write great Perl 5 code that takes advantage of all of the hard-earned wisdom and effort we've put into improving the language in the past several years.

Of course, you can always tell people to buy the printed copy—I'm a fan of paper—but the important part is spreading the word, whether through reviews on Perl Monger lists, web sites like Slashdot and Ars Technica and LWN, and of course booksellers. You're also very welcome to upload and seed torrents, post it on ebook sites, or just hand around USB sticks whenever you get together.

Bilingual Perl hackers have already begun translations into other languages, and we're very happy to see this and help in any way possible.

If we're going to show off how great Perl programming can be, let's make sure everyone has a chance to read a good book about it.

This sort of thing will happen again. Perhaps we can learn from this experience and handle it better next time.

What should you do the next time you see example code with a security flaw and/or clunky style?

• Contact the author directly and privately.
• Explain the security flaw and demonstrate how to fix it and it alone.
• Expand further on stylistic issues (explaining that, although there is not complete agreement in the entire Perl 5 community over some stylistic issues, the dominant trends are fairly widely understood as something we can all call modern or enlightened Perl), revising the code as necessary.
• Suggest that the Perl 5 community has many resources such as PerlMonks, Perl Mongers, and mailing lists full of people who can and will perform technical review on code.
• Be polite.
• ... if that does not work, publish an explanation and corrected code elsewhere.

Where possible, stick with the constraints of the article. For example, Ms. Özses's code has good reasons to use CGI instead of a Plack-capable module. Respect that. (Though where it's obvious that a CPAN module is far superior solution, such as in the case of CSV parsing, do recommend the module.)

We share a goal, that useful and usable and correct and secure and sufficiently elegant code is available. We do not have to write the same code nor the best possible code to achieve that goal, but...

... people who publish code examples have a responsibility to write good code.

... people who review and comment on code have a responsibility to be both accurate and polite.

To that end, I owe Ms. Özses an apology. However I intend neither personal attack nor insult in my writings here, I failed to live up to my personal standards of kindness and politeness in this case. She deserves much credit for taking the suggestions of various Perl 5 programmers despite unwarranted criticisms.

I wrote T::MO several years ago when I noticed a pattern in test code I'd written. I wanted to isolate a couple of cases of behavior to test all of the code paths within specific functions and methods. Invariably I took advantage of Perl's dynamism to construct objects which adhered to a specific protocol of behavior but which were under my direct control.

Think of it this way. You want to test the error handling code for a database abstraction layer. If the network connection goes away or if the remote process crashes, you want to retain the data and perform the appropriate logging and recovery.

How do you test that? You force a failure. No, you don't hire an intern to yank the network cable during your tests. You mock up the code which raises the "Wow, it's suddenly quiet in here!" exception.

This is where I wrote T::MO wrong. Instead of mocking just one piece of the underlying plumbing, T::MO encourages people to mock the entire system, from the faucets down to the sewer system. You do get more control over the process, but you can lose yourself in a sea of sweaty little details.

T::MO can be an invaluable tool. Sometimes it's exactly what you need, especially when you have to clean up a pile of legacy code written without good design taste. Even so, I wish I'd written Test::MockObject::Extends first to encourage people to mock only part of the behavior of an object. If your code is anywhere close to well-factored, you should be able to splice in the specific behavior you need, scalpel precise, and have confidence that the rest of the code, the real code you run when you run the code for real, behaves appropriately.

The more you need to mock, the more coupled (and, generally, the poorer) your design. The less you do mock, the better your tests.

This is fixable.

From the conceptual side, all someone has to do is to change what Perl 5 throws for its core exceptions and warnings from a string to an object. That object can overload stringification so that all Perl code which treats it as a string will continue to get the string value. All code which treats it as an object will continue to work correctly even if the string value changes. (I haven't thought about how this might break XS code which pokes into SV guts with macros....)

Someone could even provide numeric overloading so that you can compare exception types numerically without having to call methods to figure out exception information.

Reconfiguring Perl 5's guts to make this possible is fairly simple, at least at the point of the API which actually throws the errors. A few functions in util.c such as Perl_croak() need to build an object instead of a string, but that's a modest amount of code. It's much more difficult to find every place in the Perl 5 core which calls Perl_croak and friends to change them to use the new API...

... because the right way to make this API work better is not to pass C strings as the text of error messages but instead to pass symbolic constants which represent error messages. For example, instead of calling Perl_croak( "Can't invoke non-invocant" );, the calling code should instead use something like Perl_croak( PERL5_NON_INVOCANT_EXCEPTION );. This allows a quick lookup of the right exception information as well as another benefit: localization of exception messages.

Even so, that's still a lot of code to change (590 uses of Perl_croak* in the .c files of bleadperl alone, not to mention everything on the CPAN)—and this code won't be available for wide use until 5.14 next spring at the earliest. In a few years, maybe enough people will use exception objects by default that it's possible to clean up error messages throughout the core without worrying about breaking fragile old code. Then again, fragile code tends to do the lazy thing, not the correct thing.

As an intermediary step, perhaps it's possible to refactor the core to use exception type concepts instead of literal (or sprintf-style) strings within the bleadperl source code. That would allow for localization, if that's desirable, and it gets Perl 5.13.x closer to making real exception objects useful.

The real question is whether real exception objects in the core are sufficiently worthwhile to justify this change. If the best rationale for this work is "Someday, we may be able to fix old, misleading, crufty, or wrong diagnostic messages!" then ... well, how soon will be too late? This is why to plan for making—and correcting—mistakes in your language design.

In Perl 5.10, that message changed to include the name of the uninitialized variable when it's available. This was a huge usability improvement and it's one of my favorite features of modern Perl. Sometimes changing a warning or error message for user clarity is the kindest improvement ever.

(Having an interactive shell react to the user typing exit with the message Type Ctrl-D to exit! is a response to user confusion in the wrong direction.)

Unfortunately, changing error and warning messages in Perl 5 is a quagmire because parsing unstructured data is a mess. Want to catch an exception in Perl 5? You can do it safely with a module such as Try::Tiny, but to find out what the exception is, you have to perform string manipulations on $@.

If you're performing an exact match, a substring match, or a regular expression, any change to the text of that message in a subsequent release of Perl 5 could change the way your code behaves. Thus p5p must be very, very, very careful about even improving the text of internal exceptions and warnings, lest they break the DarkPAN.

The only real solution in Perl 5 is never to attempt to parse the text of exceptions and to consider throwing exception objects with something like Exception::Class. Adding true exception object support to the Perl 5 core in a backwards-compatible fashion is also very possible, though that's a subject for another post.

A client had asked me to review the memory usage of a suite of programs running on a shared host. His account had a low hard memory limit, and when multiple programs ran simultaneously, the Linux kernel's out of memory killer killed one of the processes.

I ran the program on my machine and saw about half the memory usage. "Ah," I thought to myself. "I have a 32-bit Perl 5 installed and he has a 64-bit Perl 5 installed. "There's lots of little data in the program, so the double sized pointers on his installation compared to mine account for the memory difference."

This is where I should have realized that profiling such an assumption would have taken only a few minutes and could have saved an afternoon.

Installing a 32-bit Perl 5 on a 64-bit machine takes some work. You have to convince your C compiler to compile in 32-bit mode, and this must take place during the point at which you configure Perl 5. After a few failed attempts and a couple of patches to perlbrew, I had a new custom installation of Perl 5.

I installed the dependent modules and ran the application... and the memory use was only slightly better.

Then I thought for a bit. Then I instrumented some code for a quick profile and realized the problem. The application fetches a list of tasks out of a data store it uses as a queue. This queue has an iterator interface, as the list of tasks can be arbitrarily large. Within the code which performs the fetch, I'd assumed that the dataset would be reasonably short at any given time, so I'd flattened the iterator into an anonymous array and kept the whole thing in memory.

The program now processed more data than I'd ever planned, and the problem had only grown worse as more and more tasks piled up, as the consumer couldn't keep up with the producer thanks to OOM killing.

Five minutes later, I'd switched back to the iterator interface, and the process used a fraction of the memory than it did before. If I'd asked myself "What within the program grows over time?", I'd have figured out my mistake before I recompiled Perl 5.

So it goes sometimes.

The Perl ecosystem has room for other projects, however.

For example, one of the clearest benefits Perl 6 has over Perl 5 is its portability to other virtual machines and runtimes. By design Perl 6 encourages multiple implementations. Perl 5 is its own specification; in many places, what Perl 5 is is solely what Perl 5 happens to do. Sometimes that behavior gets enshrined in the specification tests, but other times it's folklore and institutional community knowledge.

Just as Parrot's Lorito project intends to make Parrot at least an order of magnitude faster, so too a reorganization of Perl 5 internals could make amazing things more possible.

What if there were a project to implement a minimal set of Perl 5 on the Parrot virtual machine as a prototype and exploration of how much of Perl 5 you can support, the effort it takes to do so, and what kind of utility you can expect? Parrot's compiler tools let the Rakudo developers write most of Perl 6 in Perl 6; surely it's possible to write Perl 5 in a similar fashion. (Credit to other projects such as Rubinius and PyPy for demonstrating that such things are possible.)

I know other projects have attempted this in the past. Perhaps the best place to steal information is Bradley Kuhn's masters thesis, Considerations on Porting Perl to the Java Virtual Machine.

As Jesse wrote in his comments, bug-for-bug compatibility isn't necessary. Nor is full compliance with the existing Perl 5 test suite. A simple proof of concept to produce the 80% of Perl 5 most people use in most programs should suffice. (Parrot gives you a lot of that anyway.)

As a bonus, you get cheap and easy interoperability with Perl 5, access to Parrot features such as multidispatch, grammars, continuations, and bytecode serialization, and you could even replace some of the uses of Perl 5 within Parrot's and perhaps even Rakudo's configuration and build processes.

It doesn't even have to be a pony of full size.

Bad code is possible in any language and wrong code is possible with any API. Even so, it's possible to create languages and APIs which make the right thing so much easier than the wrong thing that only the most incompetent (or dangerously malicious) write bad code.

Imagine, for example, a database access layer which forbids the use of raw strings to create SQL queries. You might have to write:

my $sth = $dbh->select( @tables )->join( %relations )->where( %conditions );

That's not necessarily a beautiful interface dashed off after a moment of thinking, but it has an important security property: it avoids the interpolation of untrusted user input. All data sent to the database may go through a quoting or untainting process without the user having to remember to do so.

A similar library could help avoid malicious user input from interfering with the display or operation of a web site, for example. These are both specific cases of a general principle: replace unstructured string data with structured data. In both cases, the structure of the data makes the intent of the data clear, which allows the library to ensure as much safety as possible.

This principle has other implications as well; more on that next time.

{
    package Cat;

    use Moose;

    has 'name', is => 'ro', isa => 'Str';
    has 'age',  is => 'ro', isa => 'Int';
    has 'diet', is => 'rw';
}

... than the corresponding code where you must write your own accessors, poke into a blessed hash directly (and bless it yourself), perform your own coercions and verifications, and the like.

Of course, the preferred syntax for doing this within the Moose documentation is different from how I've done things. Moose recommends:

{
    package Cat;

    use Moose;

    has 'name' => (is => 'ro', isa => 'Str');
    has 'age' =>  (is => 'ro', isa => 'Int');
    has 'diet' => (is => 'rw');
}

Sometimes you quote the name of the attribute and sometimes you don't.

Should I drop the parentheses? Should I drop the fat arrow between the name of the attribute and its specializers? I do in my own Moose code for my preferences, and I did in the book. Then I thought about it and realized why I write code this way.

First, a digression. Perrin Harkins mentioned the inability of the "Takes a block!" prototype to replicate builtin syntaxes as a reason to dislike syntax-bending modules such as Error. For example:

use Error ':try';

try
{
    ...
}
catch
{
    ...
};

... really needs that trailing semicolon. For similar reasons, many modules which use Devel::Declare magic go through contortions to add trailing commas and semicolons. Perl 5's syntax is malleable, but when the parser wants something from the lexer, it really really wants something from the lexer. (When it wants to know that a statement or a group of terms has ended, you don't get to lie.)

In other words, even though you have a lot of options for mangling Perl 5's syntax any way you like it, the semantics of the host language will shine through. A parenthesis is a parenthesis. A labeled block is a labeled block. A bare sub { ... } is never an expression on its own, and it can never terminate an outer expression.

This is one of the downfalls of the so-called "embedded domain specific languages". If you haven't written your own parser, you'll have to take what you can get. This is even true if you do write a parser and generate and eval code, and it's especially true if your EDSL desugars to chained function or method calls.

I'm not suggesting a flaw with Moose's approach: it's clever and Perlish and doesn't succumb to the saccharine cutery of so many other so-called DSLs. (To my knowledge, no one in the Moose world has claimed it's anything other than Perl 5 syntax bent slightly into something which looks declarative enough.)

My concern—especially when explaining object orientation in Perl 5 to novices—is that any extra syntactic elements might confuse people to think that they mean more than they mean. You and I might both understand that the grouping parentheses in the Cat attribute declaration are merely visual hints to the reader that the specializers are subordinate to the attribute itself and that the fat arrow between the name of the attribute and its grouped specializers confers the notion of pairing between the attribute and its specializers, but how do you explain that to someone who's still struggling to figure out what this encapsulation thing is all about?

I've attempted caution throughout the book such that the fat comma always signifies a pairish relationship, such as for hash keys or named arguments. Certainly you can always use it in place of the skinny comma (and, barring any quoting changes, vice versa), but is it clear to do so?

Likewise, you can wrap parentheses around almost any old rvalue (barring precedence changes) and not change the behavior of lists, yet this confuses novices all the time:

my @lololol = ( 1, 2, ( 3, 4, (5, 6) ) );

I'm not criticizing the Moose documentation or the standard approaches to formatting Moose code. I'm not suggesting a change. I don't like deviating from community standards for declaring Moose attributes. Even so, avoiding the need to explain the equivalencies of syntax to people for whom learning syntax is still a really big deal is itself to me a big deal.

Nicholas Clark reminded me over the weekend of something I heard long ago. In particular:

... as well as action at a distance, there's a speed hit on every class method call because first the code does a stash lookup to see if the package name string is actually a filehandle ...

— Nicholas Clark, methods and bareword file handles, action at a distance, (un)speed

That is, when invoking a method on a bareword (such as the name of a class), Perl 5 has to check whether a filehandle of that name exists, then fetch the underlying filehandle object, then invoke the method on that object. This takes place even if you never use bareword filehandles in your program. It's likely too risky to change now (at least in Perl 5) lest millions of programs written since 1994 suddenly and mysteriously break with odd error messages.

(The correct design solution is to forbid the use of ambiguous barewords at the syntax level in order to avoid costly and often unnecessary runtime checks. In other words, similar things should look similar and different things—filehandles and class names—should look different.)