Yesterday I explained inheritance in the context of Moose and Perl 5 OO.

I'd already explained Perl roles. If I didn't have to acknowledge that people reading the book will eventually run across legacy code, I could happily gloss over inheritance as an idea that turns out to be much less useful than it first appears. I suspect that inheritance seems great to OO novices because it's spectacular and interesting: code appears seemingly for free out of nowhere! That's easy to understand, while the subtle utility of polymorphism and parametric polymorphism require you to have struggled with the problems they solve so you understand their value.

Would so many of us appreciate Moose if we hadn't wasted so much time doing things the hard way?

I spend a lot of time thinking about how to teach people to appreciate and to understand modern Perl in ways that do not lead them down the dark alleys and bumpy roads where I had to learn to program the hard way. I'm very stubborn, and I'm perfectly comfortable taking apart something to figure out how it works. (The people who fix bugs in your favorite programming languages likely share this trait.)

I don't believe that attribute is a necessary precondition for learning to program well.

However, I do wonder if it's more difficult to understand the value of a non-obvious system until you realize the drawbacks of the system you do understand. I've seen novices insist that they absolutely must use symbolic references instead of hashes, or that manual iteration instead of list transformation is so much easier, or what have you that I wonder about the diactic value of trying to show people the best general approach as I understand it now first, then mentioning other ways that aren't as good.

Consider an example. I know better ways to write Test::Builder because I've implemented it three times now, in Perl 5, Perl 6, and PIR. I like to think I'd have listened if someone suggested infelicities in its original design, but only the experience of implementing and using it helped me to understand what it really needed to do.

I wonder. Is it possible to teach Perl 5 in such a way that novices don't have to suffer all of the miscues that the rest of us have in the past decade? If so, how do I do that? (If not, what does that imply about new adoption of Perl 5 and adoption of Perl 6 in general?)

Maybe I'm conflating learning a language with learning how to design programs—but how do you separate the two when your goal is merely to teach people how to solve problems?

The fundamental assumption of the CPAN that makes managing software through CPAN clients difficult is that newer software can always replace older software.

That's not the CPAN's problem, not really. Perl 5 has this version number problem too, and it's the source of much of the backwards compatibility troubles in the Perl 5 world.

On the CPAN, no one agrees on what version numbers mean. No one agrees on backwards compatibility concerns. No one agrees about what makes one release "stable" and another release "testing" or what you can change in an API.

I'm not sure it's the job of CPAN or the CPAN maintainers to try to solve this problem. The lack of rules and ceremony over the function of uploaded code and the loose agreement on the form of uploaded code have allowed CPAN to thrive where other languages with their own packaging and distribution systems have fragmented into incompatible, warring fiefdoms.

Even still, fighting over whether Module::Build is of the devil because (insert silly reason here) or complaining that Module::Install is a stupid patch over a pile of nasty, almost unsupported hacks misses the real point. The best possible improvement in the Perl 5/CPAN ecosystem is to know which versions of which distributions work together, and for end users to be able to download a known-working graph of their desired distribution and its dependencies.

In other words, the big problem isn't that Module::Install has a nicer interface (arguable) or ExtUtils::MakeMaker is a horrific pile of barely-working hacks that shouldn't have survived into the 21st century (indisputable). The problem is the integration of separate components managed by separate maintainers with wildly disparate ideas of the semantics of change over indeterminate periods of time.

At any single given point in time, any given CPAN distribution should work correctly with its currently available upstream dependencies. With the current system, we cannot guarantee that perturbation in those dependency graphs will percolate to users at any point in the future.

In my mind, arguing over whether an improvement to the CPAN ecosystem can take over for every use case of the current tools is stupid and wrong (or, more charitably, a silly distraction). Improving the user and developer experiences of the current tools and the current system is good, but the real improvements in the system will not occur at the interface level.

(Of course, the problem with Adam's point of view is that requiring any credible replacement to handle every possible failure case before stamping it with an imprimatur is that "better" is not an all or nothing concern. I have no illusion that it's possible to fix the Perl 5 versioning scheme for every case. If we can fix it for 90% of cases, surely that's a reasonable improvement.)

From the Terminal window, paste this in at the prompt:

perl -le'for (@ARGV){m{/(\w+?)\.SpeechVoice};' \
  -e'$_=$1;s/([a-z])([A-Z])/$1 $2/g;' \
  -e'print qq{say -v "$_" This is $_};}' \
  /System/Library/Speech/Voices/* | sh -x

On my Mac, "Organ" isn't found, but I don't know why.

• What's going on with San Francisco Perl Mongers? (blogs.perl.org)
• Contributing is easy: A first-timer talks about adding to CPAN (blogs.perl.org)
• Help keep the world safe from SQL injection: (perlbuzz.com)
• How to learn how to get help in Perl (use.perl.org)
• The Perl Foundation wants your input on grant proposals for 2010 (news.perlfoundation.org)
• Why Ruby is prettier and Padre changes the Perl community (use.perl.org)
• Manage your Perl modules with git (effectiveperlprogramming.com)
• Call for papers for YAPC::NA 2010 in Columbus. Theme: Modern Perl 5 (news.perlfoundation.org)
• Famous Perl One-Liners Explained part 5: Text conversion and substitution (catonmat.net)
• The Italians like ack, too (cattlegrid.info)
• Bash completion for perldoc (blogs.perl.org)
• cpanminus, the new CPAN shell superstar (marcus.nordaaker.com)
• RT @chromatic_x Kudos to *many* active #perl 5 porters for the successful 5.11 release series: (ur1.ca)
• Testing a DB-intensive app (blog.urth.org)
• More news on the TPF wiki overhaul (use.perl.org)
• Do you still program in Perl? (reddit.com)
• Benchmarking is for losers, Profiling rulez! (blog.urth.org)
• Damian Conway's been posting a series of columns on vim: (blogs.perl.org)
• Testing and looging with Dist::Zilla (rjbs.manxome.org)
• Safe.pm 2.25 fixes security hole (blogs.perl.org)
• Serving production web requests with the Catalyst development server (blog.afoolishmanifesto.com)

I should note that the original deadline for finishing CT2.0 was March 1 and clearly we’re not there yet. I’ve discussed the situation with Robert and Ask at the Perl NOC, and they’ve been willing to extend the deadline for cutting off CT1.0 for a while longer. Thank you, Robert and Ask, for your understanding!

Progress in the last couple weeks:

• I did some alpha testing of the CT2.0 Metabase hosted on Amazon. Based on that, I revised yet again (sigh) the user registration/credentials approach to minimize the hassle for old and new registrants.
• Florian Ragwitz wrote a Catalyst app to help distribute new credentials files to legacy CT1.0 users. I haven’t deployed it yet (since I now need to regenerate all the credentials), but greatly appreciate his quick turnaround.
• I implemented the Metabase search capabilities that Barbie will need to update the CPAN Testers statistics database. This will be based on the excellent SQL::Abstract approach to WHERE clause construction.
• I wrote several helper modules to simplify configuration of a CPAN Testers metabase in preparation for deployment. The first of these has already been uploaded to CPAN: Net::Amazon::Config
• I finalized the “version 0″ API for the Metabase web service and revised the interface between the Catalyst app and the Metabase to reflect the latest changes to the library.

In the last several weeks, members the #catalyst and #moose IRC channels were very, very helpful and patiently answered my many stupid questions. Thank you in particular to confound, perigrin, rafl, rjbs, stevan and t0m.

Coming up:

• Deploy all my new code onto the server for “beta” testing
• Release all the code to CPAN that people will need to configure their clients for beta testing
• Regenerate user profiles and deploy rafl’s app to distribute them to legacy users
• Whip the server into production shape (e.g. proper boot scripts to auto-start the CT2.0 apps on restart)
• Get back to work on legacy report migration

It’s at the point now where I suspect the “hard thinking” part is pretty much done and it’s a lot of grotty but straightforward tasks to go. Hopefully, beta testing won’t reveal any major issues and the end of March update will be focused on planning on orderly transition from CT1.0 to CT2.0.

Unfortunately, it didn't help—at least, not entirely. According to the completely unscientific process by which I install CPAN distributions, Module::Install accounts for a greater amount of pain than it should, at least according to its frequency of use. (Again, this is completely unscientific. I could guess that half of the CPAN client sessions which encounter Module::Install require me to fix things manually, but it's probably closer to 20%. It's more memorable because of my severe dislike for M::I prompting to install dependencies during configuration time.)

M::I addresses a real bootstrapping problem. I want to be able to use libraries during configuration, building, testing, and installation. I don't know which versions of those libraries you have available. Bundling known-good versions of those libraries with the distribution itself solves part of that problem...

... except when it doesn't. If I were to use M::I, I would have to re-release all of my distributions for every new release of every bundled library, at least if they contain important bug fixes for the various platforms about which I care. The cheap perfume of static linking leaves its musk heavy in the air.

It's easy to fall into the trap of a false dilemma. "You fool!" you prepare to comment below. "It's either that or the chaos of trying to make do with whatever version of those dependencies users may or may not have installed on their systems!" You're right; those are two possibilities. They're not the only two possibilities.

Part of the real problem is that bootstrapping during configuration is much too late. By the time you're running the configuration system, you're already running the configuration system. If your version of the configuration system is too old or too new, you have a problem. Bail out? Revert? Upgrade? There's no good heuristic for determining this. (The CPAN itself has an opinion. That's part of the problem.)

M::I hackers do deserve credit for helping to develop the META.yml standard. (I think M::I is the wrong approach, but I intend no slight toward its users, advocates, and developers. Invention requires the courage to get things wrong sometimes, even as it requires the courage to abandon false leads.) The META.yml specification is a big step in the right direction. If most CPAN modules have static requirements and follow a standard set of conventions, there's little or no configuration necessary. A sufficiently smart CPAN client can perform the appropriate configuration without running code from the distribution itself.

You can't avoid that in all cases; distributions with XS components, for example, need to probe system information. Good luck writing a sufficiently smart CPAN client and getting the community to agree on specific standards that let you find OpenGL headers in a cross-platform fashion, for example. Yet if 80% of CPAN distributions can get by with static, upload-time configuration, a lot of complexity of installation can go away.

Yes, that would make Module::Build and ExtUtils::MakeMaker unnecessary for (probably) most CPAN distributions, at least at the point of configuration, building, and installation. (I'm a recent fan of Dist::Zilla for automating away tedium on behalf of distribution maintainers; there's less need for Module::Install in such a world. If I never write another Build.PL again, so much the better.)

That helps, but the real problem with CPAN installations is that the CPAN itself is merely an uploading, indexing, and mirroring system. Projects such as META.yml attempt to add (and extract) meaning from the system, but they cannot work around one fundamental design feature of the CPAN. That limitation is the source of most woes for end users.

Clever readers (or experienced CPAN users) have already identified this limitation. I'll reveal it in the next installment.

Don't overlook something else insightful that David wrote:

It's a lot of work to be all things to all people and I keep wondering whether making things simpler and better for 99% of people would be a better choice.

The only reliable way I've ever seen to "make the easy things easy and the hard things possible" is to make the easy things the default without preventing customization of the hard things. That's a design principle for languages, APIs, and tools.

Figure out what's most common (though not necessarily what people think they want, but what they need). Optimize for that. Consider what they might need and don't prevent it.

That's not easy, and what people need will change over time, but if you want to solve problems well, you have to solve the right problems.

It’s a remarkable achievement, not only technically, but in the reaction it has sparked. As one of the (sometimes reluctant) maintainers of CPAN and CPANPLUS, I’ve realized that I both love and hate cpanminus.

• I love that Miyagawa has done so much with so little and in such a short span of time
• I hate that fanboy-types flocked to it and trashed the older clients without noting cpanminus’ limitations
• I love that Perl toolchain maintainers have rallied around Miyagawa and contributed their wisdom to make cpanminus better instead of rejecting it
• I hate that one of Perl’s great strengths (CPAN) has legacy clients that are so unwieldy, hated and difficult to maintain

Miyagawa graciously acknowledges standing on the shoulders of giants. Still, I can’t shake the nagging thought that cpanminus should never have been necessary in the first place.

What I’ve come to realize is that cpanminus is another example of the power of not being all things to all people. Miyagawa doesn’t promise that it works for all of CPAN or that it works everywhere that Perl does. He doesn’t have to. Making it work for 99% of CPAN for 99% of people is more than good enough.

I’ve been co-maintaining various parts of the Perl toolchain for a while now. It’s a frustrating challenge needing to make thing work everywhere, for everything, and trying as hard as possible not to break backwards compatibility. Plus, I don’t even get to use CPAN to make life easier. I don’t get to use handy tools like Moose or DateTime or Regexp::Common or SQLite or anything in the Config::* namespace or even basic tools like Archive::Zip. Nearly everything is done by hand.

Things have to work with just core Perl on a diverse set of platforms and with an incredibly limited set of assumptions. For example, the Perl core still doesn’t come with an HTTP client, so CPAN has to rely on FTP or command line programs to bootstrap LWP. (This is something I personally plan to tackle during the Perl 5.13 development series later this year.)

I think this is an ongoing challenge for core Perl development in general. It’s a lot of work to be all things to all people and I keep wondering whether making things simpler and better for 99% of people would be a better choice. (Anyone else for use strict by default? I hope that finally comes to pass in Perl 5.14.) chromatic writes about this topic often in his Modern Perl blog and I usually tend to agree with the points he makes. (October February 2009 had a particularly good series of posts.)

In the meantime, I look at cpanminus with greed and envy. Miyagawa++

I don't believe people who use Module::Install should be shot on sight, but I do believe that Module::Install has set the usability of the CPAN back by several years.

Ingy did identify a real problem: there's too much code strewn about the configuration and build systems of the CPAN and not enough code shared. When he found himself writing something complicated to configure, compile, and install a new distribution, he'd crib code from someone like Tim Bunce or Graham Barr or Nick Ing-Simmons. In other words, to make the CPAN—perhaps the world's largest repository of redistributable and sharable library code—work, he had to copy and paste code.

M::I did get that right; turning the configuration and build system into reusable, redistributable libraries also available from the CPAN helped reduce the amount of boilerplate code and the amount of copy and paste code in configuration systems. The people behind M::I have also helped push for better configuration of CPAN clients and better tracking of dependencies and versions and types of dependencies (optional, compilation, bundling, testing, et cetera). The CPAN ecosystem is better off for that work, even though M::I itself isn't the answer.

One of M::I's biggest failings, of course, is that it prolongs the lifespan of EU::MM. Unfortunately, I think Ingy missed the big problem when he saw copy and paste code. To do anything reasonably complex with EU::MM, you have to be able to write Perl 5 code which generates (and modifies with regular expressions) cross-platform Makefiles which themselves call into Perl 5 code because Perl 5 has a sane baseline of well-understood and reliable behavior across platforms in a way that Makefiles and shells do not.

If that's not sufficient horror, consider that the way to customize EU::MM behavior the last time I looked at it (Two notes here. First, its current maintainer refuses to add new features or change existing features because it's so awful to maintain. Second, I wrote tests for some of those behaviors, so I've read and understood the code.) you write a custom superclass called MY from which EU::MM inherits to change the behavior of the various steps of generating cross-platform Makefiles which may or may not invoke Perl 5 to perform shell functions.

I am not making this up and I did not make any typos. EU:MM inherits from your custom class.

Keeping EU::MM viable long past the point where Module::Build did everything n a saner way is but a little crime. (There's a reason almost no one recommends the use of h2xs to make skeletons for new modules anymore; most new modules aren't mere wrappers around C libraries. The need for a compilation step with a pure-Perl distribution seems more than a little bit superfluous.)

Module::Install reinvoking your current CPAN client recursively to install dependencies when your current CPAN client already has a perfectly good way to install dependencies is a slightly larger crime. (I understand the justification; what if someone is trying to install a distribution from a tarball manually without a CPAN client, but is there a Principle of Least Possible Differentiation at work with that design choice?)

I don't particularly care that using M::I as a distribution maintainer means that you have to keep track of every new release of M::I which could fix bugs or make upgrading impossible and release a new version of every one of your distributions with the new M::I because of its autobundling problems, because if you get your kicks that way, more power to you. (Don't expect me to jump up and down for joy at upgrading all of your distributions on my machines, though.)

I never particularly cared for the FUD about Module::Build from some M::I discussions, especially the nonsense about "Module::Build doesn't support --install so it spews files all over the place!" (as if EU::MM ever worked properly there) or "Module::Build doesn't support uninstalling!" (as if anyone ever used that from the CPAN client to know that it worked with any system.)

I appreciate that Module::Install provided a much nicer interface than EU::MM did, and that that interface worked transparently to hide the nasty details of EU::MM. Those are true benefits, and I don't blame anyone for choosing M::I for that reason.

Even so, Module::Install's greatest crime is that it's been a distraction from identifying and fixing the real problems of the CPAN... but that's another post.

Thus any support document must explain the responsibilities of users who encounter bugs and what they should expect from developers.

A bug you discover in a new major release of Perl 5 is a candidate for a new minor release if it is:

• A security or dataloss bug
• A regression introduced in the new major release
• A failure to build on a supported platform combination
• A failing core test on a supported platform

I've organized that list in rough order of decreasing severity. The most likely candidate for a fix is the second; it indicates an undertested aspect of the system. Behavior should not change between Perl 5 releases accidentally. If a patch modified behavior on which your code depends and if that change did not occur as part of a deliberate, communicated plan, a fix is likely.

Of course, any fix in a minor release needs to maintain binary compatibility within the release family.

The easiest way (at least for developers) to find and fix such bugs is before the release of a new major version of Perl 5. That's one goal of the monthly releases: to encourage you to test all of the code you care about with versions of Perl 5 in development. That's also one reason for the Perl 5 release candidates (though it's likely too late for big fixes by then).

If you can't do that, the next step to reporting a bug is to reproduce it in the smallest example possible. 10-15 lines of Perl 5 is good. 2-3 is ideal. More than 20 is usually too big. If you can provide a test case suitable for adding to the core test suite, so much the better. From there, test your code with multiple releases of Perl 5. It helps to browse the perldelta documentation, but the amount of detail between even minor releases can be daunting. A post on PerlMonks is a good step.

If you've gone through all of that, see perldoc perlbug.

This is all no guarantee that your bug will get fixed in a minor version—you should prepare for the possibility that, given enough time since the release of the corresponding major version, the best approach for p5p is not to backport a fix to a new minor version. Even so, you will likely get one of several options:

• An explanation of why it's not a bug
• A suggested workaround
• A fix in the current version

In the latter case, you will have the option of applying the relevant patch yourself or asking someone to backport it to your own custom Perl 5 if you wish. That may not seem like the ideal situation (it isn't!), but at least with free software such as Perl, you always have that option.