Using modern Perl 5 for one portion of the project does not necessarily require the use of Perl 5 for all of the project.

I've long been an advocate of Perl 6 the language, so is it worth using Perl 6 for the other part of the project? The big contender is obviously Rakudo, because it is the most active project and runs on a free software stack I can debug myself. (The other possible candidate implementtation, Niecza, runs on Mono, which disqualifies it from my considerations. Your personal technology choices may not be mine. I have no interest in discussing Mono here.)

I posted the other day my list of requirements to use Rakudo for practical purposes. What do I need technically?

This little app must:

• Communicate with a database. I don't necessarily need DBIx::Class for a project of this size, but I do need the ability to work with SQLite now and PostgreSQL later. Without DBIC, I'd have to do more work mapping tables and rows to objects, but the schema is simple enough it's not too onerous.
• Communicate with websites. I do need the equivalent of LWP. While I do use at least one specialty web service consumer module from the CPAN, I could as easily perform HTML scraping.
• Manage regular expressions, not grammars. Part of the project requires HTML scraping. If I were prone to overengineering, I'd write a full Perl 6 grammar for this. As it is, some quick and dirty data munging is more than sufficient right now for this proof of concept.
• Work with a decent templating system. I know there's no Template Toolkit system for Perl 6 right now, but I don't use all of TT's power. I need only a few features: variable interpolation, loops, conditionals, and subtemplates. If I had to reinvent a template system or work around it I could, but why would I want to? (I am in no mood to reinvent HTML escaping or to correct UTF-8 encoding either.)
• Work with dates and times. I use Perl 5's DateTime. Never again will I reinvent date or time handling. Never.
• Deploy easily. I use Dist::Zilla to manage bundling and testing. The simpler the work to deploy new versions and corrections, the more frequently I will do it. Again, Dzil does far more than I need, but what it needs I hate to give up.

Speed doesn't really matter for this application. It's a batch process which runs once a day and takes 10 seconds in Perl 5, mostly because I haven't done any work to exploit the embarrassingly parallel nature of the processing. If Rakudo ran it in ten minutes, that would be fine. (With that said, the data set will probably eventually scale by two orders of magnitude, so improving the Perl 5 version's parallelism is useful but would likely help a Rakudo version less.)

Memory use does matter, because I've deployed this application to a shared server and want to be a good citizen.

The stability of Rakudo as a target platform also matters, because I've set up this program to run without manual intervention for days, weeks, and even months. I understand that one of Rakudo's goals is to produce new and better versions every month or so, and I understand that me babysitting a Perl 6 version of this little app would provide valuable feedback to Rakudo developers...

... but I can't justify turning a fire-and-forget tiny side project into the tip of a spear intended to open the tent for further, larger projects.

I could work around the lack of most of the few modules I need. (Database access is the only real sticking point.) Yet even for a small side project—a toy project, even—with minimal needs, I find myself not wanting to use Rakudo because I'd rather spend my small side project time working on the small side project and not trying to get the technology stack beneath that small side project to stay put.

The conventional wisdom in #perl6 seems to be that Perl 6 needs more people playing with it to find bugs and more people building its equivalent of CPAN to attract more people to play with it. In my experience, Perl 6 needs something at a lower level: Perl 6 needs an implementation which provides what Rakudo Star was supposed to be. A year after the release of the first Rakudo Star, Rakudo isn't it, at least for me.

Maybe your project is different, and maybe your needs are different from mine. All I know is that while I'd liked to have used Perl 6 for this project, it didn't work out, and that's disappointing.

• The RSS feed now updates correctly.
• No more crashing the website for unknown reasons.

I'd tried to figure out the core problems behind these, but could never track them down. Thanks to Patrick for converting over to a new system and taking over maintenance of the blog!

• PPM Index: A new way to browse Perl packages (activestate.com)
• perl.org is now hosting the master CPAN mirror (log.perl.org)
• DTrace and Perl (dgl.cx)
• Why aren't there more women in open source? Most of the same reasons they aren't contributing to Wikipedia. (nytimes.com)
• Language trends as reflected in paper book sales (radar.oreilly.com)
• Unifying the two worlds of Perl 5 (modernperlbooks.com)
• The death of tribal knowledge, and why it matters to Perl (modernperlbooks.com)
• How to replicate a CPAN module failure (dagolden.com)
• Dealing with legacy Perl code: A free webcast (oreillynet.com)
• Introduction to Smart::Comments and Devel::Cover (perltraining.com.au)
• Interview with me about ack (blueparen.com)
• Why @chromatic_x's Modern Perl teaches OO with Moose (modernperlbooks.com)
• Call for venue for YAPC::Europe::2012 (yapceurope.org)
• What should be core in Perl 5.16? (blogs.perl.org)
• Facebook authentication with Perl (perl.com)
• Perlnews.org lauched by Leo Lapworth and Dave Cross. (perlnews.org)
• Perl to the rescue: case study of deleting a large directory (blogs.perl.org)
• Why PSGI/Plack matters (modernperlbooks.com)
• Big CPAN.org update (log.perl.org)
• Planet Perl is retiring. (perlnews.org)
• Reviewing "Perl Best Practices" re: objects (blog.urth.org)
• search.cpan.org has syntax highlighting (blogs.perl.org)
• Parrot Foundation on Google Code-In (google-opensource.blogspot.com)
• Python for Perl programmers (everythingsysadmin.com)
• Sign up to help Perl in Google Summer of Code now (news.perlfoundation.org)

Instead, here are the three most important stories in Perl from 2010, from my perspective:

• Perl 5.12 released on time, with little fuss or mess. The change to monthly development releases and yearly stable releases settled down to a well-understood project heartbeat. Perl 5.14 is a few months away, which should allow the graceful ending of support for all Perl 5 versions older than 5.10.1. Welcome to the age of modern Perl, everyone.
• Moose reached version 1.0. After several years of rapid development, all of the pieces were in place to replace Perl 5's default object system with something more powerful, more concise, more correct, and easier to use and to explain. Moose isn't quite the de facto object system in Perl 5 yet, but with ancillary projects adopting Moose-compatibility while providing additional features (quicker startup, less memory usage), it's clearly the right way to write modern OO Perl 5.
• The first Rakudo Perl 6 Star release came out. While various Perl 6 implementations had been available since 2003, Rakudo Star was the first distribution release intended for end users to play with. Development continues on the Perl 6 specification and Rakudo, but the monthly releases of Rakudo demonstrate that Perl 6 is useful and usable today and improving regularly.

There are plenty more great projects to mention—cpanminus, perlbrew, plack, Catamoose, CPAN testers upgrades, and a couple of good books—but these three are the most important in terms of the long term health and vitality of Perl. With them, the Perl community has a firm foundation for a great 2011.

• Use when() as a statement modifier (effectiveperlprogramming.com)
• What's coming in Perl 5.14? (effectiveperlprogramming.com)
• given/when is the Perl switch statement (transfixedbutnotdead.com)
• Holy cow is the new @github pull request feature slick (github.com)
• What's going right in Perl (modernperlbooks.com)
• Take a look at the speed improvements in Rakudo Star (gil.di.uminho.pt)
• swaks, an SMTP transaction tester in Perl (blogs.perl.org)
• Ten years of Birmingham Perl Mongers (blogs.perl.org)
• use.perl.org shutting down indefinitely (use.perl.org)
• It's the Perl event of the summer! The first Perl mongers meeting to have its own trailer (blogs.perl.org)
• Extracting bullet lists from PowerPoint with Perl and Win32::OLE (blog.nu42.com)
• Perl 5.12.2 released (search.cpan.org)
• I &heart; the Perl Community (blog.fox.geek.nz)
• HTML::Tidy 1.54 is released, which should make building easier. (perlbuzz.com)
• Perl is my community. (blogs.perl.org)
• Less than three weeks until Pittsburgh Perl Workshop 2010 (pghpw.org)
• Try Rakudo Perl 6 in your browser. (try.rakudo.org)
• Company behind only real Perl blogging platform seems to be veering away from blogging. (mashable.com)
• Why not just help the guy out, and allow him to progress faster to the more interesting parts of Perl? (bit.ly)
• A comparison of Perl serialization modules (blogs.perl.org)
• Job boards no more "help" you find a job than a billboard "helps" you find a new pair of shoes. (bit.ly)
• A few things every job-seeking programmer should know about project manager (stellman-greene.com)
• Book review: "Effective Perl Programming" (books.dzone.com)
• I love that the MySQL conference wants PostgreSQL content. Would love to see YAPC do the same, maybe some Ruby talks. (chesnok.com)
• ARP poisoning with Perl (sharpsproductions.com)
• Set default values with the defined-or operator. (effectiveperlprogramming.com)
• Google Open Source and a recap of Summer of Code (google-opensource.blogspot.com)
• Colonizing the Lacuna Expanse with Perl (perl.com)
• If you ever need a regex, look first at Regexp::Common. It's like a CPAN for regexes! (search.cpan.org)

Key points from his article:

• Rakudo Star is the first prototype of an end-user distribution tarball
• There is no way reactions to Rakudo Star can possibly live up to the hopes and dreams of those involved the project (and I say that it can't live up to the expectations of those outside the project, either)
• Rakudo Star still is a significant step forward for Perl 6
• If anyone was waiting for Perl 6 to rescue Perl, then they'll need to keep waiting.

And then, down in the comments, Moritz Lenz points out something I'd overlooked:

Yes, people now say "Rakudo is slow as $funny_metaphor", but that's much better than "Rakudo is vapourware". Once we speed up Rakudo, we can simply post benchmarks and say "look, it's now $n times faster than before". "Rakudo is slow" implies "Rakudo is", and that's a big step forward.

Thanks for the reminder, Moritz. Next it will be "Rakudo is faster, but not fast enough." And then maybe "Rakudo doesn't have enough documentation," and then "Rakudo doesn't have all the modules it should." All of it is progress.

• Running tests in MySQL with Perl via MyTAP (justatheory.com)
• chromatic rebuts the FUD of startup times for Rakudo Star (modernperlbooks.com)
• Top 10 tricks of Perl one-liners (blog.ksplice.com)
• A checklist for writing maintainable Perl (modernperlbooks.com)
• A gentle introduction to Perl 6 with Rakudo Star (blogs.perl.org)
• Opening Perl modules with vim (blogs.perl.org)
• Your Perl test suite is broken (blogs.perl.org)
• Diving into Perl 6 (perlbuzz.com)
• What's up with Proud To Use Perl (http://proudtouseperl.com/) Are people no longer proud?
• Why are Perl web apps less popular than those in PHP? (blogs.perl.org)
• Perl 5 and Perl 6 are different languages, and that's a good thing (use.perl.org)
• Perl 6 is easier to read than write (blogs.perl.org)
• The official Perl 6 cheat sheet (svn.pugscode.org)
• Currying with Perl 6 (perlgeek.de)
• Try Rakudo and learn Perl 6 in your browser: (try.rakudo.org)
• Perl 6 has data dumping baked right in (perlbuzz.com)
• What to respond to "Perl 6 isn't Perl any more" (perlbuzz.com)
• How to write about Perl 6 (blogs.perl.org)
• When should I use database abstraction? (blogs.perl.org)
• Videos from YAPC::NA (presentingperl.org)
• What I learned at YAPC::EU 2010 (blogs.perl.org)
• Where to now in Rakudo Perl 6 development? How to speed up Rakudo? (perlgeek.de)
• Colorized code snippets on an ANSI terminal (blogs.perl.org)

On behalf of the Rakudo and Perl 6 development teams, I'm happy to announce the July 2010 release of "Rakudo Star", a useful and usable distribution of Perl 6. The tarball for the July 2010 release is available from http://github.com/rakudo/star/downloads.

Rakudo Star is aimed at "early adopters" of Perl 6. We know that it still has some bugs, it is far slower than it ought to be, and there are some advanced pieces of the Perl 6 language specification that aren't implemented yet. But Rakudo Perl 6 in its current form is also proving to be viable (and fun) for developing applications and exploring a great new language. These "Star" releases are intended to make Perl 6 more widely available to programmers, grow the Perl 6 codebase, and gain additional end-user feedback about the Perl 6 language and Rakudo's implementation of it.

In the Perl 6 world, we make a distinction between the language ("Perl 6") and specific implementations of the language such as "Rakudo Perl". "Rakudo Star" is a distribution that includes release #31 of the Rakudo Perl 6 "compiler, version 2.6.0 of the Parrot Virtual Machine, and various modules, documentation, and other resources collected from the Perl 6 community. We plan to make Rakudo Star releases on a monthly schedule, with occasional special releases in response to important bugfixes or changes.

Some of the many cool Perl 6 features that are available in this release of Rakudo Star:

• Perl 6 grammars and regexes
• formal parameter lists and signatures
• metaoperators
• gradual typing
• a powerful object model, including roles and classes
• lazy list evaluation
• multiple dispatch
• smart matching
• junctions and autothreading
• operator overloading (limited forms for now)
• introspection
• currying
• a rich library of builtin operators, functions, and types
• an interactive read-evaluation-print loop
• Unicode at the codepoint level
• resumable exceptions

There are some key features of Perl 6 that Rakudo Star does not yet handle appropriately, although they will appear in upcoming releases. Thus, we do not consider Rakudo Star to be a "Perl 6.0.0" or "1.0" release.

In many places we've tried to make Rakudo smart enough to inform the programmer that a given feature isn't implemented, but there are many that we've missed. Bug reports about missing and broken features are welcomed.

See http://perl6.org/ for links to much more information about Perl 6, including documentation, example code, tutorials, reference materials, specification documents, and other supporting resources. Rakudo Star also contains a draft of a Perl 6 book -- see in the release tarball.

The development team thanks all of the contributors and sponsors for making Rakudo Star possible. If you would like to contribute, see http://rakudo.org/how-to-help, ask on the perl6-compiler@perl.org mailing list, or join us on IRC #perl6 on freenode.

Rakudo Star releases are created on a monthly cycle or as needed in response to important bug fixes or improvements. The next planned release of Rakudo Star will be on August 24, 2010.

Subsequent releases will bring improvements on completeness, correctness, and competitiveness.

Even so, how unfortunate to see a thoroughly misleading comparison of Rakudo Star to Perl 5.

Rakudo's perl6 binary is a couple of hundreds of lines of C code (if that much) linking to libparrot and mostly unoptimized Parrot bytecode. By unoptimized, I mean "There is no optimization of Parrot bytecode in Rakudo Star." — not constant folding, not size optimization, not inlining. None.

The perl5 binary is of course all compiled C code generated (hopefully) by an optimized compiler. Does that make for a valid comparison? I think not.

Of course, the worst flaw by far is in comparing the feature set of the Perl 5 binary to that of Rakudo Star. Perl 5 contains no advanced object system, no runtime manipulation of the parse tree, no grammars, no continuations or coroutines, no junctions, and no interoperability with other languages. Oh, and it does not contain hyperoperators, function and method signatures, multidispatch, laziness, or built-in language redefinition, no strictness by default, no autothreading, no REPL, and no automatic binding to shared libraries.

It's easier to add the loading of the appropriate CPAN modules to the Perl 5 process than to remove them from Rakudo Star, so for an accurate comparison, you must add:

I ordered the list in terms of dependencies to show cumulative costs. I also left out a few CPAN distributions you need to add to Perl 5 to add features enabled by default in Perl 6: for example, I couldn't get Data::Alias to build.

My perl5 binary is Perl 5.12.1, built fresh today as a 32-bit binary without threading. (A 64-bit binary uses more memory and a binary with threading is some 10-15% slower, reportedly.) It's 1,239,686 bytes in size.

My perl6 binary (built from today's checkout, not Rakudo Star) is 14,280,276 bytes in size. At the REPL, it uses 99,944 KB of virtual memory and 81,132 KB of resident memory. To run the program -e 1, it requires 1.035s. By my calculations, Rakudo Star starts up 1.5% more slowly than Perl 5 with all of the other modules loaded and uses 2.45 times as much virtual memory and 4.39 as much resident memory.

By way of comparison, Perl 5 running -Mperl5i::2 -E 1 runs in 0.149s. It requires 12,080 KB of virtual memory and 6,952 KB of resident memory. Rakudo Star starts 6.72 times slower than Perl 5 with perl5i. Rakudo Star uses 8.27 times more virtual memory and 11.67 times more resident memory.

I believe it's very possible to improve Rakudo's startup time by an order of magnitude, such that it starts faster than Perl 5 with perl5i, much faster than Perl 5 with all of the feature parity modules loaded, and modestly slower than Perl 5 without feature parity. I also believe that it's possible to improve Rakudo's memory requirements—perhaps not an order of magnitude, but at least by cutting them in half and then perhaps in half again.

Better yet, new features planned for Parrot offer the opportunity to deploy applications with only those features actually used, so if you avoid the object system or grammar manipulation altogether, you won't pay the memory or startup costs for them. That's a plan for sometime in the future, however.

Rakudo Star has a long way to go. We'll improve its performance and cut its memory requirements. There are many places we know need improvement, and we're working on them. That's why we keep releasing new versions.

... but don't fool yourself: comparing bare Perl 5 to Rakudo Star without understanding the context of the comparison does no one any good.

The Perl ecosystem has room for other projects, however.

For example, one of the clearest benefits Perl 6 has over Perl 5 is its portability to other virtual machines and runtimes. By design Perl 6 encourages multiple implementations. Perl 5 is its own specification; in many places, what Perl 5 is is solely what Perl 5 happens to do. Sometimes that behavior gets enshrined in the specification tests, but other times it's folklore and institutional community knowledge.

Just as Parrot's Lorito project intends to make Parrot at least an order of magnitude faster, so too a reorganization of Perl 5 internals could make amazing things more possible.

What if there were a project to implement a minimal set of Perl 5 on the Parrot virtual machine as a prototype and exploration of how much of Perl 5 you can support, the effort it takes to do so, and what kind of utility you can expect? Parrot's compiler tools let the Rakudo developers write most of Perl 6 in Perl 6; surely it's possible to write Perl 5 in a similar fashion. (Credit to other projects such as Rubinius and PyPy for demonstrating that such things are possible.)

I know other projects have attempted this in the past. Perhaps the best place to steal information is Bradley Kuhn's masters thesis, Considerations on Porting Perl to the Java Virtual Machine.

As Jesse wrote in his comments, bug-for-bug compatibility isn't necessary. Nor is full compliance with the existing Perl 5 test suite. A simple proof of concept to produce the 80% of Perl 5 most people use in most programs should suffice. (Parrot gives you a lot of that anyway.)

As a bonus, you get cheap and easy interoperability with Perl 5, access to Parrot features such as multidispatch, grammars, continuations, and bytecode serialization, and you could even replace some of the uses of Perl 5 within Parrot's and perhaps even Rakudo's configuration and build processes.

It doesn't even have to be a pony of full size.