• Catalyst chained dispatching is great
• Dave Rolsky is a fantastic, underpraised hacker, a great guy, and one of the first people to say that an interface or implementation could use improvement
• Martin Fowler is smart and a deep thinker
• I like MooseX::Declare and its ilk
• I don't object to what these systems do; I question how they do it
• If it makes sense for you—if you've thought about the benefits and drawbacks and the benefits outweigh the drawbacks and costs—use it with no quibbles or qualms on my part. I do sometimes.

Your embedded so-called "domain specific language" is only pretty if your language's liberation army has kidnapped and brainwashed you so long ago that your notion of beauty and concision is horribly warped.

To wit:

  use CatalystX::Routes;

  BEGIN { extends 'Catalyst::Controller'; }

  # /user/:user_id

  chain_point '_set_user'
      => chained '/'
      => path_part 'user'
      => capture_args 1
      => sub {
          my $self = shift;
          my $c    = shift;
          my $user_id = shift;

          $c->stash()->{user} = ...;
      };

... is not particularly pretty. It may make the intent of the declaration of routing clearer than the alternative:

sub _set_user :Chained('/') :PathPart('user') :CaptureArgs(1)
{
    my $self = shift;
    my $c    = shift;
    my $user_id = shift;

    $c->stash()->{user} = ...;
}

... but suggesting that it's pretty goes a bit far. I don't have many better syntaxes in mind, but if I could remove the inline => sub { ... } and the need for fat arrows everywhere, I would. Something like this might be more aesthetically pleasing:

chained:      /
path_part:    user
capture_args: 1
method _set_user($c, $user_id)
{
    ...
}

... although it might be fun to write:

method _set_user($c, $user_id)
as user, chained to /, capturing one arg
{
    ...
}

... if route declarations must be in the same file or attached to the same method as declarations. Then again, the Catalyst debugging syntax is awfully clear:

//*

Perl 5 is hardly the only offender in the syntax department though. Here's a so-called embedded DSL in Python 2:

with create_table("Employee") as t:
    t.first_name = {"type" : "char", "length" : 30 }
    t.last_name  = {"type" : "char", "length" : 30 }
    t.age        = {"type" : "int"}

To whom is this nicer than SQL? (You'd almost think someone had already created a language specific to the domain of describing relational data structures.)

Somehow Ruby fans fail to notice the baffling need to sprinkle syntax all over their tooth-decaying sugar, as in Cucumber, where you get the illusion of writing test specifications in plain English as well as the absolute joy of writing and maintaining regular expressions to extract the necessary data from them.

That's right, the important data structure in your test suite is semi-structured text:

Feature: Addition
  In order to avoid silly mistakes
  As a math idiot
  I want to be told the sum of two numbers

  Scenario Outline: Add two numbers
    Given I have entered <input_1> into the calculator
    And I have entered <input_2> into the calculator
    When I press add
    Then the result should be <output> on the screen

  Examples:
    | input_1 | input_2 | output |
    | 20      | 30      | 50     |
    | 2       | 5       | 7      |
    | 0       | 40      | 40     |

... and you still have to parse it yourself, in an ad hoc fashion:

Given /I have entered (.*) into the calculator/ do |n|
    calculator = Calculator.new
    calculator.push(n.to_i)
end

There's a cleverness there to be sure, but is that really the best the programming world can provide?

Lisps fare little better, where outside of Dylan, every Lisp program ends up looking like every other Lisp program. At least XSLT had the good grace to reveal that homoiconicity means that ugly languages make for programs that are ugly to the bone.

Designing great languages is difficult. That's one reason why there are so many languages. If the best you can do as the writer of a so-called embedded dynamic language is to find some corner cases in the syntax of your host language then sprinkle regular expressions around, you haven't done much at all. (Tell me a non-programmer is always going to follow Cucumber's "But it's only semi-structured plain text, it's so eeeeeeeeeezy to write!" semi-structured format in a way that your ad hoc regular expressions always match perfectly, and I'll show you non-programmers who write their plain-text behaviors in Microsoft Word.)

Again, if these tools make your life easier, great. I use them sometimes myself. I just don't dislocate my shoulder patting myself on the back for my cleverness in doing so. Maybe some of the energy spent trying to twist a host language into not looking like the host language would be better spent writing simpler, more fluent interfaces in the dominant and effective idioms—if not writing actual new domain-specific languages, with parsers and everything.

Getting an accurate count of the number of distinct modules on the CPAN is useful, but it's insufficient to compare the awesomeness of Perl to the fauxhawked machismo of Ruby to the humorless pseudocode-spewing Cylonification of Python to the cubist performance art of PHP.

Why?

Because what is a module and what does it represent?

In Perl 5, a module is a namespace. It may or may not contain code. It may or may not contain data. It may or may not exist in its own file. It may or may not be part of a bundle of other modules. It may or may not be useful on its own.

Count all of those up on the CPAN and compare them to the count of classes or packages or gems or eggs or copy and paste snippets for other languages, and you can learn... well, not nothing but not much useful.

You may as well compare the utility of (say) code written in Java to the same code written in Forth, despite the fact that Java's capability for abstraction begins and ends with the idea that everything in the world is a class or an instance of a class (except for those nasty primitives, and what are you, some sort of anti-business Smalltalk hippie anyway!) and that Forth's capability for abstraction is really good Forth programmers who create their own little languages that look and act like Forth but better, at least for their specific problem domains.

Oh, and a Perl 5 module may or may not be a class and it may or may not export a procedural interface and it may or may not be primarily documentation.

You may as well compare the goodness of two comparable modules chosen randomly from two languages with regard to use of standard language features, standard library features, and re-use of other modules, because ... well, it all depends, doesn't it? Sometimes the standard library is weak in one area, or one language lacks a feature of abstraction critical to another language, or there's clearly one really good way to do something in one language (Perl's DBI) and someday a standard might evolve and slowly spread through the ecosystem including a morass of hosting providers (PHP's database access layers).

... or what if one language has one more-or-less blessed way of doing something (Python's Django) and a small handful of alternate approaches or one obvious first choice to consider to do things (Perl's Catalyst) and a larger handful of worthy approaches or is in fact itself the primary way people do things (PHP) or overshadows other projects in the language so dramatically (Rails)?

... or what if one repository is primarily applications and one is reusable components, or one is a vast swath of untested abandonware ripped out of three-commit Git projects and another has an annual refresh rate of 30% and millions of test reports every year?

If you still insist on measuring your value as a person by comparing incomparables between your favorite language and the barbarian mutterings of those unwashed Philistines who {like sigils|hate sigils|love blocks|couldn't indent their code with a tab key and a flashlight|smell of body spray|know how to read documentation|haven't upgraded to a newer version yet|deprecated an insecure feature you relied on anyway|do it like Java|don't do it like Java|have a mustache|aren't Danish|aren't Dutch}, you might despair and consider returning to the time-honored practice of counting the number of mentions on the Internet of a random moniker sometimes and somehow related to the name of your languages—but resist that temptation, for That Is Obviously Bad Epistemology.

You might as well count the lines of code written in each language to find the bestest, most gratifyingest language in which only real, true programmers program.

I hope to see you all at Yet Another COBOL Conference 2011.

That's not necessarily the fault of the website, but the graph and the extrapolation on the graph make Ruby's momentum with gems look all but unbeatable. That's wrong too.

CPAN's number (as of today) is 18936 modules, which corresponds closely with the number on www.cpan.org. You can forgive someone not closely associated with the Perl 5 community for thinking that number represents all of CPAN.

In truth, it represents the CPAN modules list—specifically the registered modules list. The important notice on that page says:

This Perl 5 Registered Module List document is currently not being maintained and is several years out of date.

search.cpan.org provides a much better set of numbers: 21585 distributions and 88698 modules. Almost 80% of the modules available on CPAN are not on the registered list.

What's the registered list? Back in the day, when you uploaded a new distribution on the CPAN, community standards suggested that you should choose a location in the module naming hierarchy and that you should describe your upload with a short code of metadata to indicate the support level, the type of interface, and the like. This was/is the DSLIP code.

Some experienced CPAN authors might even remember DSLIP codes; they're used even less often than the initial hierarchy of the CPAN index. Registering modules is much less useful than merely uploading them and letting people search for them by name and description with search.cpan.org.

To summarize, the module count on www.cpan.org represents a fraction of the available modules available on the CPAN because it only counts modules which the uploaders have bothered to register with the registered module list. You can write a lot of great modern Perl code while using unregistered modules. There's no correlation between appearance on the module list and quality or utility, except that most of the registered modules are likely older projects first uploaded when the registered module list and DSLIP codes were used more widely.

Any count of CPAN modules should use the numbers from search.cpan.org instead of cpan.org. I've submitted a pull request to update the sources of the CPAN module counts for modulecounts.com.

I've worked on a few shared library bindings for various dynamic languages: several libraries for Parrot, a few for Perl 5, and one for Ruby. I've embedded Perl 5 and I've embedded Parrot. (I figured out how to get Perl 5's reference counting working correctly with Parrot's "true" GC and how to get Parrot's GC working when embedded in Ruby.)

I even wrote a proof of concept silly port of Parrot's foreign function interface to Perl 5 before the Python folks adopted the much better ctypes (and can't wait to use ctypes for Perl 5).

All of this reveals to me that there's something rotten about writing simple bindings to shared libraries from dynamic languages. It's mostly tedious, uninteresting work with far too many chances of bugs and far too many repetitive details. You'd think computers would be good at solving both problems.

I have generalized from my psyche-scarring experiences two fundamental assumptions:

• C (and specifically C headers) are a terrible layer of interoperability because they cannot express some of the most important details (Does this function acquire a shared resource? Whose responsibility is it to manage the lifespan of that resource?) and they obscure the clarity of intent through the use of abstractions such as C declarations and macros.
• Requiring end users to install a full development environment along with the development headers for any library to which they want to install your bindings is a recipe for madness on the part of installers and soul-crushing despair on your part, as you try to figure out precisely which version of OpenGL is available on which version of Windows with which specific release of a given video card and oh goodness no, please do not tell me you just upgraded your Cygwin.

In other words, parsing headers at the configuration time of a CPAN module which binds to, for example, libcurl, is madness, and we should stop.

Assume that ctypes for Perl 5 exists very soon in a form in which you can rely on its presence on a modern Perl 5 installation. Assume that if you prefer Python or Lua or Ruby or Haskell or Factor or even some form of Common Lisp not tightly bound to the JVM or the CLR that you have a similar library which knows how to translate from your language's calling conventions to the C ABI to which the library's exported functions conform and that the type mapping problem is solved for 80% of the cases.

Now you need some mechanism to identify the symbols exported from the shared library to generate the appropriate thunks.

I've tried (and failed) to use Swig, and I blame myself more than anyone else for that—but Swig is the wrong answer. Parsing C headers is the wrong answer in 2010 and it was the wrong answer 20 years ago. C headers do not provide the right information in the right form. Effectively you have to have a bug-free C preprocessor to expand headers into literal C source code and then hope that your C parser will identify the correct information you need.

What's the right level of abstraction? That depends on the information a thunk library such as ctypes needs to know:

• The name of an exported symbol
• Its input and output types (and in specific, bit width, signedness, any varargs)
• Constness of pointers or expected modification of out parameters
• Exceptional conditions such as control flow modifications through longjmp
• Error handling, such as setting errno or special return values
• Resource handling, such as a function which returns a malloced value but expects you to free it yourself (or some combination)

... and probably more.

I've set aside the concept of opaque pointers versus raw structs, because that's another rathole full of platform-specific concerns (and besides that, any library which does not expose only opaque pointers to external uses is in a state of denial of reality and deserves a very good refactoring), but you probably already get the idea.

Wouldn't it be nice if shared libraries could provide some sort of machine-parseable, semantics-preserving, declarative (that is, no cpp necessary!) file which all of us poor users could parse once with our thunk generators to produce bare-bones, no sugar added interfaces to these wonderful libraries, then get on with the interesting work such as building Pygame and SDL_perl in wonderfully Pythonic and Perlish ways instead of manually reading SDL_video.h and figuring out how to map all of that implicit information into XS ourselves?

I'm not asking for another section crammed into ELF files and I'm not suggesting that the fine people behind libxslt need to compile a manual file of machine-extractable information themselves—if we had a nice format all of our dynamic languages could understand, anyone could make this file once for any API/ABI version of the library and we could all share for a change. Wouldn't that be nice?

(and yes, I'm aware of CORBA and COM and their IDLs, but the existence of Monopoly money by no means renders a $20 useless at the grocery store)

Isn't that an axiom of Perl? Laziness wins, so let the compiler and runtime and libraries do the work so that lazy people will still get their jobs done well?

The question is "What do you encourage?" Which design decisions does your language allow people to make with the least friction? (Leave out the people who distrust arrays and so they use the filesystem to process aggregate data structures, line by line—I've seen this, but not in Perl. These people need more help than language design; this is active malice and hostility toward programming and programmers.)

I come back to the example of Moose because it's a great example. Moose doesn't explicitly say the right way to handle instance variables is through accessors, even though that's probably the best way to handle instance variables. Moose also doesn't explicitly say that the best way to initialize instance variables is through named parameters passed to a constructor, even though that's a great approach. Instead, Moose makes those seem natural and easy such that you must be actively malicious (or be very curious, or have very specific, special needs in your project) to stumble upon a different approach.

Moose doesn't explicitly teach encapsulation is good! I'm not sure how well you can teach that, until the student has experienced the problem of poor encapsulation. Even so, you can teach the student good habits and you can hope that osmosis will drill in the point that encapsulation of instance variables leads to fewer problems.

Compare that to the Perl 5 documentation about how you can bless any type of reference, how you can manipulate data structures, and—oh, look over there at that blessed typeglob in core libraries!

The old Perl 5 way of doing something special when you need something different than default is to cram something else in a C-shaped box to fit the C-shaped prejudice of the core Perl 5 VM. After all, experienced C programmers know that an object is just a struct with an extra pointer hanging off it for method dispatch. (Larry has since resolved not to borrow features from Python wholesale.)

(Incidentally, this is why many novice programmers who've just learned Ruby think that Ruby is more flexible than Perl: Perl 5's Perlish API for metaprogramming is horribly ugly because of early and foolish consistencies. Even so, Perl 5 lets you manipulate more things than Ruby does because while ugly is skin deep, its manipulexity is chromosonal.)

The Moose approach is simpler: there's an official metaobject protocol with a very Perlish interface and no distinction between the Perl side and the C side. (You can't teach consistency either, but you can certainly demonstrate its presence and its absence.)

The lesson from all of this is simple: if you want to help the lazy programmers in the middle of the spectrum, make their lives easier by making the right thing so easy to do that their laziness will prevent them from doing the wrong things.

Our work involves manipulating and warehousing external data feeds and developing web interfaces to create home search tools for prospective buyers and lead management tools for real estate agents. We’re looking for a skilled coder to join our small team of talented engineers in Ann Arbor. We hope to find an experienced programmer who is a good fit with our team, well-versed in multiple languages, able to learn quickly and work independently. We work in a Linux environment, and tools and languages we use include Perl, Ruby on Rails, PostgreSQL, and GIT. Perl experience is a significant plus, but your current comfort level with any of these specific tools is less important than overall technical aptitude and ability to learn quickly and fit in well with the current team.

That’s a little thin on details partly because the work is varied. If you think you might be interested, take a look at the full job posting.

TigerLead is a lovely company to work for and this is a great opportunity. Highly recommended.

We’re looking for a skilled coder / database wrangler to play a key role within our Operations and Engineering teams. The various responsibilities of the job include working with the large databases underlying our real estate search tools, setting up services for new clients, communicating with clients to evaluate bug reports, troubleshooting technical issues escalated by our client services team, and interfacing with the engineering team on systems maintenance and development. The scope of work that we do involves managing hundreds of external data feeds that feed into in-house databases totaling several million property listings. These listing databases power hundreds of real estate search sites used by more than a million home-buyer leads, who are tracked and cultivated by the thousands of Realtors using our management software. This position is critical to the robustness of these systems.

If that sounds like interesting work to you then take a look at the full job posting.

TigerLead is a lovely company to work for and this is a great opportunity. Highly recommended.

... it turns out that there's almost nobody who's actually getting paid to work on actual core Ruby (less than the number who are getting paid to work on JRuby and IronRuby and MagLev and so on). Ruby really needs to find a sugar daddy -- in my opinion, a deep-pocketed Japanese corporate sugar daddy -- and find it soon.

Do I recall correctly that Tim helped convince Sun to hire a handful of JRuby developers in 2007? (They've since left the Oracle-eclipsed Sun.) Hmm. Hirer's regret?

Ian Bicking criticized Sun for the JRuby hire at the time; apparently I'm not the only one who detects a whiff of provincialism here. It's odd; John Ousterhout worked at Sun for a while, putting resources behind dynamic languages in the form of Tcl. (Anyone who says their favorite dynamic language is unsurpassed in automation abilities, easy embedding, Unicode, event handling, and GUI integration probably knows nothing about Tcl. Those who do not know programming language theory condem the rest of us to hear their incessant chest beating.)

I don't really want to talk about Ruby, though. I want to talk about Perl.

I wrote How Perl Happens as a subtle reminder that Perl has little corporate sponsorship. To my knowledge, there's no IronPerl nor JPerl. There's no Enterprise Perl fork of the Perl 5 interpreter. There's no Perl on the Smalltalk VM, nor on V8, nor on SpiderMonkey.

Booking.com did make a large donation of $50,000 to contribute to Perl 5 development -- and they deserve credit and thanks and acknowledgement for that, as do all of the contributors to TPF... but try to find someone (anyone!) employed anywhere, as an agent of TPF or otherwise, with the full-time job to implement, support, maintain, or manage any implementation of Perl.

TPF hasn't spent (to my knowledge) any of that $50,000 -- though not because TPF is unwilling, but because no one has made a plausible, workable plan for spending that money.

Ian Hague made a generous donation to help with Perl 6 development, and that's sponsored several grants to people such as Patrick Michaud, Jonathan Worthington, and Jerry Gay. As well, Vienna.pm has sponsored a portion of Jonathan Worthington's time for several months -- but again, these are modest grants, not resembling anything close to full-time employment.

The thought of a business benefactor is interesting, especially in comparison to other language communities. Does the close-knit Japanese community primarily developing MRI form a barrier to sponsorship from outside of Japan? Does Python gain an edge from Google's deep pockets? Can anyone argue that Zend's control over PHP has helped the language more than it has hurt the language? Would Lua have worked anywhere outside a research university?

I don't mean to criticize or condemn alternate Ruby implementations for their own sake. RubySpec is the most important project to come out of completing implementations; it can turn them into complementary implementations. (I do choke back chuckles every time I read a starry-eyed programmer who's only ever learned Ruby say that the Ruby community has the best testing of any programming language ever invented in heaven or on earth, because they figured out how to write test descriptions and invented a new acronym for it, but I'm a history-aware jerk that way.) There's nothing wrong with multiple implementations of a programming language built around a common specification (hey, Python has a pretty good one!) and a comprehensive test suite (fill in your own parenthetical statement here).

The problem comes, as Tim Bray is just starting to discover, when the reason for those competing implementations is to divide up a pie of consulting and support fees by capturing market segments tapped, untapped, unrelated but tappable, and not invented yet but looking very tapperific without replenishing the commons.

In other words, while I have no doubt of the good intentions of someone porting Perl 5 to the CLR or the JVM to run in businesses where standard Perl 5 might not be allowed or appropriate or to give access to libraries not available in Perl 5 yet, I suspect that the corporate interest in sponsoring such development is to expand the walled garden around the CLR or the JVM, not to make a better Perl 5 or to help programmers get their work done with less ceremony and mess and frustration.

This leaves me with two questions:

• Why doesn't Perl 5 have sponsored development beyond a few donations here and there? (Again, those businesses and individuals who have donated to support Perl development in any form deserve praise. By no means do I intend any slight or anything other than gratitude toward them.)
• Is the Perl community fortunate that it hasn't fragmented by implementation on various platforms, or unfortunate that it couldn't perform corporate aikido to turn walled-garden sharecropper sponsorship into dollars to improve the Perl core for everyone?

For example, many Perl community websites are unattractive visually. There are exceptions. The London Perl Mongers site is very attractive. I also like the design of the Perl Moose site.

Yet a common refrain in many (more heat than light, sadly) discussions is that functional is more important than attractive. Yes, that's a false dichotomy.

I don't want to restart the debate over the attractiveness of Perl 5 websites. (I'd love to see variant templates that writers, developers, and projects could borrow and adapt for consistency of good design, but that's a different story.)

I want instead to talk about installation. This ties in to Who Benefits from the CPAN?. Consider, for example, the amount of work required to configure a CPAN client in 2000 versus the amount of work required to configure the same client in 2009. This has improved.

Yet compare that to installing a random PHP application.

Preemptive ad hominem: if at any time you feel the need to comment here or elsewhere saying "You fool! You foolish fool! You want to turn Perl into Java, you drooling maniac!", you are a masochist and your opinion does not matter.

One commenter on the previous post praised Ruby Gems for not failing tests and not spewing diagnostic output and asking confusing questions. Of course the thought of not running tests on installation horrifies me (how do you have any idea if your software works?), but the point is a good one. What information do users need to know to install your software? What questions do you really need to ask them?

We've optimized the Perl ecosystem for Perl developers. That's good. That helps us. That doesn't necessarily mean that what's good for Perl developers is good for Perl users. That also doesn't mean that we can only optimize the experience for one group at the expense of the other.

We can do better.

I argue often for better defaults -- not just at the language level but at every level users might see. Yes, I've overloaded the word "user". Deal with it.

One important question for me as I ponder visions for Perl 5 and its ecosystem is "How can we improve the default behavior for novices and for most uses by experienced programmers while allowing experienced programmers to customize behavior for advanced uses?" That's a language design principle. If you analyze the design of Perl 5, you'll see Larry's answers everywhere. The same goes for Perl 6.

I don't have complete, systemic answers for that yet. We have a lot to ponder. As food for thought, consider the (oblique) connection between two comments. The first comes from a critique of Python:

Also, its libraries require compilation. What a nonsense.

The second comes from Scott Walters, specifically his Language choice motivated by greed journal:

Moose is good and Moose is great, but the real win would be to automatically install modules, automatically call the OS's package manager to install libraries and other deps, to have created PHP before php did (or after) and have a libperl linking executable that outputs web pages (period), and most of all, to have a sandbox mode enabled by default (tired idea, I know) where users can write code without getting yelled at by humans, only by the computer, even if just because the code is labeled (all is fair if you predeclare) as "babytalk". Perhaps this would be a mix of strict, diagnostics/warnings, and a Perl::Critic policy that tries to help them in only the most immediate sense likely to be useful to novices in the short term.