• YAPC::NA will be livestreamed in real-time for free (blog.yapcna.org)
• Introduction to how to do Perl one-liners (catonmat.net)
• NYTProf, File IO, and an Optimization Gone Awry (modernperlbooks.com)
• Reddit API for Perl (blogs.perl.org)
• Lots of Perl authors at YAPC::NA this year (blog.yapcna.org)
• Review of "Programming Perl, 4th ed." (i-programmer.info)
• Don't use Cache::Memcached for UTF8 strings (domm.plix.at)
• PerlGSL, interfaces to the GNU Scientific Library (blogs.perl.org)
• Learning Perl with testing (nbezzala.blogspot.com)
• Separating presentation and content in TT (modernperlbooks.com)

I've used it successfully for a few projects and it's been great.

It's a lot better now that I've written my own silly little Template Toolkit plugin to reduce the need for writing lots of repetitive HTML in my templates. (It's like Haml but less ugly and more Perlish and easier to extend.)

Writing a TT2 plugin is relatively easy. Of course I do it the wrong way; when you initialize your plugin, you have the ability to manipulate TT2's stash. This is the data structure representing the variables in scope in your templates. Where a well-behaved template should use object methods to perform its operations, my code stuffs function references in the stash. Here's the relevant code:

sub new
{
    my ($class, $context, @params) = @_;

    $class->add_functions( $context );

    return $class->SUPER::new( $context, @params );
}

sub add_functions
{
    my ($class, $context) = @_;
    my $stash             = $context->stash;

    while (my ($name, $ref) = each %exports)
    {
        $stash->set( $name, $ref );
    }

    $stash->set( process => sub { $context->process( @_ ) } );
}

I'll fix this eventually, but the process of making this work was interesting.

In my first attempt (see Write the Wrong Code First for the justification), I'd write the function I needed, like row(), which creates a new Bootstrap row or maincontent() which creates the main content area of the page. Then I'd add that function to the %exports hash and everything would work.

After the sixth function, keeping that list up to date was tedious. Then I kept forgetting it. After all, any time you have to update the same data in two places, you're doing something wrong.

Now the code looks more like:

sub row :Export
{
    return <<END_HTML;
<div class="row">
    @_
</div>
END_HTML
}

... with a single code attribute marking those functions which I want to stuff into the template stash. I've used Attribute::Handlers before, but I always end up reading the manual and playing with things to get them to work correctly. (Something about the way you have to write another package and inherit from it to get your attributes to work correctly always confuses me.)

My second attempt lasted no longer than ten minutes. I switched to Attribute::Lexical. This is almost as trivial to use as to explain:

use Attribute::Lexical 'CODE:Export' => \&export_code;

Whenever any function has the :Export attribute, Perl wil lcall my export_code() function:

my %exports;

sub export_code
{
    my $referent = shift;
    my $name     = Sub::Identify::sub_name( $referent );

    return unless $name;
    $exports{$name} = $referent;
}

The first argument to this function is a reference to the exported function. I use Sub::Identify to get the name of the function reference. (That wouldn't work for anonymous functions, but I can control that here.) Then I store the name of the function and the function reference in a hash.

It took as long to write as it does to explain.

A lot of people dislike the use of attributes. Used poorly, they create weird couplings and plenty of action at a distance. Attribute::Handlers can be confusing.

I like to think that I'm using attributes well here (even if I'm abusing TT2 more than a little), and that they've simplified my code so that I can avoid repeating myself and performing manual busywork that I'm likely to forget. Even better, the code to use them isn't magical at all: it's all hidden behind the pleasant interfaces of Attribute::Lexical and Sub::Identify.

I designed this system around the idea of a pipeline of related processes, where each component is as independent and idempotent as possible. This has positives and negatives; it's an abstraction like any other.

I initially wrote the "fetch remote web page" and "analyze data from that page" as a single step, because I thought "analyze" was the main goal and "fetch" was a dependent task. I separated them a couple of weeks ago to simplify the system: analysis now expects data to be there, while fetching can be parallel on a single or across multiple machines. (Testing the analysis step is also much easier because feeding in dummy data is now trivial.)

I use the filesystem as a cache for these fetched files. That's easy to manage. I modified the role I use to grab data for the analysis stage to look in the cache first, then fall back to a network request. That was easy too. The get_formatted_data_for_analysis() method looked something like:

sub get_formatted_data_for_analysis
{
    my ($self, $type, $key) = @_;

    my $cached_path         = $self->get_cached_path( $type, $key );
    if (-e $cached_path)
    {
        my $text = read_file( $cached_path );
        return $self->formatter->format_string( $text ) if $text;
    }

    return $self->formatter->format_string( $self->fetch_by_url( $type, $key ) );
}

I thought I was done. This trivial caching layer took five minutes to write and gave my project a lot of flexibility.

I thought this would speed up the processing stage, because I was able to make the fetching stage embarrassingly parallel so that more than one fetch could block on network IO simultaneously. My rough benchmark didn't show any speed improvement, but it was fast enough, so I moved on.

On Friday I decided to profile the slowest stage of the application with Devel::NYTProf. The slowest stage was the processing stage. I isolated it so that it performed no network fetching. It was still slow.

One of the formatter modules used to extract data from web pages is HTML::FormatText::Lynx. It allows me to run lynx --dump to strip out all of the HTML and other formatting of a document. The formatter allows you to pass in the name of a file or the contents of a file as a string.

For some reason, most of the time in the processing stage in the profile was spent in file IO. That wasn't too surprising; these aren't all small files and there may be thousands of them. I dug deeper.

Most of the time in the processing stage in the profile was spent in reading the files in my method and reading files in the formatter—reading files, even though I was passing the contents of those files to the formatter as strings.

I poked around at a few other things, but came back to the source code of the formatter. A comment in HTML::FormatExternal says:

format_string() takes the easy approach of putting the string in a temp file and letting format_file() do the real work. The formatter programs can generally read stdin and write stdout, so could do that with select() to simultaneously write and read back.

In other words, all of the work I was doing to read in files was busy work, duplicating what the formatter was about to do anyway. (Okay, I stared at the code for a couple of minutes, thinking about various approaches of rewriting it and submitting a patch or monkey patching it. Then I turned lazier and wiser.) I rewrote my code:

sub get_formatted_data_for_analysis
{
    my ($self, $type, $key) = @_;

    my $cached_path         = $self->get_cached_path( $type, $key );
    return $self->formatter->format_file( $cached_path ) if -e $cached_path;

    return $self->formatter->format_text( $self->fetch_by_url( $type, $key ) );
}

The result was a 25% performance improvement.

Three things jumped out at me in this process. First, how nice is it to have a working tool like NYTProf and a community that distributes source code, so that I could examine the whole stack of my application to isolate performance problems? Second, how interesting that an assumption and an admitted shortcut in a dependency could have such an effect on my own code. Third, how much more I like my new code with all of the file handling gone; pushing that responsibility elsewhere is a nice simplification without the performance improvement.

Perhaps the two tools I miss most from my C programming days are Valgrind/Callgrind and KCachegrind, but NYTProf goes a long way toward filling that gap. Besides, I'm at least 20 times more productive with a language like Perl.

• Always encode and decode Unicode at the edges of your program (perl.com)
• mod_perl 2.0.6 released (blogs.perl.org)
• Comments are open on the Perl Foundation's latest grant proposals (news.perlfoundation.org)
• Creating weekly schedule charts with HTML::Template (blog.nu42.com)
• Playing music with Audio::Beep (blog.und3f.com)
• Tad McClellan has passed away (blogs.perl.org)
• Coding before having expert-level understanding is OK (modernperlbooks.com)

• In case you have a burning need to use ಠ_ಠ for warnings/errors: (metacpan.org)
• Expanded the "More Tools" page of the ack website: (betterthangrep.com) Help your users by including alternatives for them to choose from!
• YAPC::NA is NOT sold out. Still 30 tickets left. Sign up now! (blog.yapcna.org)
• Before you send a patch, send an email (petdance.com)
• For want of a newline (blogs.perl.org)
• Use the command "perldoc -q whatever" to search the Perl FAQ for "whatever"
• Perl 5.16.0 is coming soon, probably early May (rjbs.manxome.org)
• Regexp::Common has regexes for just about everything. Check there first. (search.cpan.org)
• Is Try::Tiny still the best choice for exception handling? (stackoverflow.com)
• Using named regexes in ack output (stackoverflow.com)
• Automating gov't websites w/WWW::Mechanize + HTML::DOM (blogs.perl.org)
• Using Moose's "lazy" attribute for caching (blogs.perl.org)

I keep this schema DDL in version control. I also keep a SQLite database around with some test data (but the database isn't in version control).

I usually find myself writing a little shell script or other program to to regenerate the DBIC schema from that test database. That usually requires me to make manual changes to the test database representing the changes I've just made to the DDL.

After doing this one too many times, I decided to combine DBIx::RunSQL with the schema loader. By creating a SQLite database from my DDL in memory, I can create a schema without me modifying any databases manually.

This was easier than I thought:

#!/usr/bin/env perl

use Modern::Perl;

use DBIx::RunSQL;
use DBIx::Class::Schema::Loader 'make_schema_at';

my $test_dbh = DBIx::RunSQL->create(
    dsn     => 'dbi:SQLite:dbname=:memory:',
    sql     => 'db/schema.sql',
    force   => 1,
    verbose => 1,
);

make_schema_at( 'MyApp::Schema',
    {
        components => [ 'InflateColumn::DateTime', 'TimeStamp' ],
        debug => 1,
        dump_directory => './lib' ,
    },
    [ sub { $test_dbh }, {} ]
);

The next step is to connect everything to DBIx::Class::Migration—but first things first.

Yes, the Modern::Perl module was as much a polemic as it was a convenience.

I know File::Slurp exists, but my fingers by now know how to read from a file in a single line of (impenetrable to the uninitiated) code:

my $text = do { local (@ARGV, $/) = $file; <> };

... and in each case, my initial feeling of "Why bother? What does that offer? How silly!" were wrong. In every one of these cases, the ability to write (and the requirement to read) less code has made my code better.

With say I don't have to worry about single- versus double-quotes, or even quoting at all sometimes. With use Modern::Perl;, I don't have to worry about enabling various features and pragmas. With File::Slurp, all I have to care about when reading from a file is typing read_file( $path ).

None of these are big deals on their own, but they're little details I don't have to worry about anymore. The same principle which says that Proc::Fork is easier to manage than writing your own forking code (I've written far too much of my own forking code) applies.

Sometimes getting the little nuisances out of the way makes me more productive and ready to tackle the big nuisances. Maybe saving my brainpower for complicated problems (what's the standard deviation from a least square fit?) is a better approach to typing my own read_file() function on every project.

As silly as it once seemed to use a CPAN module for a one liner, I've realized that not reusing good code is even sillier.

• User-friendly CLI apps with MooseX::App (blogs.perl.org)
• An age report for git repos (blog.urth.org)
• Sane time zone management in PostgreSQL (justatheory.com)
• Integrating Perl with the nginx web server (blogs.perl.org)
• The price of cleverness (YAML is not safe) (blogs.perl.org)
• By the way, all Perlbuzz tweets are from the command line via @ttytter, a Perl 5 Twitter client
• Expanded the "More Tools" page of the ack website: (betterthangrep.com) Help your users by including alternatives for them to choose from!
• Recapping DC-Baltimore Perl Workshop 2012 (domm.plix.at)
• Poet, a web framework for Mason (openswartz.com)
• Writing init scripts w/Daemon::Control (hashbang.ca)
• Padre 0.96 released. (blogs.perl.org)

(If that hasn't offended you yet, note that that sentence doesn't include "a better type system", on purpose.)

While you can do some amazing things with modern JavaScript (see also ClubCompy, a retro-style programming environment designed for kids of all ages, for which we have a compiler and interpreter written in JavaScript), its flaws of language and ecosystem are obvious. The latter are obviously products of its environment.

Consider: you don't have anything like the CPAN for JavaScript in the browser. (Yes, I'm aware of NPM. No, it doesn't count. The point of client-side JavaScript delivered from a web page is that you don't have to have anything other than a web browser installed.)

Consider: this means you either make lots of requests for your dependent libraries (jQuery, any plugins you use, the JavaScript you've written), which is good in that if you use these libraries unmodified and load them from a public CDN, there's a chance some cache in the middle will already have them cached, but you still pay the network penalty for loading all of those libraries n at a time or you glue them all together on the server side somehow and send the client only one thing, except that it's only cached for your site.

Also, if you find a bug in a dependency, you get to regenerate that big blob of code. (If you don't find that bug, you get to live with it.)

I think about these things when I see a big lump of code stuffed into YAML.pm. Because we've left 1994 behind in the Perl world, we're able to take advantage of an amazing library distribution and dependency management system in the CPAN, where installing dependencies (and knowing they pass their tests) is so well understood that it's an exceptional condition when it doesn't work. In the past couple of years, installations have become so easy thanks to newer tools like perlbrew and cpanm that (if you're in the know) it's easier to manage code this way than to consider not.

... except for when you stuff generated code in your repository instead of as a dependency. (Test::Builder is a strange case. You want your underlying test library to be as stupidly simple as possible and not to rely on anything else so it's as unlikely to fail as possible and as impossible to interfere with what you're testing as ever.)

Now when there's a bug in the dependency in the generated code, everything which uses the dependency has to be updated too. Read carefully. You can't merely update the dependency. You have to know everything on which it depends and wait for the authors to get around to updating their generated code.)

I admit, I'll probably never understand the mindset which says "I'm distributing software for end-users to install in the worst possible way so that they won't have to install software." I understand the use of things like App::FatPacker to make one-file installations possible, but actively distributing generated code in CPAN distributions? Where CPAN has a working dependency resolution model already in place? Where your distribution is already an upstream dependency of thousands of other distributions?

I just don't understand it. I understand that the business of shipping software is the art of managing competing needs, but I can't see how optimizing for fragility helps anyone.

• K=R=P (Kindness = Repeat business = Profit) says @tom_peters. I say that for open source, K=U=G+S, Kindness = Users = Growth+Success.
• You probably don't want to be using use_ok() (modernperlbooks.com)
• Interpolate, concatenate or join? (perlmonks.org)
• Quick-n-dirty tool to send messages to HipChat (blogs.perl.org)
• Painless RSS processing with Mojo (blogs.perl.org)
• RT @chromatic_x If you use SQLite for your CPAN distribution's tests, consider an in-memory database. Helps parallelism/Reduces disk IO.
• Explaining web programming via Plack (blogs.perl.org)
• Common Perl pitfalls (perlmonks.org)
• Test::WWW::Mechanize adds scraping functions (perlbuzz.com)