In the April CPAN Testers Summary, Barbie noted that CPAN Testers has nearly reached 12 million reports submitted.

I was recently looking at some Metabase statistics and noticed that it now contains over 6.3 million reports since going live. That means that over half of all reports were submitted by web instead of email. It also means that CPAN Testers has nearly doubled in a year.

The year has not been without hiccups, as the recent Amazon-related outage showed. I'll be spending some time this summer addressing some deficiencies and will be encouraging people to get involved in improving the CPAN Testers clients to make it easier for more people to participate.

Next year, will we see 24 million reports? Let's make it happen!

If you've been building up a backlog, please throttle your uploads.

I apologize for the interruption. Over the next month or two, I'll be making some changes to make sure that CPAN Testers is ready for a repeat of this particular failure mode (as well as other similar failure modes).

If you are already a CPAN Tester, a "profile file" has been pre-generated for you and you should submit a request that it be emailed to you. This pre-generated profile will identify your new CT 2.0 reports in a way that will make it easier to link them to the CT 1.0 old reports as those are converted.

If you are new to CPAN Testing, you will need to generate a profile file. Install Metabase::Fact and then run the metabase-profile program from the command line. It will prompt you to provide some information and output a profile file. The first time you use this profile file to submit a report, your profile will be automatically registered on the CT 2.0 server.

While the CT 2.0 server has launched, the clients that submit reports are, for now, the same ones used for CT 1.0 -- either CPAN::Reporter or CPANPLUS (or one of the smoke frameworks written for them like minismokebox or CPAN::Reporter::Smoker). The transport framework originally intended for email gets hijacked to send reports via http to CT 2.0 instead.

To send reports to CT 2.0, you will need to install the Test::Reporter::Transport::Metabase module and then configure your client to use that transport instead of the default one. Instructions are available on the CPAN Testers Wiki. While the docs are a bit sparse as I write this, I expect they will continue to evolve and improve as more people contribute to them.

If you're really stuck, please check the FAQ or ask for help on the CPAN Testers Discuss mailing list. If you are a CPAN Tester, I highly recommend joining the list to keep up on the latest issues and solutions in the migration to CT 2.0. If you do ask for help, please be sure to include the version number of all the relevant testing modules you're using along with your perl -V output.

Another good place to go for quick questions is the #cpantesters-discuss channel on irc.perl.org.

In other news, I'll be speaking about CPAN Testers at OSCON in my Free QA! talk. If you or someone you know will be there, please check it out or recommend it.

Apocalypse then helpfully batched several thousand reports and tried to overwhelm the server with parallel submissions. At about 450 reports per minute we finally started seeing some submission errors and now have some work to do to use the Amazon SimpleDB more efficiently or to queue submission peaks and process them offline.

In other interesting news, the CT2.0 server now supports Test::Smoke::Metabase reports, which Tux implemented during the Vienna QA hackathon and refined in the last couple weeks. He uploaded 150,000 or so historical Perl smoke reports as a test and is exploring using the CT2.0 server instead of the existing email system that supports Perl smoke testing. It’s the exact same problem (at somewhat lower volumes) that CT2.0 was designed to address and it’s nice to see some synergy in the work to date.

In the end, I redirected to a new Amazon “bucket” and reset the server. So the old results are offline (but should be safe) and I’ll migrate them over when I get some free time.

Thank you to the testers who reported issues and were patient while I fixed things.

Reports per Minute

Total Report Count

Reports have been received from most major operating systems and versions of Perl.

So far, I’ve fixed several issue relating to character encoding, removed a number of dependencies and fixed a few other oddball bugs. Chris Williams developed a new “zero dependency” report proxy, Test::Reporter::Transport::Socket and metabase-relayd to allow CT2.0 testing without any Metabase prerequisites on the target perl.

What to look forward to:

• Barbie has started working on extracting report data from the beta site. This is the critical path before we can plan the full launch of CT2.0
• I plan to start working on the conversion of the 6.9 million reports from CT1.0
• Once I have a chance to deploy an app that gives legacy testers an ID file to associate their new CT2.0 identity with their legacy CT1.0 reports, I’ll open up the beta to anyone who wants to participate

That’s all for now. Thank you, again, to all the beta testers for their efforts and thank you to Perl NOC administrators for their forebearance and support.

Some of the immediate discoveries and improvements include:

• Chris Williams released a new version of CPANPLUS to provide an additional data field needed for CT2.0.
• I revised the Metabase::Fact distribution to avoid some test bugs in bleadperl and downgrade several prereqs so it can be used on Perls back to 5.006.
• I released a couple new versions of Test::Reporter. 1.55 was a patch from Dan Collins to improve how reports are saved/loaded for testers that save files to disk, move them to another machine and then upload. 1.56 attempts to fix line-ending issues moving report files between Windows and Unix (reported by Serguei Trouchelle)
• I released a new version of Metabase::Client::Simple with Net::SSL as a prerequisite. This may be a temporary fix (and it makes it not so “simple”), but for now it helps beta testers avoid annoying and hard-to-diagnose errors from LWP

I want to particularly thank all the intrepid CT2.0 beta test volunteers: Andreas Koenig, Apocalypse, Dan Collins, MW487, Nigel Horne, Serguei Trouchelle, and Shlomi Fish. While I’m sure they might feel more like crash-test dummies, I greatly appreciate the time they are taking and the excellent feedback they’ve been giving. Keep up the great work!

After another week or so of shakedown, I hope to open the beta test to anyone interested in participating.