Testing on blead

Thread view:

Enable EMail Alerts

Start New Thread

Thread rating:

Johan Vromans - 28 Jul 2008 14:11 GMT

We all love the CPAN testers and the results publised at
http://testers.cpan.org.

But recently I suddenly saw a big load of failures in one of my
packages. All these failures were for 5.11.0.

Now, 5.11.0 is a development version, and a moving target. A test
failure with blead reveals often more about blead than about the
tested package.

So, on one hand, it is a good thing that packages are being tested with
blead versions. On the other hand, when users want to find out if a
package is usable they're only interested in released versions of
perl.

What are your ideas about this? Should blead test results be separated
from the other results?

-- Johan

Reply to this Message

Jerry D. Hedden - 28 Jul 2008 15:14 GMT

> We all love the CPAN testers and the results publised at
> http://testers.cpan.org.

Except when failure reports reflect problems in a particular
tester's environment, and not problems with the modules being
tested. (But that goes without saying, right?)

> But recently I suddenly saw a big load of failures in one of my
> packages. All these failures were for 5.11.0.
>
> Now, 5.11.0 is a development version, and a moving target. A test
> failure with blead reveals often more about blead than about the
> tested package.

I routinely report tests against blead. I don't see failures
often, and when I do, they usually reflect some new change in
Perl that requires a subsequent change in the module. When I
encounter such cases, I try to figure out a patch and then
report it as bug (along with the patch) against the affected
module.

> So, on one hand, it is a good thing that packages are being tested with
> blead versions. On the other hand, when users want to find out if a
[quoted text clipped - 3 lines]
> What are your ideas about this? Should blead test results be separated
> from the other results?

I agree they should be segregated. However, they should also be
reported to the module's maintainers so they can fix things
appropriately.

Reply to this Message

Ken Williams - 28 Jul 2008 15:28 GMT

> What are your ideas about this? Should blead test results be separated
> from the other results?

Wholeheartedly, yes. It's useful to get the reports on blead, but
probably not so useful for people who just want to cruise through
reports to see whether the module will work for them - they may not
even usually be aware that they're looking at a blead test.

-Ken

Reply to this Message

Andy Armstrong - 28 Jul 2008 15:31 GMT

>> What are your ideas about this? Should blead test results be
>> separated
[quoted text clipped - 4 lines]
> reports to see whether the module will work for them - they may not
> even usually be aware that they're looking at a blead test.

I'd expect that the main audience for tests run against blead are Perl
5 core developers and the developers of the module itself. There seems
to me to be little need to pollute the public test results with blead
tests at all.

Signature

Andy Armstrong, Hexten

Reply to this Message

Paul Johnson - 28 Jul 2008 22:59 GMT

>>> What are your ideas about this? Should blead test results be separated
>>> from the other results?
[quoted text clipped - 8 lines]
> to be little need to pollute the public test results with blead tests at
> all.

At best publicising those results is harmless. I'm always happy to
receive smoke reports against blead (and any other version except those
that are below the minimum specified version) but I can't imagine them
being of any use to anyone other than me, and possibly p5p when we are
coming up to a release.

Signature

Paul Johnson - paul@pjcj.net
http://www.pjcj.net

Reply to this Message

Webmaster Forum / Perl / Modules / July 2008

Testing on blead