Missing deps modules

[ambs]

After some chat with @dagolden, he suggested to post here ideas on what is and what should be Smoker behavior in some circumstances.

In my humble Win32 smoking machine I am having a big problem with modules that are tested everyday (every time the smoker script reindexes cpan and restarts).

I am not sure in which situations this happens, but it seems it happens when:

• There is a missing dependency. A module XYZ says it depends on ABC, that is not on CPAN. Then, next day, XYZ is tested again, checking if ABC is already available. Although this is a nice behavior (the CPAN mirror might had trouble copying a file, and in the next day it will be available), but it is also a problem for some modules released in the last century and whose dependencies are not yet in CPAN (or aren't there anymore)
• The second situation is when a dependency fails to build. In this case I am not sure if I understood what is done, but it seems that if a module A depends on B, that depends on C, that depends on a failing module, D, A will also be tested everyday (because the smoker script failed to install dependencies, so it is not A's fault). Problem, next day, A will be built again, installing B, and C.. and finding D is missing. This is quite fast if there are few dependencies, but in some situations modules have a hundred dependencies, and the last one will fail. Why this is a good procedure? because D's author might release a fixed release, and A will start working again.

I think these two situations need to be handled in a different way. I am not sure if they are feasible, if they are desirable, or even if they are possible (do not know the details behind the smoker script).

• For the first situation, and if there is a way to compute the age of a distribution, set a delay before testing the distribution again. Can be something simple, or complicate, last, compute the number of months since the last release, and use that number as the number of days to wait until testing the module again. So, if the module was released less than a month, it will be tested everyday. If it was released in 2000, it will be tested every 156 day. Cool, huh? Problem... is it possible to do this kind of management in the queue.
• For the second situation, and this is something @karenetheridge already suggested. Author of module A would love to know that he is depending his work in a module that is failing to install. So, we probably should report a NA result. This would help the author. To help the smoker, it would be interesting to save which modules (kleene closure) a module depends on, so we could check right ahead if we have some fail reports on any of them. If so, report the NA and continue.

Is any of this possible? Does this make sense? If so, can I help somehow?

Cheers

[dagolden]

For the first situation, I don't understand how one would do this in practice. I'd be more inclined to throw the distribution name and epoch timestamp into a file and skip testing for some arbitrary amount of time (at which point the DB is cleared). DB_file would probably be fine for this.

For the second situation, I continue to vehemently oppose using "NA" for this purpose. I do favor creating a new grade to cover it, but that requires @barbie to buy in since it affects statistics and reporting. I think it's likely to irritate a lot of authors if it's a regular report that shows up in their inbox or on metacpan, because most of them won't have the time or inclination to do anything about upstream problems.

[barbie]

For both situations this was handled by CPAN::YACSmoke (and possibly CPANPLUS::YACSmoke as was based on the former).

In the first situation CPAN::YACSmoke kept a history of what was tested, and if module A was tested and its dependency module B failed (and even further up the tree), it was prevented from being tested again until the dependency version changed.

In the second situation, a FAIL would be sent for the dependency, and like the first situation, the dependent would be blocked from being tested until the dependency was released with a newer version. In both cases the tester could clear the history and start again if they so wished.

Like David I am very much against using NA for this, as it has nothing to do with the author of the dependent module. The author might want to know they have a bad dependency, but currently that is sort of covered by the CPAN Deps website. I don't believe a new grade is called for, but a mashup website, that takes the reports and the basis of CPAN Deps, but gives the author a clearer picture of the dependency tree based on OS/Perl versions. Or possibly a watch-list website that reports on dependency failure to dependent authors. It would have to be opt-in, as I know from bitter personal experience that some authors will take this as an eager opportunity to tear strips off you if you don't.

I might consider a watch-list website in the future, unless someone else gets to it first, but creating new grades for this is not the way to go in my opinion.

[ambs]

Hello

I wouldn't mind to get NAs for the failiing deps, but that is not my problem now.

So, good we agree something needs to be done to missing deps, or failing deps.
Should I try CPAN::YACSmoke? Didn't know there was this diversity of approaches. What is the most maintained one?

Best,

[chorny]

This can be solved using CPAN.pm configuration. "halt_on_failure" in CPAN::Config should be set to 0.

[barbie]

CPAN::YACSmoke is now obsolete. The one to look at is CPANPLUS::YACSmoke, however, this only works with CPANPLUS (hence the name change). If chorny's solution above works for you, that might be a better solution for you now.

[ambs]

Will check @chorny solution later today. Will give feedback then.

[ambs]

I confirm I had it as 1. Changed to 0. News soon :-)

[ambs]

ok, now there are fewer modules testing again. Namely, modules which build process I need to kill (infinite loops, et all), and some other non standard modules foo.pm.gz. But still taking too much time to do a cycle of 3000 already processed modules...