Allow optional downloading of metadata

[hybby]

Hi, great tool.

I've got a use case where I need to completely clone all contents of a repo, including metadata. The checksum files are used for some purpose, and they must also be transferred.

I see these are hardcoded to have exclusion lists at the moment:

https://github.com/netmikey/mvncloner/blob/master/src/main/java/io/github/netmikey/mvncloner/mvncloner/Scraper.java#L45-L49

Would it be possible to make this optional?

I've been able to work around it by replacing the existing lists with nonsense strings that will never match.

[netmikey]

These are all files that Maven Servers generated themselves. As a client, you upload your artifacts using Maven's http APIs and the Maven Server is the one who generates checksum and maven-metadata files. The tool will therefor avoid transferring such generated files.

Do you have a particular use case where this doesn't apply?

[gdude2002]

Been a bit since this issue was opened, but I figured I'd point this out:

According to the Apache Maven docs:

Repositories contain metadata (aka “repository metadata”) files as well, that enables several “discovery” and “resolution”-like operations for Maven. These metadata files are not Artifacts, and hence, are not addressable by Maven users. They are instead transparently operated and handled by Maven itself in automatic manner. These files are XML files named as maven-metadata.xml (are deployed with checksums just like artifacts are).

It appears that Maven repos aren't supposed to generate these automatically - it just happens that this is something that Nexus does. When copying artefacts from a Nexus server to a Reposilite v3 server, this results in clients being unable to resolve snapshot releases, as mvncloner doesn't upload the metadata files.

[netmikey]

Thanks for the clarification and for pointing out a use case @gdude2002 !

If someone would like to submit a PR that adds a command-line option like --include-metadata that results in using another set of blacklists (empty ones if all metadata is required I guess?), I'd be happy to merge :-)