Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add some args (upload/download interval and parallelism on upload) #5

Closed
wants to merge 14 commits into from
Closed

add some args (upload/download interval and parallelism on upload) #5

wants to merge 14 commits into from

Conversation

cleciusjm
Copy link

Created some args to allow more personalized usage:

  • dry-run: to test all args without any write action on target
  • target.upload-interval: allow to customize delay between uploads, useful on internal servers when load could be bigger
  • target.concurrent-uploads: allow concurrent uploads in same folder, useful as previous
  • source.download-interval: allow to customize delay between downloads, also useful on internal servers

netmikey
netmikey requested changes Aug 15, 2023
View reviewed changes
src/main/java/io/github/netmikey/mvncloner/mvncloner/Publisher.java Outdated Show resolved Hide resolved
src/main/java/io/github/netmikey/mvncloner/mvncloner/Publisher.java Outdated Show resolved Hide resolved
src/main/java/io/github/netmikey/mvncloner/mvncloner/Publisher.java Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
@cleciusjm cleciusjm requested a review from netmikey August 16, 2023 12:14
@netmikey netmikey linked an issue Aug 18, 2023 that may be closed by this pull request
Sleeps make tool slower than it needs to be? #3
Open
@netmikey
Copy link
Owner

@cleciusjm I added a couple of improvements according to what we discussed above. I don't have test Nexus servers at hand anymore unfortunately. Would you mind testing my changes to see if they work as described in the README and don't break existing behavior?

@cleciusjm
Copy link
Author

nice, only check action feel like duplicated code to me, but ok

my new nexus is in production, i cant run publish action, but mirror and check seems ok to validate here

@netmikey
Copy link
Owner

I agree about the code duplication. The 3 lines of HttpClient code felt too trivial to extract and the WebClient API with its try-with-resource would've been cumbersome to make DRY...

Anyway, let me know when you've been able to test-drive the mirror and check actions :-)

@netmikey
Copy link
Owner

nvm, I got rid of the duplicated WebClient code.

@cleciusjm
Copy link
Author

Check is failing because target address itself resolves as 404, but internal artifact addresses prefixed by it are funcional.

2023-08-23 09:14:38.715  INFO 16841 --- [           main] i.g.n.mvncloner.mvncloner.Checker        : Source connection check succeeded! (http://xxxxxx/nexus/service/rest/repository/browse/releases/, responded with 200)
2023-08-23 09:14:39.547  INFO 16841 --- [           main] ConditionEvaluationReportLoggingListener : 

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2023-08-23 09:14:39.552 ERROR 16841 --- [           main] o.s.boot.SpringApplication               : Application run failed

java.lang.IllegalStateException: Failed to execute CommandLineRunner
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) ~[spring-boot-2.3.1.RELEASE.jar:2.3.1.RELEASE]
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:779) ~[spring-boot-2.3.1.RELEASE.jar:2.3.1.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:322) ~[spring-boot-2.3.1.RELEASE.jar:2.3.1.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1237) ~[spring-boot-2.3.1.RELEASE.jar:2.3.1.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) ~[spring-boot-2.3.1.RELEASE.jar:2.3.1.RELEASE]
	at io.github.netmikey.mvncloner.mvncloner.MvnclonerApplication.main(MvnclonerApplication.java:10) ~[main/:na]
Caused by: java.lang.IllegalStateException: Connection check failed: target at https://xxxxxx/nexus/repository/releases/ responded with http status code 404
	at io.github.netmikey.mvncloner.mvncloner.Checker.checkTarget(Checker.java:89) ~[main/:na]
	at io.github.netmikey.mvncloner.mvncloner.Checker.check(Checker.java:53) ~[main/:na]
	at io.github.netmikey.mvncloner.mvncloner.MvnCloner.run(MvnCloner.java:33) ~[main/:na]
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:795) ~[spring-boot-2.3.1.RELEASE.jar:2.3.1.RELEASE]
	... 5 common frames omitted

@cleciusjm
Copy link
Author

that's why my previus check action was targeting artifacts from mirror, and check everything working status without only uploading new data

@netmikey
Copy link
Owner

Check is failing because target address itself resolves as 404, but internal artifact addresses prefixed by it are funcional.
that's why my previus check action was targeting artifacts from mirror, and check everything working status without only uploading new data

Well, if you ran your initial code against an empty target repository, this repository would've also returned 404 since you were doing an http HEAD on the target artifacts themselves, which aren't present on an empty repository. The thing is that you accepted http 404 as the expected return code and went on.

So in short, I think without writing to the target there's nothing else we can do but check that we get an http 200 (or 404) in response to the root URL, which would be the only existing URL in an empty repository.

I added a commit that accepts http 404 now too for the target check.

@cleciusjm cleciusjm closed this Feb 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@netmikey netmikey Awaiting requested review from netmikey

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Sleeps make tool slower than it needs to be?
2 participants
@cleciusjm @netmikey