-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migration of the scheduler and orchestrator packages #65
base: main
Are you sure you want to change the base?
Conversation
1aba782
to
937bb4b
Compare
As you requested @javiertuya #59 (review), I've migrated the module that generates the Jenkinsfiles (nothing related to the UsageProfile or the Cost estimation). |
retorch-orchestration/retorchfiles/customscriptscode/custom-tjob-setup
Outdated
Show resolved
Hide resolved
...stration/src/main/java/giis/retorch/orchestration/generator/OrchestrationGenericToolBox.java
Outdated
Show resolved
Hide resolved
retorch-orchestration/src/main/java/giis/retorch/orchestration/generator/RetorchMainClass.java
Outdated
Show resolved
Hide resolved
retorch-orchestration/src/main/java/giis/retorch/orchestration/model/ExecutionPlan.java
Outdated
Show resolved
Hide resolved
...orchestration/src/main/java/giis/retorch/orchestration/orchestrator/RetorchOrchestrator.java
Outdated
Show resolved
Hide resolved
...orchestration/src/main/java/giis/retorch/orchestration/orchestrator/RetorchOrchestrator.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code review done, but problems generating the jenkinsfile according to the documentation.
@@ -223,25 +223,11 @@ Once created the different properties and configuration files, the single module | |||
``` | |||
|
|||
### Executing the Orchestration generator |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm trying to execute the main method from eclipse to generate fullteaching (fullteaching repo is a sibling of retorch). This is what I see:
- Logs are not shown:
SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.
- Executing with parameters:
"com.fullteaching.e2e.no_elastest.common","FullTeaching","../../retorch-st-fullteaching"
does not find the resource file: `Exception in thread "main" java.nio.file.NoSuchFileException: retorchfiles\configurations\FullTeachingSystemResources.json
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@javiertuya
I was experiencing some problems with the IDE during the past week, I've managed more or less to solve it (working with the LAPTOP and DESKTOP in parallel). Some remarks (that I don't understand):
- FullTeaching was with the old orchestration generation tool, which is why you don't find the Resources file or the configuration file. Now, the ft_AlignOrchGeneratorVersion is fully updated and working.
- I've created a class with a simple call to the main in the FullTeaching project, importing the snapshot orchestration using the dependency:
<dependency>
<groupId>io.github.giis-uniovi</groupId>
<artifactId>retorch-orchestration</artifactId>
<version>1.1.1-ft_migrateschedulerorchestrator-SNAPSHOT</version>
</dependency>
The class looks like this:
package com.fullteaching.e2e.no_elastest.functional.test;
import giis.retorch.orchestration.classifier.EmptyInputException;
import giis.retorch.orchestration.generator.OrchestationGeneratorMainClass;
import giis.retorch.orchestration.orchestrator.NoFinalActivitiesException;
import giis.retorch.orchestration.scheduler.NoTGroupsInTheSchedulerException;
import giis.retorch.orchestration.scheduler.NotValidSystemException;
import java.io.IOException;
import java.net.URISyntaxException;
public class RetorchMain {
public static void main(String[] args) throws NoFinalActivitiesException, NoTGroupsInTheSchedulerException, EmptyInputException, IOException, URISyntaxException, NotValidSystemException, ClassNotFoundException {
String [] argum={"com.fullteaching.e2e.no_elastest.functional.test", "FullTeaching", "./"};
OrchestationGeneratorMainClass.main(argum);
}
}
I've also added some test cases to check that the scripts generated are correct and solved some problems with the current implementation of the execution plan generation.
If I add the path com.fullteaching.e2e.no_elastest.functional
, it fails because It finds java classes without annotations, Should I change the actual implementation to allow this behavior?
If I understood well, you have the following directories:
.
├── retorch/
│ ├── annotations
│ └── orchestration/
│ └── -YourMainClass
└── retorch-st-fullteaching
And you're trying to call RETORCHOrchestrationGeneratorMain.main() from YourMainClass in orchestration folder. I am pretty sure that it's not working because is trying to find the Resources.json and retorchCI.files (as well as the docker-compose.yml) in the orchestration folder (not in retorch-st-fullteaching). I can change the orchestration generator to provide the basepath, and allow the use case that you're trying.
This PR is related and closes #58 contains the following changes: