Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multi-threading during tests #41

Open
kkrugler opened this issue Sep 20, 2017 · 1 comment
Open

Support multi-threading during tests #41

kkrugler opened this issue Sep 20, 2017 · 1 comment

Comments

@kkrugler
Copy link
Member

Currently our static UrlLogger will have an issue if multiple tests that use it are run at the same time.

One option is to get the job name in any function that needs to log, and have it pass that in the logging request. Then the UrlLoggerImpl could segment results by this key (and provide a clear(key) call).

Each test that relies on logging would need to call UrlLogger.clear(test name), and ensure that the job being run sets the job name to be the test name.

@Schmed
Copy link
Member

Schmed commented Sep 28, 2017

I poked at this a little. The problem is getting at the job name from within a function. We've got it in the CrawlTopology, and it gets passed to the ExecutionEnvironment, but it doesn't seem to be accessible from the ExecutionEnvironment, let alone anything actually accessible to a function.

I was hoping that we didn't have to add jobName as a parameter in the constructor of every single function we're using.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants