Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Java: Add Logger #1422

Merged
merged 38 commits into from
Jul 2, 2024
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
fec36b1
Implement Logger for Java client
jonathanl-bq May 7, 2024
24af480
Run spotless
jonathanl-bq May 7, 2024
d3b2f81
Annotate NonNull for Level in public API
jonathanl-bq May 7, 2024
e21f773
Fix build error
jonathanl-bq May 7, 2024
a45ab3a
Merge branch 'main' into java/dev_lotjonat_logging
jonathanl-bq May 7, 2024
604066e
Fix clippy lint
jonathanl-bq May 7, 2024
18509b3
Update logger
jonathanl-bq May 13, 2024
a291ad1
Fix FFI function names.
Yury-Fridlyand May 13, 2024
73fc219
Remove %n from logs
jonathanl-bq May 13, 2024
c998a2c
Fix Logger test
jonathanl-bq May 13, 2024
2c3a1e4
Address Yury's PR comments
jonathanl-bq May 16, 2024
cba7ad2
Update java/src/lib.rs
jonathanl-bq May 16, 2024
a7062cd
Apply suggestions from code review
jonathanl-bq May 16, 2024
3a13ee2
Update docs and import Logger.Level
jonathanl-bq May 16, 2024
6bf8fed
Split out native function definitions to a new resolver
jonathanl-bq May 17, 2024
01e6b09
Remove static loader
jonathanl-bq May 17, 2024
7035243
Run Spotless
jonathanl-bq May 17, 2024
fcb4026
Change references to Rust core to Glide core
jonathanl-bq May 17, 2024
5094a4e
Spotless again
jonathanl-bq May 17, 2024
4b4bb44
Make minor documentation fixes
jonathanl-bq May 17, 2024
405f9a3
Merge pull request #265 from Bit-Quill/java/dev_lotjonat_logging
jonathanl-bq May 17, 2024
19b96cf
Merge branch 'main' into java/integ_lotjonat_logging
jonathanl-bq May 17, 2024
1eccd3c
Fix failing tests
jonathanl-bq May 17, 2024
0cfd92b
Run spotless
jonathanl-bq May 17, 2024
8c42435
Address PR comments
jonathanl-bq Jun 12, 2024
6c9b7ae
Apply Spotless
jonathanl-bq Jun 12, 2024
23c9957
Minor documentation fixes
jonathanl-bq Jun 12, 2024
b851598
Don't log when DISABLED log level is passed to log method
jonathanl-bq Jun 28, 2024
c9533fa
Resolve merge conflict and add panic handling for logger FFI code
jonathanl-bq Jul 1, 2024
593702e
Fix build errors and handle errors in FFI layer
jonathanl-bq Jul 1, 2024
9e75012
Make logger messages lazily evaluated
jonathanl-bq Jul 1, 2024
cc978ab
Resolve merge conflict
jonathanl-bq Jul 1, 2024
16aaeb0
Run Spotless
jonathanl-bq Jul 1, 2024
fea98cd
Add overload for log method
jonathanl-bq Jul 1, 2024
ed7fec6
Address Logger TODOs in MessageHandler
jonathanl-bq Jul 1, 2024
1ce4d60
Update java/client/src/main/java/glide/connectors/handlers/MessageHan…
jonathanl-bq Jul 1, 2024
fb3b75b
Use suppliers for logging in MessageHandler
jonathanl-bq Jul 1, 2024
d48f4fb
Address PR comments
jonathanl-bq Jul 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
193 changes: 193 additions & 0 deletions java/client/src/main/java/glide/api/logging/Logger.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
/** Copyright Valkey GLIDE Project Contributors - SPDX Identifier: Apache-2.0 */
package glide.api.logging;

import static glide.ffi.resolvers.LoggerResolver.initInternal;
import static glide.ffi.resolvers.LoggerResolver.logInternal;

import java.util.function.Supplier;
import lombok.Getter;
import lombok.NonNull;

/**
* A singleton class that allows logging which is consistent with logs from the internal rust core.
* The logger can be set up in 2 ways -
*
* <ol>
* <li>By calling <code>Logger.init</code>, which configures the logger only if it wasn't
* previously configured.
* <li>By calling <code>Logger.setLoggerConfig</code>, which replaces the existing configuration,
* and means that new logs will not be saved with the logs that were sent before the call.
* </ol>
*
* If <code>setLoggerConfig</code> wasn't called, the first log attempt will initialize a new logger
* with default configuration decided by Glide core.
*/
public final class Logger {
@Getter
public enum Level {
DISABLED(-2),
DEFAULT(-1),
ERROR(0),
WARN(1),
INFO(2),
DEBUG(3),
TRACE(4);

private final int level;

Level(int level) {
this.level = level;
}

public static Level fromInt(int i) {
switch (i) {
case 0:
return ERROR;
case 1:
return WARN;
case 2:
return INFO;
case 3:
return DEBUG;
case 4:
return TRACE;
default:
return DEFAULT;
}
}
}

@Getter private static Level loggerLevel;

private static void initLogger(@NonNull Level level, String fileName) {
if (level == Level.DISABLED) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we dont have DISABLED option in node and python - can you please open another PR that adds this option to the other wrappers too?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've started work on this. It's not ready yet and probably won't be in until post-GA, but there's a draft PR up here: Bit-Quill#399

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DISABLED option was added to Java side for testing, since we don't want to log when we have mocks. There's other ways to do it (mocking static objects is annoying, but possible), but this is the best way.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonathanl-bq adding it to redis-rs isn't urgent, you can open an issue and we'll do it post GA

loggerLevel = level;
return;
}
loggerLevel = Level.fromInt(initInternal(level.getLevel(), fileName));
}

/**
* Initialize a logger if it wasn't initialized before - this method is meant to be used when
* there is no intention to replace an existing logger. The logger will filter all logs with a
* level lower than the given level. If given a <code>fileName</code> argument, will write the
* logs to files postfixed with <code>fileName</code>. If <code>fileName</code> isn't provided,
* the logs will be written to the console.
*
* @param level Set the logger level to one of <code>
* [DISABLED, DEFAULT, ERROR, WARN, INFO, DEBUG, TRACE]
* </code>. If log level isn't provided, the logger will be configured with default
* configuration decided by Glide core.
* @param fileName If provided, the target of the logs will be the file mentioned. Otherwise, logs
* will be printed to the console.
*/
public static void init(@NonNull Level level, String fileName) {
if (loggerLevel == null) {
initLogger(level, fileName);
}
}

/**
* Initialize a logger if it wasn't initialized before - this method is meant to be used when
* there is no intention to replace an existing logger. The logger will filter all logs with a
* level lower than the default level decided by Glide core. Given a <code>fileName</code>
* argument, will write the logs to files postfixed with <code>fileName</code>.
*
* @param fileName The target of the logs will be the file mentioned.
*/
public static void init(@NonNull String fileName) {
init(Level.DEFAULT, fileName);
}

/**
* Initialize a logger if it wasn't initialized before - this method is meant to be used when
* there is no intention to replace an existing logger. The logger will filter all logs with a
* level lower than the default level decided by Glide core. The logs will be written to stdout.
*/
public static void init() {
init(Level.DEFAULT, null);
}

/**
* Initialize a logger if it wasn't initialized before - this method is meant to be used when
* there is no intention to replace an existing logger. The logger will filter all logs with a
* level lower than the given level. The logs will be written to stdout.
*
* @param level Set the logger level to one of <code>[DEFAULT, ERROR, WARN, INFO, DEBUG, TRACE]
jonathanl-bq marked this conversation as resolved.
Show resolved Hide resolved
* </code>. If log level isn't provided, the logger will be configured with default
* configuration decided by Glide core.
*/
public static void init(@NonNull Level level) {
init(level, null);
}

/**
* Logs the provided message if the provided log level is lower then the logger level.
*
* @param level The log level of the provided message.
* @param logIdentifier The log identifier should give the log a context.
* @param message The message to log.
*/
public static void log(
acarbonetto marked this conversation as resolved.
Show resolved Hide resolved
acarbonetto marked this conversation as resolved.
Show resolved Hide resolved
@NonNull Level level, @NonNull String logIdentifier, @NonNull Supplier<String> message) {
if (loggerLevel == null) {
initLogger(Level.DEFAULT, null);
}

if (level == Level.DISABLED) {
return;
}

if (!(level.getLevel() <= loggerLevel.getLevel())) {
return;
}
logInternal(level.getLevel(), logIdentifier, message.get());
}

/**
* Creates a new logger instance and configure it with the provided log level and file name.
*
* @param level Set the logger level to one of <code>
* [DISABLED, DEFAULT, ERROR, WARN, INFO, DEBUG, TRACE]
* </code>. If log level isn't provided, the logger will be configured with default
* configuration decided by Glide core.
* @param fileName If provided, the target of the logs will be the file mentioned. Otherwise, logs
* will be printed to stdout.
*/
public static void setLoggerConfig(@NonNull Level level, String fileName) {
initLogger(level, fileName);
}

/**
* Creates a new logger instance and configure it with the provided log level. The logs will be
* written to stdout.
*
* @param level Set the logger level to one of <code>
* [DISABLED, DEFAULT, ERROR, WARN, INFO, DEBUG, TRACE]
* </code>. If log level isn't provided, the logger will be configured with default
* configuration decided by Glide core.
*/
public static void setLoggerConfig(@NonNull Level level) {
setLoggerConfig(level, null);
}

/**
* Creates a new logger instance and configure it with the provided file name and default log
* level. The logger will filter all logs with a level lower than the default level decided by the
* Glide core.
*
* @param fileName If provided, the target of the logs will be the file mentioned. Otherwise, logs
* will be printed to stdout.
*/
public static void setLoggerConfig(String fileName) {
setLoggerConfig(Level.DEFAULT, fileName);
}

/**
* Creates a new logger instance. The logger will filter all logs with a level lower than the
* default level decided by Glide core. The logs will be written to stdout.
*/
public static void setLoggerConfig() {
setLoggerConfig(Level.DEFAULT, null);
}
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
/** Copyright Valkey GLIDE Project Contributors - SPDX Identifier: Apache-2.0 */
package glide.connectors.handlers;

import static glide.api.logging.Logger.Level.ERROR;

import glide.api.logging.Logger;
import glide.api.models.exceptions.ClosingException;
import glide.api.models.exceptions.ConnectionException;
import glide.api.models.exceptions.ExecAbortException;
Expand Down Expand Up @@ -115,11 +118,15 @@ public void completeRequest(Response response) {
}
future.completeAsync(() -> response);
} else {
// TODO: log an error thru logger.
// probably a response was received after shutdown or `registerRequest` call was missing
System.err.printf(
"Received a response for not registered callback id %d, request error = %s%n",
callbackId, response.getRequestError());
Logger.log(
ERROR,
"callback dispatcher",
() ->
"Received a response for not registered callback id "
+ callbackId
+ ", request error = "
+ response.getRequestError());
distributeClosingException("Client is in an erroneous state and should close");
}
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
/** Copyright Valkey GLIDE Project Contributors - SPDX Identifier: Apache-2.0 */
package glide.connectors.handlers;

import static glide.api.logging.Logger.Level.ERROR;

import glide.api.logging.Logger;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import lombok.NonNull;
Expand Down Expand Up @@ -29,9 +32,7 @@ public void channelRead(@NonNull ChannelHandlerContext ctx, @NonNull Object msg)
/** Handles uncaught exceptions from {@link #channelRead(ChannelHandlerContext, Object)}. */
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
// TODO: log thru logger
System.out.printf("=== exceptionCaught %s %s %n", ctx, cause);
cause.printStackTrace();
Logger.log(ERROR, "read handler", () -> "=== exceptionCaught " + ctx + " " + cause);

callbackDispatcher.distributeClosingException(
"An unhandled error while reading from UDS channel: " + cause);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
/** Copyright Valkey GLIDE Project Contributors - SPDX Identifier: Apache-2.0 */
package glide.ffi.resolvers;

public class LoggerResolver {
// TODO: consider lazy loading the glide_rs library
static {
NativeUtils.loadGlideLib();
}

public static native int initInternal(int level, String fileName);

public static native void logInternal(int level, String logIdentifier, String message);
}
1 change: 1 addition & 0 deletions java/client/src/main/java/module-info.java
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
module glide.api {
exports glide.api;
exports glide.api.commands;
exports glide.api.logging;
exports glide.api.models;
exports glide.api.models.commands;
exports glide.api.models.commands.bitmap;
Expand Down
12 changes: 12 additions & 0 deletions java/client/src/test/java/glide/ExceptionHandlingTests.java
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,15 @@
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.Mockito.mockStatic;
import static redis_request.RedisRequestOuterClass.RequestType.CustomCommand;
import static response.ResponseOuterClass.RequestErrorType.Disconnect;
import static response.ResponseOuterClass.RequestErrorType.ExecAbort;
import static response.ResponseOuterClass.RequestErrorType.Timeout;
import static response.ResponseOuterClass.RequestErrorType.Unspecified;

import connection_request.ConnectionRequestOuterClass;
import glide.api.logging.Logger;
import glide.api.models.configuration.RedisClientConfiguration;
import glide.api.models.exceptions.ClosingException;
import glide.api.models.exceptions.ConnectionException;
Expand All @@ -34,26 +36,36 @@
import java.util.concurrent.ExecutionException;
import java.util.stream.Stream;
import lombok.SneakyThrows;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;
import org.mockito.MockedStatic;
import redis_request.RedisRequestOuterClass.RedisRequest;
import response.ResponseOuterClass.RequestError;
import response.ResponseOuterClass.RequestErrorType;
import response.ResponseOuterClass.Response;

public class ExceptionHandlingTests {
private MockedStatic<Logger> mockedLogger;

@BeforeEach
public void init() {
mockedLogger = mockStatic(Logger.class);
var threadPoolResource = ThreadPoolResourceAllocator.getOrCreate(() -> null);
if (threadPoolResource != null) {
threadPoolResource.getEventLoopGroup().shutdownGracefully();
ThreadPoolResourceAllocator.getOrCreate(() -> null);
}
}

@AfterEach
public void teardown() {
mockedLogger.close();
}

/**
* This test shows how exception handling works in the middle of future pipeline The client has
* similar stuff, but it rethrows an exception.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
import connection_request.ConnectionRequestOuterClass.ConnectionRequest;
import connection_request.ConnectionRequestOuterClass.NodeAddress;
import glide.api.RedisClient;
import glide.api.logging.Logger;
import glide.api.models.exceptions.ClosingException;
import glide.connectors.handlers.CallbackDispatcher;
import glide.connectors.handlers.ChannelHandler;
Expand All @@ -37,6 +38,7 @@ public class ConnectionWithGlideMockTests extends RustCoreLibMockTestBase {
@BeforeEach
@SneakyThrows
public void createTestClient() {
Logger.setLoggerConfig(Logger.Level.DISABLED);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why disabled?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We needed to disable logging to get these tests to work. They're only unit tests and shouldn't be using a real logger anyway.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right - this is why we need a DISABLED level logger.
Can we add a TODO to add this level to the redis-core side?

channelHandler =
new ChannelHandler(
new CallbackDispatcher(null),
Expand Down
1 change: 0 additions & 1 deletion java/integTest/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,6 @@ tasks.withType(Test) {
events "started", "skipped", "passed", "failed"
showStandardStreams true
}
jvmArgs "-Djava.library.path=${project.rootDir}/target/release"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is no longer needed now that we use the NativeLibLoader. It would've been better to put this in the PR that had the changes to publish to local Maven repository, but that's already been merged.

afterTest { desc, result ->
logger.quiet "${desc.className}.${desc.name}: ${result.resultType} ${(result.getEndTime() - result.getStartTime())/1000}s"
}
Expand Down
Loading
Loading