OOM when master node forwards large file upload request to static node

[mycxu]

I tried to upload a 2GB file, but when my master node forwards the upload request to the static node, the following error occurs：

Caused by: java.lang.OutOfMemoryError: Required array size too large: 2147483648 = 2147483648 - 0
	at java.base/sun.nio.ch.ChannelInputStream.readAllBytes(ChannelInputStream.java:122) ~[na:na]
	at org.springframework.util.FileCopyUtils.copyToByteArray(FileCopyUtils.java:149) ~[spring-core-6.0.12.jar:6.0.12]
	at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest$StandardMultipartFile.getBytes(StandardMultipartHttpServletRequest.java:244) ~[spring-web-6.0.12.jar:6.0.12]
	at org.eclipse.jifa.server.service.impl.WorkerServiceImpl.forwardUploadRequestToStaticWorker(WorkerServiceImpl.java:215) ~[main/:na]
	at org.eclipse.jifa.server.service.impl.FileServiceImpl.handleUploadRequest(FileServiceImpl.java:233) ~[main/:na]
	at org.eclipse.jifa.server.controller.FileController.upload(FileController.java:122) ~[main/:na]
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:578) ~[na:na]
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-6.0.12.jar:6.0.12]
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) ~[spring-web-6.0.12.jar:6.0.12]
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118) ~[spring-webmvc-6.0.12.jar:6.0.12]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:884) ~[spring-webmvc-6.0.12.jar:6.0.12]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797) ~[spring-webmvc-6.0.12.jar:6.0.12]
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-6.0.12.jar:6.0.12]
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1081) ~[spring-webmvc-6.0.12.jar:6.0.12]
	... 79 common frames omitted

I have traced the error to occur within the file.getBytes() call in the following function：

jifa/server/src/main/java/org/eclipse/jifa/server/service/impl/WorkerServiceImpl.java

Lines 212 to 245 in feb93c9

	public long forwardUploadRequestToStaticWorker(StaticWorkerEntity worker, FileType type, MultipartFile file) throws Throwable {
	MultipartBodyBuilder builder = new MultipartBodyBuilder();
	try {
	builder.part("file", new ByteArrayResource(file.getBytes()))
	.filename(file.getOriginalFilename() != null ? file.getOriginalFilename() : Constant.DEFAULT_FILENAME)
	.contentType(MediaType.APPLICATION_OCTET_STREAM);
	builder.part("type", type.name());
	} catch (IOException e) {
	throw new RuntimeException(e);
	}
	UriBuilder uriBuilder = new DefaultUriBuilderFactory().builder()
	.scheme("http")
	.host(worker.getHostAddress())
	.port(worker.getPort())
	.path(HTTP_API_PREFIX + "/files/upload");
	WebClient.RequestBodySpec spec = webClient.method(HttpMethod.POST)
	.uri(uriBuilder.build());
	String jwtToken = userService.getCurrentUserJwtTokenOrNull();
	if (jwtToken != null) {
	spec.header(HttpHeaders.AUTHORIZATION, "Bearer " + jwtToken);
	}
	return spec.contentType(MediaType.MULTIPART_FORM_DATA)
	.body(BodyInserters.fromMultipartData(builder.build()))
	.exchangeToMono(response -> {
	if (!response.statusCode().is2xxSuccessful()) {
	return response.createError();
	}
	return response.bodyToMono(String.class).map(s -> GSON.fromJson(s, Long.class));
	}).toFuture().get();
	}

Is there a way to solve it?

[D-D-H]

@mycxu

Could you try to make the following change to see if the problem can be resolved?

         builder.part("file", new ByteArrayResource(file.getBytes()))

->

         builder.part("file", new MultipartFileResource(file))

[mycxu]

@D-D-H
I solved it by using:

 builder.part("file", file.getReource())

Thx

[D-D-H]

Please feel free to submit a PR to fix it.

[mycxu]

PR: #310