RESOURCE_EXHAUSTED: Received message larger than max (4224741 vs. 4194304)

[Longxiang-Bai]

Hi, we are using zetasql 2023.04.1 and we are facing the following error:
com.google.zetasql.io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: Received message larger than max (4224741 vs. 4194304)
when processing a huge sql with Analyzer.extractTableNamesFromScript().
I am wondering if anyone could advise how can we make it work? Thanks!

[matthewcbrown]

Getting this error on extractTableNamesFromScript is very unexpected, since it doesn't actually serialize the AST. I would expect this on almost any other API call, are you sure it's that call that is failing?

I assume the raw sql itself doesn't acceed this size?

[Longxiang-Bai]

Hi @matthewcbrown, thanks for your reply. I am pretty sure it is this API. Since our sql has over 400k parameters lol. So it is possible it actually exceeds the size. Do you have any comment or suggestions?

[kylase]

I experienced this error as well. However, the circumstances which it arises is different.

We are only running a simple SELECT 1 statement with ZetaSQL Toolkit, but with minor difference with regards how the catalog is build with.

If we build the catalog with a project with a lot of tables (415 tables) (:

catalog.addAllTablesInProject("projectId")

(note that this is ZetaSQL Toolkit API and what it does is to populate the SimpleCatalog)

We will encounter similar error:

Exception in thread "main" com.google.zetasql.toolkit.AnalysisException: Received message larger than max (4266687 vs. 4194304)
	at com.google.zetasql.toolkit.ZetaSQLToolkitAnalyzer$StatementAnalyzer.analyzeNextStatement(ZetaSQLToolkitAnalyzer.java:232)
	at com.google.zetasql.toolkit.ZetaSQLToolkitAnalyzer$StatementAnalyzer.next(ZetaSQLToolkitAnalyzer.java:211)
	at com.google.zetasql.toolkit.ZetaSQLToolkitAnalyzer$StatementAnalyzer.next(ZetaSQLToolkitAnalyzer.java:148)
	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
	at com.google.zetasql.toolkit.examples.AnalyzeBigQuery.main(AnalyzeBigQuery.java:79)
Caused by: com.google.zetasql.SqlException: Received message larger than max (4266687 vs. 4194304)
	at com.google.zetasql.Analyzer.analyzeNextStatement(Analyzer.java:220)
	at com.google.zetasql.toolkit.ZetaSQLToolkitAnalyzer$StatementAnalyzer.analyzeNextStatement(ZetaSQLToolkitAnalyzer.java:229)
	... 4 more
Caused by: com.google.zetasql.io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: Received message larger than max (4266687 vs. 4194304)
	at com.google.zetasql.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:262)
	at com.google.zetasql.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:243)
	at com.google.zetasql.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:156)
	at com.google.zetasql.ZetaSqlLocalServiceGrpc$ZetaSqlLocalServiceBlockingStub.analyze(ZetaSqlLocalServiceGrpc.java:1506)
	at com.google.zetasql.Analyzer.analyzeNextStatement(Analyzer.java:218)
	... 5 more

If the catalog has small number of tables (45 in our case), this error will not occur.