Fixes issue which can be caused when non-utf-8 files are uploaded

DuckDB requires CSV to be encoded as UTF-8, additionally its a requirement of the welsh languague toolkit. However most of the tools used in the Welsh Gov will output ISO-8859-2. So this adds a bit of code to try to do the conversation before passing to the backend.
Marvell-Consulting · Dec 19, 2024 · 5fd939e · 5fd939e
1 parent 4527afa
commit 5fd939e
Show file tree

Hide file tree

Showing 4 changed files with 2,727 additions and 150 deletions.
diff --git a/Dockerfile b/Dockerfile
@@ -1,6 +1,7 @@
 # Dockerfile
 
 FROM node:20
+RUN apt update && apt install build-essential
 
 WORKDIR /app