Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

changed Dockerfile for airflow, fuseki and digest-api.Also changed requirements.txt #546

Merged
merged 1 commit into from
Jul 31, 2024
+50 −36
Merged

changed Dockerfile for airflow, fuseki and digest-api.Also changed requirements.txt #546

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 10 additions & 17 deletions infra/airflow-cluster/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,15 @@ FROM docker.io/apache/airflow:2.5.1-python3.8

# quick sudo
USER root
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A8D3785C
RUN apt-get update && apt-get install make && apt-get install build-essential git -y && apt-get install -y wget && apt-get install -y unzip
RUN apt install -y default-jre

RUN apt-get update && apt-get install -y \
build-essential=12.9 \
default-jre=2:1.11-72 \
git=1:2.30.2-1+deb11u2 \
make=4.3-4.1 \
wget=1.21-1+deb11u1 \
unzip=6.0-26+deb11u1 \
&& rm -rf /var/lib/apt/lists/*

# back to normal user
USER airflow
Expand All @@ -14,20 +20,7 @@ COPY requirements.txt /opt/airflow

# working in the /opt/airflow
WORKDIR /opt/airflow
RUN mkdir -p ./.rmlmapper ./dags ./ted_sws

COPY ted_sws /opt/airflow/ted_sws

COPY dags /opt/airflow/dags

RUN wget -c https://github.com/RMLio/rmlmapper-java/releases/download/v6.2.2/rmlmapper-6.2.2-r371-all.jar -O ./.rmlmapper/rmlmapper.jar


RUN wget -c https://github.com/Saxonica/Saxon-HE/releases/download/SaxonHE10-9/SaxonHE10-9J.zip -P .saxon/
RUN cd .saxon && unzip SaxonHE10-9J.zip && rm -rf SaxonHE10-9J.zip

RUN mkdir -p ./.limes
RUN wget -c https://github.com/dice-group/LIMES/releases/download/1.7.9/limes.jar -P ./.limes
RUN mkdir -p ./dags ./ted_sws


RUN pip install --upgrade pip
Expand Down
27 changes: 10 additions & 17 deletions infra/airflow/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,15 @@ FROM docker.io/apache/airflow:2.5.1-python3.8

# quick sudo
USER root
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A8D3785C
RUN apt-get update && apt-get install make && apt-get install build-essential git -y && apt-get install -y wget && apt-get install -y unzip
RUN apt install -y default-jre

RUN apt-get update && apt-get install -y \
build-essential=12.9 \
default-jre=2:1.11-72 \
git=1:2.30.2-1+deb11u2 \
make=4.3-4.1 \
wget=1.21-1+deb11u1 \
unzip=6.0-26+deb11u1 \
&& rm -rf /var/lib/apt/lists/*

# back to normal user
USER airflow
Expand All @@ -14,20 +20,7 @@ COPY requirements.txt /opt/airflow

# working in the /opt/airflow
WORKDIR /opt/airflow
RUN mkdir -p ./.rmlmapper ./dags ./ted_sws

COPY ted_sws /opt/airflow/ted_sws

COPY dags /opt/airflow/dags

RUN wget -c https://github.com/RMLio/rmlmapper-java/releases/download/v6.2.2/rmlmapper-6.2.2-r371-all.jar -O ./.rmlmapper/rmlmapper.jar


RUN wget -c https://github.com/Saxonica/Saxon-HE/releases/download/SaxonHE10-9/SaxonHE10-9J.zip -P .saxon/
RUN cd .saxon && unzip SaxonHE10-9J.zip && rm -rf SaxonHE10-9J.zip

RUN mkdir -p ./.limes
RUN wget -c https://github.com/dice-group/LIMES/releases/download/1.7.9/limes.jar -P ./.limes
RUN mkdir -p ./dags ./ted_sws


RUN pip install --upgrade pip
Expand Down
9 changes: 9 additions & 0 deletions infra/digest_api/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,13 @@ RUN pip install --upgrade pip
RUN pip install -r digest_service/requirements.txt
RUN pip install -r digest_service/project_requirements.txt

# Create a non-root user and group
RUN groupadd -r appuser && useradd -r -g appuser appuser

# Set the appropriate permissions for the directories and files
RUN chown -R appuser:appuser /usr/src/api/id_manager

# Switch to the non-root user
USER appuser

ENTRYPOINT uvicorn --host 0.0.0.0 --port 8000 ted_sws.notice_transformer.entrypoints.api.digest_service.main:app
20 changes: 19 additions & 1 deletion infra/fuseki/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,22 @@ FROM docker.io/secoresearch/fuseki:4.5.0

USER root
RUN apt-get update; \
apt-get install -y --no-install-recommends procps
apt-get install -y --no-install-recommends procps


# Create a non-root user and group
RUN groupadd -r fuseki && useradd -r -g fuseki fuseki

# Ensure the /fuseki-base and /fuseki-data directories exist and set permissions
RUN mkdir -p /fuseki-base /fuseki-data/databases /tmp && \
chown -R fuseki:fuseki /fuseki-base /fuseki-data /tmp

# Switch to the non-root user
USER fuseki







3 changes: 2 additions & 1 deletion requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ pydantic~=1.9.0
requests~=2.28.2
deepdiff~=5.7.0
jinja2~=3.1.2
python-dotenv~=0.19.2
python-dotenv~=0.21.0
pymongo~=4.0.1
apache-airflow~=2.5.1
hvac==0.11.2
Expand All @@ -27,3 +27,4 @@ certifi~=2022.12.7
shortuuid~=1.0.11
pendulum~=2.1.2
saxonche~=12.4
elasticsearch-dbapi==0.2.11
Loading