Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replaces importDump.php with PagePort import script #212

Merged
merged 4 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM debian:12.5 as base

Check warning on line 1 in Dockerfile

View workflow job for this annotation

GitHub Actions / push

The 'as' keyword should match the case of the 'from' keyword

FromAsCasing: 'as' and 'FROM' keywords' casing do not match More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/

LABEL maintainers="[email protected],[email protected]"
LABEL org.opencontainers.image.source=https://github.com/WikiTeq/Taqasta
Expand All @@ -16,7 +16,7 @@
APACHE_LOG_DIR=/var/log/apache2

# System setup
RUN set x; \

Check failure on line 19 in Dockerfile

View workflow job for this annotation

GitHub Actions / test

DL3047 info: Avoid use of wget without progress bar. Use `wget --progress=dot:giga <url>`.Or consider using `-q` or `-nv` (shorthands for `--quiet` or `--no-verbose`).
apt-get clean \
&& apt-get update \
&& apt-get --no-install-recommends install -y aptitude \
Expand Down Expand Up @@ -113,11 +113,11 @@
&& mkdir -p $MW_VOLUME

# Composer
RUN set -x; \

Check failure on line 116 in Dockerfile

View workflow job for this annotation

GitHub Actions / test

DL4001 warning: Either use Wget or Curl but not both
curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer \
&& composer self-update 2.1.3

FROM base as core

Check warning on line 120 in Dockerfile

View workflow job for this annotation

GitHub Actions / push

The 'as' keyword should match the case of the 'from' keyword

FromAsCasing: 'as' and 'FROM' keywords' casing do not match More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/
# MediaWiki core
RUN set -x; \
git clone --depth 1 -b $MW_CORE_VERSION https://gerrit.wikimedia.org/r/mediawiki/core.git $MW_HOME \
Expand All @@ -139,7 +139,7 @@
cd $MW_HOME \
&& find . \( -name ".git" -o -name ".gitignore" -o -name ".gitmodules" -o -name ".gitattributes" \) -exec rm -rf -- {} +

FROM base as skins

Check warning on line 142 in Dockerfile

View workflow job for this annotation

GitHub Actions / push

The 'as' keyword should match the case of the 'from' keyword

FromAsCasing: 'as' and 'FROM' keywords' casing do not match More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/
# Skins
# The Minerva Neue, MonoBook, Timeless, Vector and Vector 2022 skins are bundled into MediaWiki and do not need to be
# separately installed.
Expand Down Expand Up @@ -179,7 +179,7 @@
cd $MW_HOME/skins \
&& find . \( -name ".git" -o -name ".gitignore" -o -name ".gitmodules" -o -name ".gitattributes" \) -exec rm -rf -- {} +

FROM base as extensions

Check warning on line 182 in Dockerfile

View workflow job for this annotation

GitHub Actions / push

The 'as' keyword should match the case of the 'from' keyword

FromAsCasing: 'as' and 'FROM' keywords' casing do not match More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/
# Extensions
#
# The following extensions are bundled into MediaWiki and do not need to be separately installed (though in some cases
Expand Down Expand Up @@ -546,7 +546,7 @@
# PagePort
&& git clone --single-branch -b master https://github.com/WikiTeq/PagePort.git $MW_HOME/extensions/PagePort \
&& cd $MW_HOME/extensions/PagePort \
&& git checkout -q a6b800c9b3f58c151cdda4ec2f1aa396536c3a7d
&& git checkout -q 65cb6908c852c5483efa5f8022b8bb97b5adb208

# R
RUN set -x; \
Expand Down Expand Up @@ -977,7 +977,7 @@
cd $MW_HOME/extensions \
&& find . \( -name ".git" -o -name ".gitignore" -o -name ".gitmodules" -o -name ".gitattributes" \) -exec rm -rf -- {} +

FROM base as composer

Check warning on line 980 in Dockerfile

View workflow job for this annotation

GitHub Actions / push

The 'as' keyword should match the case of the 'from' keyword

FromAsCasing: 'as' and 'FROM' keywords' casing do not match More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/

# Copy core, skins and extensions
COPY --from=core $MW_HOME $MW_HOME
Expand Down Expand Up @@ -1012,13 +1012,13 @@
&& ln -s $MW_VOLUME/images $MW_HOME/images \
&& ln -s $MW_VOLUME/cache $MW_HOME/cache

FROM base as final

Check warning on line 1015 in Dockerfile

View workflow job for this annotation

GitHub Actions / push

The 'as' keyword should match the case of the 'from' keyword

FromAsCasing: 'as' and 'FROM' keywords' casing do not match More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/

COPY --from=composer $MW_HOME $MW_HOME
COPY --from=composer $MW_ORIGIN_FILES $MW_ORIGIN_FILES

# Default values
ENV MW_AUTOUPDATE=true \

Check warning on line 1021 in Dockerfile

View workflow job for this annotation

GitHub Actions / push

Sensitive data should not be used in the ARG or ENV commands

SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ENV "MW_PASSWORD_SENDER") More info: https://docs.docker.com/go/dockerfile/rule/secrets-used-in-arg-or-env/
MW_MAINTENANCE_UPDATE=0 \
MW_ENABLE_EMAIL=0 \
MW_ENABLE_USER_EMAIL=0 \
Expand Down Expand Up @@ -1059,7 +1059,8 @@
MEDIAWIKI_MAINTENANCE_AUTO_ENABLED=false \
MW_SENTRY_DSN="" \
MW_USE_CACHE_DIRECTORY=1 \
APACHE_REMOTE_IP_HEADER=X-Forwarded-For
APACHE_REMOTE_IP_HEADER=X-Forwarded-For \
MW_AUTO_IMPORT=1

COPY _sources/configs/msmtprc /etc/
COPY _sources/configs/mediawiki.conf /etc/apache2/sites-enabled/
Expand Down
4 changes: 4 additions & 0 deletions _sources/canasta/DockerSettings.php
Original file line number Diff line number Diff line change
Expand Up @@ -606,6 +606,10 @@ function isEnvTrue( $name ): bool {
$wgUseCdn = false; // make sure profile is not cached
}

if ( getenv( 'MW_AUTO_IMPORT' ) ) {
wfLoadExtension( 'PagePort' );
}

# Include all php files in config/settings directory
foreach ( glob( getenv( 'MW_CONFIG_DIR' ) . '/settings/*.php' ) as $filename ) {
if ( is_readable( $filename ) ) {
Expand Down
29 changes: 11 additions & 18 deletions _sources/scripts/run-maintenance-scripts.sh
Original file line number Diff line number Diff line change
Expand Up @@ -361,25 +361,16 @@ run_autoupdate () {
}

run_import () {
# Import dumps if any
# Import PagePort dumps if any
if mountpoint -q -- "$MW_IMPORT_VOLUME"; then
echo "Found $MW_IMPORT_VOLUME, checking for XML dump presence.."
local FILES_IMPORTED=0
for filename in $MW_IMPORT_VOLUME/*.xml; do
echo "Found $filename file, importing.."
php maintenance/importDump.php \
--username-prefix="" \
--memory-limit=max \
--report 1 \
"$filename"
FILES_IMPORTED=$((FILES_IMPORTED + 1))
done
if [ "$FILES_IMPORTED" -gt "0" ]; then
echo "Running post-import maintenance scripts.."
php maintenance/rebuildrecentchanges.php
php maintenance/initSiteStats.php
echo "Found $MW_IMPORT_VOLUME, running PagePort import.."
XML_TEST=($(find $MW_IMPORT_VOLUME -maxdepth 1 -name "*.xml"))
if [ ${#XML_TEST[@]} -gt 0 ]; then
echo "WARNING! The directory contains XML files, did you forget to migrate the dump to PagePort?"
return
fi
echo "Imported $FILES_IMPORTED files!"
php extensions/PagePort/maintenance/importPages.php --source "$MW_IMPORT_VOLUME" --user 'Maintenance script'
echo "Imported completed!"
fi
}

Expand All @@ -391,7 +382,9 @@ else
fi

# Run import after install and update are completed
run_import
if isTrue "$MW_AUTO_IMPORT"; then
run_import
fi

jobrunner &
transcoder &
Expand Down
Loading