Skip to content

Commit

Permalink
[Fix](case) Ensure Kerberos starts after other components (apache#46361)
Browse files Browse the repository at this point in the history
### What problem does this PR solve?

```
Init kerberos test data
WARNING: Use "yarn jar" to launch YARN applications.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/3.1.0.0-78/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://localhost:10000/default;password=root;principal=hive/[email protected];user=root
25/01/03 13:37:16 [main]: WARN jdbc.HiveConnection: Failed to connect to localhost:10000
Could not open connection to the HS2 server. Please check the server URI and if the URI is correct, then ask the administrator to check the server status.
Error: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000/default;password=root;principal=hive/[email protected];user=root: java.net.ConnectException: Connection refused (Connection refused) (state=08S01,code=0)
beeline> CREATE DATABASE IF NOT EXISTS `test_krb_hive_db`;
```
Kerberos initialization depends on the startup of some other components
To avoid potential resource conflicts during parallel startup

1. Keep parallel startup for other components
2. Wait for all parallel components to complete
3. Start Kerberos sequentially after other components
4. Add error checking and logging for Kerberos startup
  • Loading branch information
CalvinKirs authored Jan 5, 2025
1 parent 26d68d7 commit 4c40b4e
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 5 deletions.
2 changes: 2 additions & 0 deletions docker/thirdparties/docker-compose/kerberos/kerberos.yaml.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ services:
- ./common/hadoop/hadoop-run.sh:/usr/local/hadoop-run.sh
- ./health-checks/hadoop-health-check.sh:/etc/health.d/hadoop-health-check.sh
- ./entrypoint-hive-master.sh:/usr/local/entrypoint-hive-master.sh
restart: on-failure
hostname: hadoop-master
entrypoint: /usr/local/entrypoint-hive-master.sh
healthcheck:
Expand All @@ -45,6 +46,7 @@ services:
image: doristhirdpartydocker/trinodb:hdp3.1-hive-kerberized-2_96
container_name: doris--kerberos2
hostname: hadoop-master-2
restart: on-failure
volumes:
- ./two-kerberos-hives:/keytabs
- ./sql:/usr/local/sql
Expand Down
16 changes: 11 additions & 5 deletions docker/thirdparties/run-thirdparties-docker.sh
Original file line number Diff line number Diff line change
Expand Up @@ -703,11 +703,6 @@ if [[ "${RUN_LAKESOUL}" -eq 1 ]]; then
pids["lakesoul"]=$!
fi

if [[ "${RUN_KERBEROS}" -eq 1 ]]; then
start_kerberos > start_kerberos.log 2>&1 &
pids["kerberos"]=$!
fi

if [[ "${RUN_MINIO}" -eq 1 ]]; then
start_minio > start_minio.log 2>&1 &
pids["minio"]=$!
Expand All @@ -727,4 +722,15 @@ for compose in "${!pids[@]}"; do
fi
done

if [[ "${RUN_KERBEROS}" -eq 1 ]]; then
echo "Starting Kerberos after all other components..."
start_kerberos > start_kerberos.log 2>&1
if [ $? -ne 0 ]; then
echo "Kerberos startup failed"
cat start_kerberos.log
exit 1
fi
fi
echo "docker started"
docker ps -a --format "{{.ID}} | {{.Image}} | {{.Status}}"
echo "all dockers started successfully"

0 comments on commit 4c40b4e

Please sign in to comment.