Skip to content

Commit

Permalink
Merge pull request #564 from jburke-cadc/CADC-13234
Browse files Browse the repository at this point in the history
CADC-13234 fix ringhold for ArtifactDAO API change
  • Loading branch information
pdowler authored Apr 3, 2024
2 parents bf7c302 + a0f5f27 commit 1cf7ea5
Show file tree
Hide file tree
Showing 7 changed files with 294 additions and 179 deletions.
5 changes: 2 additions & 3 deletions .github/workflows/gradle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,6 @@ jobs:
- name: java build -- ratik
run: cd ratik && ../gradlew --info clean build javadoc checkstyleMain

## disabled until updated for cadc-inventory-db API changes
# - name: java build -- ringhold
# run: cd ringhold && ../gradlew --info clean build javadoc checkstyleMain
- name: java build -- ringhold
run: cd ringhold && ../gradlew --info clean build javadoc checkstyleMain

25 changes: 17 additions & 8 deletions ringhold/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Storage Inventory local artifact removal process (ringhold)
# Storage Inventory local artifact deletion process (ringhold)

Process to remove local artifacts that are no longer being synchronised by fenwick. This tool is used
to perform quick cleanup at a storage site after changing the fenwick artifact-filter policy.
Process to remove the local copy of artifacts from a storage site inventory database and
generate DeletedStorageLocationEvent(s) so the removal will propagate correctly to a global inventory.
This does not remove the files from storage (see `tantar`).

## configuration
See the [cadc-java](https://github.com/opencadc/docker-base/tree/master/cadc-java) image docs for general config requirements.
Expand All @@ -18,17 +19,25 @@ org.opencadc.ringhold.inventory.schema={schema for inventory database objects}
org.opencadc.ringhold.inventory.username={username for inventory admin}
org.opencadc.ringhold.inventory.password={password for inventory admin}
org.opencadc.ringhold.inventory.url=jdbc:postgresql://{server}/{database}
# artifact namespace(s) to remove
org.opencadc.ringhold.namespace={storage site namespace}
# artifact uri bucket filter (optional)
org.opencadc.ringhold.buckets={uriBucket prefix or range of prefixes}
```
The `inventory` account owns and manages all the content (insert, update, delete) in the inventory schema. Unlike
other components that modify inventory content, this component **does not initialise** the database objects because
it never makes sense to run this in a new/empty database. The database is specified in the JDBC URL. Failure to
connect to a pre-initialised database will show up in logs.

### artifact-deselector.sql
Contains a SQL clause used as a WHERE constraint. The clause returns Artifact's that match the URI pattern.
```
WHERE uri LIKE 'cadc:CFHT/%'
```
The `namespace` is the prefix of the Artifact URI's to be deleted. The `namespace` must end with a colon (:)
or slash (/) so one namespace cannot accidentally match (be a prefix of) another namespace. Multiple values
of `namespace` may be specified, one per line.

The `buckets` value indicates a subset of artifacts to delete. The range of uri bucket prefixes is specified
with two values separated by a single - (dash) character; whitespace is ignored. Multiple instances of `ringhold`
can be run (in parallel) to subdivide the work as long as the range of buckets do not overlap.

## building it
```
Expand Down
2 changes: 1 addition & 1 deletion ringhold/VERSION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## deployable containers have a semantic and build tag
# semantic version tag: major.minor[.patch]
# build version tag: timestamp
VER=0.2.0
VER=0.3.0
TAGS="${VER} ${VER}-$(date --utc +"%Y%m%dT%H%M%S")"
unset VER
Loading

0 comments on commit 1cf7ea5

Please sign in to comment.