Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + + + + +Top containers replace Container 1s. Unlike Container 1s in the current version of ArchivesSpace, top containers in the upcoming version can be defined once and linked many times to various archival objects, resources, and accessions.
+ +The ability to create container profiles and associate them with top containers. Optional container profiles allow you to track information about the containers themselves, including dimensions.
+ +Extent calculator. In conjunction with container profiles, the new extent calculator allows you to easily see extents for accessions, resources, or resource components. Optionally, you can use the calculator to generate extent records for an accession, resource, or resource component.
+ +Bulk operations for containers. The Manage Top Containers area provides more efficient ways to work with multiple containers, including the ability to add or edit barcodes, change locations, and delete top containers in bulk.
+ +The ability to “share” boxes across collections in a meaningful way. You can define top containers separately from individual accessions and resources and access them from multiple accession and resource records. For example, this might be helpful for recording information about an oversize box that contains items from many collections.
+ +The ability to store data that will help you synchronize between ArchivesSpace and item records in your ILS. If your institution creates item records in its ILS for containers, you can now record that information within ArchivesSpace as well.
+ +The ability to store data about the restriction status of material associated with a container. You can now see at a glance whether any portion of the contents of a container is restricted.
+ +Machine-actionable restrictions. You will now have the ability to associate begin and end dates with “conditions governing access” and “conditions governing use” Notes. You’ll also be able to associate a local restriction type for non-time-bound restrictions. This gives the ability to better manage and re-describe expiring restrictions.
+ +For more information on using the new features, consult the user manual, particularly the new section titled Managing Containers (available late April 2016).
+ + +{{ site.description | default: site.github.project_tagline }}
- - {% if site.github.is_project_page %} -View the Project on GitHub {{ site.github.repository_nwo }}
- {% endif %} - - {% if site.github.is_user_page %} - - {% endif %} - - {% if site.show_downloads %} - - {% endif %} - - - - -Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + + + + +Performing regular backups of your MySQL database is critical. ArchivesSpace stores +all of your records data in the database, so as long as you have backups of your +database then you can always recover from errors and failures.
+ +If you are running MySQL, the mysqldump utility can dump the database
+schema and data to a file. It’s a good idea to run this with the
+--single-transaction option to avoid locking your database tables
+while your backups run. It is also essential to use the --routines
+flag, which will include functions and stored procedures in the
+backup. The mysqldump utility is widely used, and there are many tutorials
+available. As an example, something like this in your crontab would backup your
+database twice daily:
# Dump archivesspace database 6am and 6pm
+ 30 06,18 * * * mysqldump -u as -pas123 archivesspace | gzip > ~/backups/db.$(date +%F.%H%M%S).sql.gz
+You should store backups in a safe location.
+ +If you are running with the demo database (NEVER run the demo database in production), +you can create periodic database snapshots using the following configuration settings:
+ + # In this example, we create a snapshot at 4am each day and keep
+ # 7 days' worth of backups
+ #
+ # Database snapshots are written to 'data/demo_db_backups' by
+ # default.
+ AppConfig[:demo_db_backup_schedule] = "0 4 \* \* \*"
+ AppConfig[:demo\_db\_backup\_number\_to\_keep] = 7
+Solr indexes can always be recreated from the contents of the +database, but backing them up can reduce your recovery time if +disaster strikes on a large site. You can create periodic Solr +snapshots using the following configuration settings:
+ + # Create one snapshot at midnight and keep only one.
+ #
+ # Solr snapshots are written to 'data/solr_backups' by default.
+ AppConfig[:solr_backup_schedule] = "0 0 \* \* \*"
+ AppConfig[:solr\_backup\_number\_to\_keep] = 1
+ArchivesSpace provides some simple scripts for backing up a single
+instance to a .zip file. You can run:
scripts/backup.sh --output /path/to/backup-yyyymmdd.zip
+and the script will generate a file containing:
+ +If you are running against MySQL and have mysqldump installed, you
+can also provide the --mysqldump option. This will read the
+database settings from your configuration file and add a dump of your
+MySQL database to the resulting .zip file.
scripts/backup.sh --mysqldump --output ~/backups/backup-yyyymmdd.zip
+When recovering an ArchivesSpace installation from backup, you will +need to restore:
+ +Of the two, the database backup is the most crucial, your ArchivesSpace records +are all stored in your MySQL database. The solr search indexes are worth restoring +if you have backups, but they can be recreated from scratch if necessary.
+ +If you are using MySQL, recovering your database just requires loading
+your mysqldump backup into an empty database. If you are using the
+scripts/backup.sh script (described above), this dump file is named
+mysqldump.sql in your backup .zip file.
To load a MySQL dump file, follow the directions in Set up your MySQL +database to create an empty database with the appropriate +permissions. Then, populate the database from your backup file using +the MySQL client:
+ +`mysql -uas -p archivesspace < mysqldump.sql`, where
+ `as` is the user name
+ `archivesspace` is the database name
+ `mysqldump.sql` is the mysqldump filename
+You will be prompted for the password of the user.
+ +If you are using the demo database, your backup .zip file will
+contain a directory called demo_db_backups. Each subdirectory of
+demo_db_backups contains a backup of the demo database. To
+restore from a backup, copy its archivesspace_demo_db directory back
+to your ArchivesSpace data directory. For example:
cp -a /unpacked/zip/demo_db_backups/demo_db_backup_1373323208_25926/archivesspace_demo_db \
+ /path/to/archivesspace/data/
+This step is optional since indexes can be rebuilt from the contents +of the database. However, recovering your search indexes can reduce +the time needed to get your system running again.
+ +The backup .zip file contains two directories used by the
+ArchivesSpace indexer:
To restore these directories from backup:
+ +/path/to/archivesspace/data/solr_index/index/path/to/archivesspace/data/indexer_stateFor example:
+ + mkdir -p /path/to/archivesspace/data/solr_index
+
+ cp -a /unpacked/zip/solr.backup-26475-1373323208/snapshot.20130709084008464 \
+ /path/to/archivesspace/data/solr_index/index
+
+ cp -a /unpacked/zip/solr.backup-26475-1373323208/indexer_state \
+ /path/to/archivesspace/data/
+ArchivesSpace ships with a script that can run Lucene’s CheckIndex
+tool for you, verifying that a given Solr index is free from
+corruption. To test an index, run the following command from your
+archivesspace directory:
# Or scripts/checkindex.bat for Windows
+ scripts/checkindex.sh data/solr_index/index
+You can use the same script to check that your Solr backups are valid:
+ + scripts/checkindex.sh /unpacked/zip/solr.backup-26475-1373323208/snapshot.20130709084008464
+Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + administration/getting_started.md + +
+ ++ + Report issue on Jira + administration/getting_started.md + +
+ArchivesSpace has been tested on Ubuntu Linux, Mac OS X, and Windows.
+ +At least 1024 MB RAM allocated to the application are required. We recommend using at least 2 GB for optimal performance.
+ +We recommend using OpenJDK. The following table lists the supported Java versions for each version of ArchivesSpace:
+ +| ArchivesSpace version | +OpenJDK version | +
|---|---|
| ≤ v3.5.1 | +8 or 11 | +
| ≥ v4.0.0 | +11 or 17 | +
Up to ArchivesSpace v3.1.1, the zip file distribution includes an embedded Solr v4 instance, which is deprecated and not supported anymore.
+ +ArchivesSpace v3.2.0 or above requires an external Solr instance. The table below summarizes the supported Solr versions for each ArchivesSpace version:
+ +| ArchivesSpace version | +External Solr version | +
|---|---|
| ≤ v3.1.1 | +no external solr required | +
| v3.1.1 up to v3.5.1 | +8 (8.11) | +
| ≥ v4.0.0 | +9 (9.4.1) | +
Each ArchivesSpace version is tested for compatibility with the corresponding Solr version listed in the table above. +That version is being used during development and the ArchivesSpace automated tests run with that version. It is therefore recommended that you use that version of Solr in production.
+ +It may be possible to use ArchivesSpace with an older version of Solr. However in that case it +is important to check the release notes +for any potential version compatibility issues.
+ +Note: the ArchivesSpace Program Team can only provide support for Solr deployments +using the “officially” supported version with the standard configuration provided by +the application. Everything else will be treated as “best effort” community-led support.
+ +See Running with external Solr for more information on installing and upgrading Solr.
+ +While ArchivesSpace does include an embedded database, MySQL is required for production use.
+ +(While not officially supported by ArchivesSpace, some community members use MariaDB so there is some community support for version 10.4.10 only.)
+ +The embedded database is for testing purposes only. You should use MySQL or MariaDB for any data intended for production, including data in a test instance that you intend to move over to a production instance.
+ +All ArchivesSpace versions can run on MySQL version 5.x or 8.x.
+ +The quickest way to get ArchivesSpace up and running is to download
+the latest distribution .zip file from the following URL:
https://github.com/archivesspace/archivesspace/releases
+ +You will need to have Java installed on your machine. You can check your Java version by running the command:
+ + java -version
+See above for the Java version needed. If you are running an earlier version of java upgrade to one of the supported ones (not the newest one). If you are running a newer version of Java you should revert back to or force your machine to use a supported version.
+ +When you extract the .zip file, it will create a directory called
+archivesspace. Next, follow the instructions for setting up:
From any ArchivesSpace version > 3.1.0 external Solr is required. Earlier versions provided an embedded Solr v4 instance, which is now unsupported due to its age.
+ +Do not proceed until MySQL and Solr are running.
+ +To run the system, just execute the appropriate +startup script for your platform. On Linux and OSX:
+ + cd /path/to/archivesspace
+ ./archivesspace.sh
+and for Windows:
+ + cd \path\to\archivesspace
+ archivesspace.bat
+This will start ArchivesSpace running in foreground mode (so it will
+shut down when you close your terminal window). Log output will be
+written to the file logs/archivesspace.out (by default).
Note: If you’re running Windows and you get an error message like
+unable to resolve type 'size_t' or no such file to load -- bundler,
+make sure that there are no spaces in any part of the path name in which the
+ArchivesSpace directory is located.
The first time it starts, the system will take a minute or so to start +up. Once it is ready, confirm that ArchivesSpace is running correctly by +accessing the following URLs in your browser:
+ +To start using the Staff interface application, log in using the adminstrator +account:
+ +adminadminThen, you can create a new repository by selecting “System” -> “Manage +repositories” at the top right hand side of the screen. From the +“System” menu, you can perform a variety of administrative tasks, such +as creating and modifying user accounts. Be sure to change the +“admin” user’s password at this time.
+ + +Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + + + + +Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + administration/indexes.md + +
+ + +There are two strategies for reindexing ArchivesSpace:
+ +A soft reindex updates the existing documents in Solr without directly +touching the actual index documents on the filesystem. This can be done +while the system is running and is suitable for most use cases.
+ +There are two common ways to perform a soft reindex:
+ +ArchivesSpace keeps track of what has been indexed by using the files
+under data/indexer_state and data/indexer_pui_state (for the PUI).
If these files are missing, the indexer assumes that nothing has been
+indexed and reindexes everything. To force ArchivesSpace to reindex all
+records, just delete the files in /path/to/archivesspace/data/indexer_state
+and /path/to/archivesspace/data/indexer_pui_state.
You also can do this selectively by record type, for example, to reindex
+accessions in repository 2 delete the file called 2_accession.dat.
system_mtime values in the databaseIf you update a record’s system_mtime it becomes eligible for reindexing.
#reindex all resources
+UPDATE resource SET system_mtime = NOW();
+#reindex resource 1
+UPDATE resource SET system_mtime = NOW() WHERE id = 1;
+A full reindex is a complete rebuild of the index from the database. This +may be required if you are having indexer issues, in the case of index +corruption, or if called for by an upgrade owing to changes in ArchivesSpace’s +Solr configuration.
+ +To perform a full reindex:
+ +rm -rf /path/to/archivesspace/data/indexer_state/rm -rf /path/to/archivesspace/data/indexer_pui_state/rm -rf /path/to/archivesspace/data/solr_index/For external Solr there is a plugin that can perform all of the re-indexing steps: aspace-reindexer
+ +Manual steps:
+ +rm -rf /path/to/archivesspace/data/indexer_state/rm -rf /path/to/archivesspace/data/indexer_pui_state/curl -X POST -H 'Content-Type: application/json' --data-binary '{"delete":{"query":"*:*" }}' http://${solrUrl}:${solrPort}/solr/archivesspace/update?commit=trueYou can watch the Tips for indexing ArchivesSpace youtube video to see these steps performed.
+ +Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + administration/logrotate.md + +
+ + +In order to prevent your ArchivesSpace log file from growing excessively, you can set up log rotation. How to set up log rotation is specific to your institution but here is an example logrotate config file with an explanation of what it does.
+ +/etc/logrotate.d/
/<install location>/archivesspace/logs/archivesspace.out {
+ daily
+ rotate 7
+ compress
+ notifempty
+ missingok
+ copytruncate
+ }
+this example configuration file:
+Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + administration/passwords.md + +
+ + +Under the scripts directory you will find a script that lets you
+reset a user’s password. You can invoke it as:
scripts/password-reset.sh theusername newpassword # or password-reset.bat under Windows
+If you are running against MySQL, you can use this command to set a +password while the system is running. If you are running against the +demo database, you will need to shutdown ArchivesSpace before running +this script.
+ + +Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + administration/unix_daemon.md + +
+ ++ + Report issue on Jira + administration/unix_daemon.md + +
+The archivesspace.sh startup script doubles as an init script. If
+you run:
archivesspace.sh start
+ArchivesSpace will run in the background as a daemon (logging to
+logs/archivesspace.out by default, as before). You can shut it down with:
archivesspace.sh stop
+You can even install it as a system-wide init script by creating a +symbolic link:
+ + cd /etc/init.d
+ ln -s /path/to/your/archivesspace/archivesspace.sh archivesspace
+Note: By default ArchivesSpace will overwrite the log file when restarted. You
+can change that by modifying archivesspace.sh and changing the $startup_cmd
+to include double greater than signs:
$startup_cmd &>> \"$ARCHIVESSPACE_LOGS\" &
+Then use the appropriate tool for your distribution to set up the
+run-level symbolic links (such as chkconfig for RedHat or
+update-rc.d for Debian-based distributions).
Note that you may want to edit archivesspace.sh to set the account +that the system runs under, JVM options, and so on.
+ +For systems that use systemd you may wish to use a Systemd unit file for ArchivesSpace
+ +Something similar to this should work:
+[Unit]
+Description=ArchivesSpace Application
+After=syslog.target network.target
+[Service]
+Type=simple
+ExecStart=/path/to/your/archivesspace/archivesspace.sh
+ExecStop=/path/to/your/archivesspace/archivesspace.sh
+PIDFile=/path/to/your/archivesspace/archivesspace.pid
+User=archivesspacespace
+Group=archivesspacespace
+[Install]
+WantedBy=multi-user.target
+Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + administration/upgrading.md + +
+ + +You should make sure you have a working backup of your ArchivesSpace +installation before attempting an upgrade. Follow the steps +under the Backup and recovery section to do this.
+ +It’s a good idea to unpack a fresh copy of the version of +ArchivesSpace you are upgrading to. This will ensure that you are +running the latest versions of all files. In the examples below, +replace the lower case x with the version number updating to. For example, +1.5.2 or 1.5.3.
+ +For example, on Mac OS X or Linux:
+ + $ mkdir archivesspace-1.5.x
+ $ cd archivesspace-1.5.x
+ $ curl -LJO https://github.com/archivesspace/archivesspace/releases/download/v1.5.x/archivesspace-v1.5.x.zip
+ $ unzip -x archivesspace-v1.5.x.zip
+( The curl step is optional and simply downloads the distribution from github. You can also + simply download the zip file in your browser and copy it to the directory )
+ +On Windows, you can do the same by extracting ArchivesSpace into a new +folder you create in Windows Explorer.
+ +To ensure you get a consistent copy, you will need to shut down your +running ArchivesSpace instance now.
+ +You will need to bring across the following files and directories from +your original ArchivesSpace installation:
+ +data directory (see Indexes note below)config directory (see Configuration note below)lib/mysql-connector*.jar file (if using MySQL)plugins directoryFor example, on Mac OS X or Linux:
+ + $ cd archivesspace-1.5.x/archivesspace
+ $ cp -a /path/to/archivesspace-1.4.2/archivesspace/data/* data/
+ $ cp -a /path/to/archivesspace-1.4.2/archivesspace/config/* config/
+ $ cp -a /path/to/archivesspace-1.4.2/archivesspace/lib/mysql-connector* lib/
+ $ cp -a /path/to/archivesspace-1.4.2/archivesspace/plugins/local plugins/
+ $ cp -a /path/to/archivesspace-1.4.2/archivesspace/plugins/wonderful_plugin plugins/
+Or on Windows:
+ + $ cd archivesspace-1.5.x\archivesspace
+ $ xcopy \path\to\archivesspace-1.4.2\archivesspace\data\* data /i /k /h /s /e /o /x /y
+ $ xcopy \path\to\archivesspace-1.4.2\archivesspace\config\* config /i /k /h /s /e /o /x /y
+ $ xcopy \path\to\archivesspace-1.4.2\archivesspace\lib\mysql-connector* lib /i /k /h /s /e /o /x /y
+ $ xcopy \path\to\archivesspace-1.4.2\archivesspace\plugins\local plugins\local /i /k /h /s /e /o /x /y
+ $ xcopy \path\to\archivesspace-1.4.2\archivesspace\plugins\wonderful_plugin plugins\wonderful_plugin /i /k /h /s /e /o /x /y
+Note that you may want to preserve the logs file (logs/archivesspace.out
+by default) from your previous installation–just in case you need to
+refer to it later.
Sometimes a new release of ArchivesSpace will introduce new
+configuration settings that weren’t present in previous releases.
+Before you replace the distribution config/config.rb with your
+original version, it’s a good idea to review the distribution version
+to see if there are any new configuration settings of interest.
Upgrade notes will generally draw attention to any configuration +settings you need to set explicitly, but you never know when you’ll +discover a new, exciting feature! Documentation might also refer to +uncommenting configuration options that won’t be in your file if you +keep your older version.
+ +Sometimes a new release of ArchivesSpace will require a FULL reindex +which means you do not want to copy over anything from your data directory +to your new release. The data directory contains the indexes created by Solr. +Check the release notes of the new version for any details about reindexing.
+ +If you’ve made modifications to you locales file ( en.yml ) with customized +labels, titles, tooltips, etc., you’ll need to transfer those to your new +locale file.
+ +A good way to do this is to use a Diff tool, like Notepad++, TextMate, or just +Linux diff command:
+ + $ diff /path/to/archivesspace-1.4.2/locales/en.yml /path/to/archivesspace-1.5.x/archivesspace/locales/en.yml
+ $ diff /path/to/archivesspace-1.4.2/locales/enums/en.yml /path/to/archivesspace-v1.5.x/archivesspace/locales/enums/en.yml
+This will show you the differences in your current locales files, as well as the +new additions in the new version locales files. Simply copy the values you wish +to keep from your old ArchivesSpace locales to your new ArchivesSpace locales +files.
+ +With everything copied, the final step is to run the database
+migrations. This will apply any schema changes and data migrations
+that need to happen as a part of the upgrade. To do this, use the
+setup-database script for your platform. For example, on Mac OS X
+or Linux:
$ cd archivesspace-1.5.x/archivesspace
+ $ scripts/setup-database.sh
+Or on Windows:
+ + $ cd archivesspace-1.5.x\archivesspace
+ $ scripts\setup-database.bat
+Full instructions for using external Solr with ArchivesSpace
+ +The steps to deploy to Tomcat are esentially the same as in the +archivesspace_tomcat
+ +But, prior to running your setup-tomcat script, you’ll need to be sure to clean out the +any libraries from the previous ASpace version from your Tomcat classpath.
+ + 1. Stop Tomcat
+ 2. Unpack your new version of ArchivesSpace
+ 3. Configure your MySQL database in the config.rb ( just like in the
+ install instructions )
+ 4. Make sure all you other local configuration settings are in your
+ config.rb file ( check your Tomcat conf/config.rb file for your current
+ settings. )
+ 5. Make sure you MySQL connector jar in the lib directory
+ 6. Run your setup-database script to migration your database.
+ 7. Delete all ASpace related jar libraries in your Tomcat's lib directory. These
+ will include the "gems" folder, as well as "common.jar" and some
+ [others](https://github.com/archivesspace/archivesspace/tree/master/common/lib).
+ This will make sure your running the correct version of the dependent
+ libraries for your new ASpace version.
+ Just be sure not to delete any of the Apache Tomcat libraries.
+ 8. Run your setup-tomcat script ( just like in the install instructions ).
+ This will copy all the files over to Tomcat.
+ 9. Start Tomcat
+You can now start your new ArchivesSpace version as normal.
+ + +Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + administration/upgrading/UPGRADING_1.1.0.md + +
+ ++ + Report issue on Jira + administration/upgrading/UPGRADING_1.1.0.md + +
+Additional upgrade considerations specific to this release. Refer to the upgrade documentation for the standard instructions that apply in all cases.
+ +In ArchivesSpace 1.0.9 the default ports configuration was:
+ +AppConfig[:backend_url] = "http://localhost:8089"
+AppConfig[:frontend_url] = "http://localhost:8080"
+AppConfig[:solr_url] = "http://localhost:8090"
+AppConfig[:public_url] = "http://localhost:8081"
+With the introduction of the optional external Solr instance functionality this has been updated to:
+ +AppConfig[:backend_url] = "http://localhost:8089"
+AppConfig[:frontend_url] = "http://localhost:8080"
+AppConfig[:solr_url] = "http://localhost:8090"
+AppConfig[:indexer_url] = "http://localhost:8091" # NEW TO 1.1.0
+AppConfig[:public_url] = "http://localhost:8081"
+In most cases the default value for indexer_url will blend in seamlessly without you needing to take any action. However if you modified the original values in your config.rb file you may need to update it. Examples:
You use a different ports sequence
+ +AppConfig[:indexer_url] = "http://localhost:9091"
+You run multiple ArchivesSpace instances on a single host
+ +Under this deployment scenario you would have changed port numbers for some (or all) instances in each config.rb file, so set the indexer_url for each instance as described above.
You include hostnames
+ +AppConfig[:indexer_url] = "http://yourhostname:8091"
+In a clustered configuration you may need to edit instance_[server hostname].rb files:
{
+ ...
+ :indexer_url => "http://[localhost|yourhostname]:8091",
+}
+Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + administration/upgrading/UPGRADING_1.1.1.md + +
+ ++ + Report issue on Jira + administration/upgrading/UPGRADING_1.1.1.md + +
+Additional upgrade considerations specific to this release. Refer to the upgrade documentation for the standard instructions that apply in all cases.
+ +There have been some scenarios in which archival objects and digital object components lose +some of the information used to order their hierarchy. This can result in issues in creation, +editing, or moving items in the tree, since there are database contraints to ensure uniqueness +of certain metadata elements.
+ +In order to ensure data integrity, there is now method to resequence the trees. This will +not reorder or edit the elements, but simply rebuild all the technical metadata used to establish +the ordering.
+ +To run the resequencing process, edit the config/config.rb file to have this line:
+ +AppConfig[:resequence_on_startup] = true
+and restart ArchivesSpace. This will trigger a rebuilding process after the application has +started. It’s advised to let this rebuild process run its course prior to editing records. +This duration depends on the size of your database, which can take seconds ( for databases with +few Archival and Digital Objects ) to hours ( for databases with hundreds of thousands of records ). +Check your log file to see how the process is going. When it has finished, you should see the application +return to it normal operation, generally with only indexer updates being recorded in the log file.
+ +After you’ve started ArchivesSpace, be sure to change the config.rb file to have the :resequence_on_startup +set to “false”, since you will not need to run this process on every restart.
+ +A common request has been to have PDF version of the EAD exported in the public application. +This has been a bit problematic, since EAD export has a rather large resource hit on the +database, which is only increased by the added process of PDF creation. We are currently +redesigning part of the ArchivesSpace backend to make PDF creation more user friendly by +establishing a queue system for exports.
+ +In the meantime, Mark Cooper at Lyrasis has made a Public Metadata Formats plugin
+that exposes certain metadata formats and PDFs in the public UI. This plugin has been included
+in this release, but you will need to configure it to expose which formats you would like
+to have exposed. Please read the plugin documentation on how to configure this.
PLEASE NOTE: +Exporting large EAD resources with this plugin will most likely cause some problems. Long requests +will timeout, since the server does not want to waste resources on long running processes. +In addition, large number of requests for PDFs can cause an increase load on the server. +Please be aware of the issues and limitations of this plugin before enabling it.
+ +Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + administration/upgrading/UPGRADING_1.5.0.md + +
+ ++ + Report issue on Jira + administration/upgrading/UPGRADING_1.5.0.md + +
+Additional upgrade considerations specific to this release, which also apply to upgrading from 1.4.2 or lower to any version through 2.0.1. Refer to the upgrade documentation for the standard instructions that apply in all cases.
+ +The upgrade process to the new data model in 1.5.0 requires considerable data transformation and it is important for users to review this document to understand the implications and possible side-effects.
+ +A quick overview of the steps are:
+ +| Follow the standard upgrading instructions. Important to note: The setup-database.sh | +bat script will modify your database schema, but it will not move the data. If you are currently using the container management plugin you will need to remove it from the list of plugins in your config file prior to starting ArchivesSpace. | +
With version 1.5.0, ArchivesSpace is adopting a new data model that will enable more capable and efficient management of the containers in which you store your archival materials. To take advantage of this improved functionality:
+Converting the container data model in version 1.4.2 and earlier versions of ArchivesSpace to the 1.5.0 version has some complexity and may not accommodate all the various ways in which container information has been recorded by diverse repositories. As a consequence, upgrading from a pre-1.5.0 version of ArchivesSpace requires planning for the upgrade, reviewing the results, and, possibly, remediating data either prior to or after the final conversion process. Because of all the variations in which container information can be recorded, it is impossible to know all the ways the data of repositories will be impacted. For this reason, all repositories upgrading their ArchivesSpace to version 1.5.0 should do so with a backup of their production ArchivesSpace instance and in a test environment. A conversion may only be undone by reverting back to the source database.
+ +How will my data be converted to the new model?
+ +When your installation is upgraded to 1.5.0, the conversion will happen as part of the upgrade process.
+ +Can I continue to use the current model for containers and not convert to the new model?
+ +Because it is such a substantial improvement (see separate announcement for the new features), the new model is required for all using ArchivesSpace 1.5.0 and higher. The only way to continue using the current model is to never upgrade beyond 1.4.2.
+ +What if I’m already using the container management plugin made available to the community by Yale University?
+ +Conversion of data created using the Yale container management plugin, or a local adaptation of the plugin, will also happen as part of the process of upgrading to 1.5.0. Some steps will be skipped when they are not needed. At the end of the process, the new container data model will be integrated into your ArchivesSpace and will not need to be loaded or maintained as a plugin.
+ +Those currently running the container management plugin will need to remove the container management plugin from the list in your config file prior to starting the conversion or a validation name error will occur.
+ +I haven’t moved from Archivists’ Toolkit or Archon yet and am planning to use the associated migration tool. Can I migrate directly to 1.5.0?
+ +No, you must migrate to 1.4.2 or earlier versions and then upgrade your installation to 1.5.0 according to the instructions provided here.
+ +What changes are being made to the previous model for containers?
+ +The biggest change is the new concept of top containers. A top container is the highest level container in which a particular instance is stored. Top containers are in some ways analogous to the current Container 1, but broken out from the entire container record (child and grandparent container records). As such, top containers enable more efficient recording and updating of the highest level containers in your collection.
+ +How does ArchivesSpace determine what is a top container?
+ +During the conversion, ArchivesSpace will find all the Container 1s in your current ArchivesSpace database. It will then evaluate them as follows:
+What can I do to prepare my ArchivesSpace data for a smoother conversion to top containers?
+ +You do not need to make any changes to Container 2 fields or Container 3 fields. Data in these fields will be converted to the new Child and Grandchild container fields that map directly to these fields.
+ +If you use the current Container Extent fields, these will no longer be available in 1.5.0. Any data in these fields will be migrated to a new Extent sub-record during the conversion. You can evaluate whether this data should remain in an extent record or if it belongs in a container profile or other fields and then move it accordingly after the conversion is complete.
+ +I have EADs I still need to import into ArchivesSpace. How can I get them ready for this new model?
+ +If you have a box and folder associated with a component (or any other hierarchical relationship of containers), you will need to add identifiers to the container element so that the EAD importer knows which is the top container. If you previously used Archivists’ Toolkit to create EAD, your containers probably already have container identifiers. If your container elements do not have identifiers already, Yale University has made available an XSLT transformation file to add them. You will need to run it before importing the EAD file into ArchivesSpace.
+ +When upgrading from 1.4.2 (and earlier versions) to 1.5.0, the container conversion will happen as part of the upgrade process. You will be able to follow its progress in the log. Instructions for upgrading from a previous version of ArchivesSpace are available at upgrade documentation.
+ +Because this is a major change in the data model for this portion of the application, running at least one test conversion is very strongly recommended. Follow these steps to run the upgrade/conversion process:
+
What are some common errors or anomalies that will be flagged in the conversion?
+ +The conversion process can resolve some of these errors for you by supplying or deleting values as it deems appropriate, but for the most control over the process you will most likely want to resolve such issues yourself in your ArchivesSpace database before converting to the new container model.
+ +Are there any known conversion issues?
+ +Due to a change in the ArchivesSpace EAD importer in 2015, some EADs with hierarchical containers not designated by a @parent attribute were turned into multiple instance records. This has since been corrected in the application, but we are working on a plugin (now available at Instance Joiner Plug-in that will enable you to turn these back into single instances so that subcontainers are not mistakenly turned into top containers.
+ + +Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + administration/upgrading/UPGRADING_2.1.0.md + +
+ ++ + Report issue on Jira + administration/upgrading/UPGRADING_2.1.0.md + +
+(these considerations also apply when upgrading to any version past 2.1.0 from a version prior to 2.1.0)
+ +Additional upgrade considerations specific to this release. Refer to the upgrade documentation for the standard instructions that apply in all cases.
+ +Following the merge of the Container Management Plugin in 1.5.0, ArchivesSpace still retained the old container model and had a number of dependencies on it. This imposed unnecessary complexity and some performance degradation on the system.
+ +In this release all references to the old container model have been removed and the parts of the application that were dependent on it (for example, Imports and Exports) have been refactored to use the new container model.
+ +A consequence of this change is that if you are upgrading from ArchivesSpace version of 1.4.2 or lower, you will need to first upgrade to any version between 1.5.0 and 2.0.1 to run the container conversion. You will then be able to upgrade to 2.1.0. If you are already using any version of ArchivesSpace between 1.5.0 and 2.0.1, you will be able to upgrade directly to 2.1.0.
+ +The migration tools are currently supported through version 1.4.2 only. If you want to migrate data to ArchivesSpace using one of these tools, you must migrate it to 1.4.2. From there you can follow the instructions for those upgrading from 1.4.2 and lower.
+ +The rights statements data model has changed in 2.1.0. If you currently use rights statements, your data will be converted to the new model during the setup-database step of the upgrade process. We strongly urge you to backup your database and run at least one test upgrade before putting 2.1.0 into production.
+ +The index schema has changed with 2.1.0. If you are using an external Solr server, you will need to update the schema.xml with the newer version. If you are using the default Solr index that ships with ArchivesSpace, no action is needed.
+ + +Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + administration/windows.md + +
+ + +Running ArchivesSpace as a Windows service requires some additional configuration.
+ +You can use Apache procrun to configure ArchivesSpace to run as a Windows service. We have provided a service.bat script that will attempt to configure procrun for you (under launcher\service.bat).
To run this script, first you need to download procrun. +Extract the files and copy the prunsrv.exe and prunmgr.exe to your ArchivesSpace directory.
+ +To find the path to Java, “Start” > “Control Panel” > “Java”, Select “Java” tab. You’ll see the path there. It will look something like C:\Program Files (x86)\Java
You also need to be sure that Java is in your system path and also to create JAVA_HOME as a global environment variable.
+To add Java to your path, edit you %PATH% environment variable to include the directory of your java executable ( it will be something like C:\Program Files (x86)\Java ). To add JAVA_HOME, add a new system variable and put the directory where java was installed ( something like C:\Program Files (x86)\Java ).
Environement varialbe be found by “Start” > “Control Panel” , search for environment. Click “edit the system environment variables”. In the section System Variables, find the PATH environment variable and select it. Click Edit. If the PATH environment variable does not exist, click New. In the Edit System Variable (or New System Variable) window, specify the value of the PATH environment variable. Click OK. Close all remaining windows by clicking OK. Do the same for JAVA_HOME
Before setting up the ArchivesSpace service, you should also configure ArchivesSpace to run against MySQL. +Be sure that the MySQL connector jar file is in the lib directory, in order for +the service setup script to add it to the application’s classpath.
+ +Lastly, for the service to shutdown cleanly, uncomment and change these lines in +config/config.rb:
+ +AppConfig[:use_jetty_shutdown_handler] = true
+AppConfig[:jetty_shutdown_path] = "/xkcd"
+This enables a shutdown hook for Jetty to respond to when the shutdown action +is taken.
+ +You can now execute the batch script from your ArchivesSpace root directory from
+the command line with launcher\service.bat. This will configure the service and
+provide two executables: ArchivesSpaceService.exe (the service) and
+ArchivesSpaceServicew.exe (a GUI monitor)
There are several options to launch the service. The easiest is to open the GUI +monitor and click “Launch”.
+ +Alternatively, you can start the GUI monitor and minimize it in your +system tray with:
+ +ArchivesSpaceServicew.exe //MS//
+To execute the service from the command line, you can invoke:
+ +ArchivesSpaceService.exe //ES//
+Log output will be placed in your ArchivesSpace log directory.
+ +Please see the procrun +documentation +for more information.
+ + +Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + + + + +This documentation provides general information on working with the API. For detailed documentation of specific endpoints, see the API reference, which is maintained separately.
+ +Most actions against the backend require you to be logged in as a user +with the appropriate permissions. By sending a request like:
+ + POST /users/admin/login?password=login
+your authentication request will be validated, and a session token
+will be returned in the JSON response for your request. To remain
+authenticated, provide this token with subsequent requests in the
+X-ArchivesSpace-Session header. For example:
X-ArchivesSpace-Session: 8e921ac9bbe9a4a947eee8a7c5fa8b4c81c51729935860c1adfed60a5e4202cb
+Since not all backend/API end points require authentication, it is best to restrict access to port 8089 to only IP addresses you trust. Your firewall should be used to specify a range of IP addresses that are allowed to call your ArchivesSpace API endpoint. This is commonly called whitelisting or allowlisting.
+ +Send request to authenticate:
+ +curl -s -F password="admin" "http://localhost:8089/users/admin/login"
+This will return a JSON response that includes something like the following:
+ +{
+ "session":"9528190655b979f00817a5d38f9daf07d1686fed99a1d53dd2c9ff2d852a0c6e",
+ ....
+}
+It’s a good idea to save the session key as an environment variable to use for later requests:
+ +#Mac/Unix terminal
+export SESSION="9528190655b979f00817a5d38f9daf07d1686fed99a1d53dd2c9ff2d852a0c6e"
+
+#Windows Command Prompt
+set SESSION="9528190655b979f00817a5d38f9daf07d1686fed99a1d53dd2c9ff2d852a0c6e"
+
+#Windows PowerShell
+$env:SESSION="9528190655b979f00817a5d38f9daf07d1686fed99a1d53dd2c9ff2d852a0c6e"
+Now you can make requests like this:
+ +curl -H "X-ArchivesSpace-Session: $SESSION" "http://localhost:8089/repositories/2/resources/1
+The ArchivesSpace API provides CRUD-style interactions for a number of +different “top-level” record types. Working with records follows a +fairly standard pattern:
+ + # Get a paginated list of accessions from repository '123'
+ GET /repositories/123/accessions?page=1
+
+ # Create a new accession, returning the ID of the new record
+ POST /repositories/123/accessions
+ {... a JSON document satisfying JSONModel(:accession) here ...}
+
+ # Get a single accession (returned as a JSONModel(:accession) instance) using the ID returned by the previous request
+ GET /repositories/123/accessions/456
+
+ # Update an existing accession
+ POST /repositories/123/accessions/456
+ {... a JSON document satisfying JSONModel(:accession) here ...}
+++ +Additional documentation is needed for these sections - please consider contributing documentation via a pull request to this repo
+
++ +Additional documentation needed
+
++ +Additional documentation needed
+
++ +Additional documentation needed
+
++ +Additional documentation needed
+
++ +Additional documentation needed
+
++ +Additional documentation needed
+
++ + +Additional documentation needed
+
Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + architecture/backend/api.md + +
+ + +++ +See API section for more detailed documentation
+
Most actions against the backend require you to be logged in as a user +with the appropriate permissions. By sending a request like:
+ + POST /users/admin/login?password=login
+your authentication request will be validated, and a session token
+will be returned in the JSON response for your request. To remain
+authenticated, provide this token with subsequent requests in the
+X-ArchivesSpace-Session header. For example:
X-ArchivesSpace-Session: 8e921ac9bbe9a4a947eee8a7c5fa8b4c81c51729935860c1adfed60a5e4202cb
+The ArchivesSpace API provides CRUD-style interactions for a number of +different “top-level” record types. Working with records follows a +fairly standard pattern:
+ + # Get a paginated list of accessions from repository '123'
+ GET /repositories/123/accessions?page=1
+
+ # Create a new accession, returning the ID of the new record
+ POST /repositories/123/accessions
+ {... a JSON document satisfying JSONModel(:accession) here ...}
+
+ # Get a single accession (returned as a JSONModel(:accession) instance) using the ID returned by the previous request
+ GET /repositories/123/accessions/456
+
+ # Update an existing accession
+ POST /repositories/123/accessions/456
+ {... a JSON document satisfying JSONModel(:accession) here ...}
+Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + architecture/backend/database.md + +
+ ++ + Report issue on Jira + architecture/backend/database.md + +
+The ArchivesSpace database stores all data that is created within an ArchivesSpace instance. As described in other sections of this documentation, the backend code - particularly the model layer and ASModel_crud.rb file - uses the Sequel database toolkit to bridge the gap between this underlying data and the JSON objects which are exchanged by the other components of the system.
Often, querying the database directly is the most efficient and powerful way to retrieve data from ArchivesSpace. It is also possible to use raw SQL queries to create custom reports that can be run by users in the staff interface. Please consult the Custom Reports section of this documentation for additional information on creating custom reports.
+ + + +It is recommended that ArchivesSpace be run against MySQL in production, not the included demo database. Instructions on setting up ArchivesSpace to run against MySQL are here.
+ +The examples in this section are written for MySQL. There are many freely-available tutorials on the internet which can provide guidance to those unfamiliar with MySQL query syntax and the features of the language.
+ +NOTE: the documentation below is current through database schema version 129, application version 2.7.1.
+ +The ArchivesSpace database schema and it’s mapping to the JSONModel objects used by the other parts of the system is defined by the files in the common/schemas and backend/models directories. The database itself is created via the setup-database script in the scripts directory. This script runs the migrations in the common/db/migrations directory.
The tables in the ArchivesSpace database can be grouped into several general categories:
+One way to get a view of all tables and columns in your ArchivesSpace instance is to run the following query in a MySQL client:
+ + SELECT TABLE_SCHEMA
+ , TABLE_NAME
+ , COLUMN_NAME
+ , ORDINAL_POSITION
+ , IS_NULLABLE
+ , COLUMN_TYPE
+ , COLUMN_KEY
+ FROM INFORMATION_SCHEMA.COLUMNS
+ #change the following value to whatever your database is named
+ WHERE TABLE_SCHEMA Like 'archivesspace'
+Additionally, a BETA version of an ArchivesSpace data dictionary has been created by members of the ArchivesSpace development team and the ArchivesSpace User Advisory Council Reports team.
+ +These tables hold data about the primary record types in ArchivesSpace. Main record types are distinguished from subrecords in that they have their own persistent URIs - corresponding to their database identifiers/primary keys - that are resolvable via the staff interface, public interface, and API. They are distinguished from supporting records in that they are the primary descriptive record types that users will interact with in the system.
+ +All of these records, except archival objects, can be created independently of any other record. Archival object records represent components of a larger entity, and so they must have a resource record as a root parent. See the parent/child relationships section for more information about the representation of hierarchical relationships in the database.
+ +A few common fields occur in several main record tables. These similar fields are defined by the parent schemas in the common/schemas directory:
| Column Name | +Tables | +
|---|---|
title |
+ accession, archival_object, digital_object, digital_object_component, resource |
+
identifier/component_id/digital_object_id |
+ accession, resource/archival_object, digital_object_component/digital_object |
+
other_level |
+ archival_object, resource |
+
repository_processing_note |
+ archival_object, resource |
+
All of the main records have a set of fields which store boolean values (0 or 1) that indicate whether the records are published in the public user interface, suppressed in the staff interface, or have some kind of applicable restriction. The exception to this is the repository table, which does not have a restriction boolean, but does have a hidden boolean. The accession table has multiple restriction-related booleans. See the section below for more information about boolean fields.
Beginning in version 2.6.0, the main record tables (and some supporting records - see below) also contain fields which hold data about archival resource keys (ARKs) and human-readable URLs (slugs):
+ +| Column Name | +Tables | +
|---|---|
slug |
+ accession, archival_object, digital_object, digital_object_component, repository, resource |
+
external_ark_url |
+ archival_object, resource |
+
Also stored in these and all other tables are enumeration values, foreign keys which correspond to database identifiers in the enumeration_value table, which stores controlled values. See enumeration section below for more detail.
All subrecord data types - i.e. dates, extents, instances - relating to a main or supporting record are stored in their own tables and linked to main or supporting records via foreign key references in the subrecord tables. See subrecord section below for more detail.
+ +The remaining data in the main record tables is text, and is unique to each table:
+ +| TABLE_NAME | +COLUMN_NAME | +IS_NULLABLE | +COLUMN_TYPE | +COLUMN_KEY | +
|---|---|---|---|---|
accession |
+ content_description |
+ YES | +text | ++ |
accession |
+ condition_description |
+ YES | +text | ++ |
accession |
+ disposition |
+ YES | +text | ++ |
accession |
+ inventory |
+ YES | +text | ++ |
accession |
+ provenance |
+ YES | +text | ++ |
accession |
+ general_note |
+ YES | +text | ++ |
accession |
+ accession_date |
+ YES | +date | ++ |
accession |
+ retention_rule |
+ YES | +text | ++ |
accession |
+ access_restrictions_note |
+ YES | +text | ++ |
accession |
+ use_restrictions_note |
+ YES | +text | ++ |
archival_object |
+ ref_id |
+ NO | +varchar(255) | +MUL | +
digital_object_component |
+ label |
+ YES | +varchar(255) | ++ |
repository |
+ repo_code |
+ NO | +varchar(255) | +UNI | +
repository |
+ name |
+ NO | +varchar(255) | ++ |
repository |
+ org_code |
+ YES | +varchar(255) | ++ |
repository |
+ parent_institution_name |
+ YES | +varchar(255) | ++ |
repository |
+ url |
+ YES | +varchar(255) | ++ |
repository |
+ image_url |
+ YES | +varchar(255) | ++ |
repository |
+ contact_persons |
+ YES | +text | ++ |
repository |
+ description |
+ YES | +text | ++ |
repository |
+ oai_is_disabled |
+ YES | +int | ++ |
repository |
+ oai_sets_available |
+ YES | +text | ++ |
resource |
+ ead_id |
+ YES | +varchar(255) | ++ |
resource |
+ ead_location |
+ YES | +varchar(255) | ++ |
resource |
+ finding_aid_title |
+ YES | +text | ++ |
resource |
+ finding_aid_filing_title |
+ YES | +text | ++ |
resource |
+ finding_aid_date |
+ YES | +varchar(255) | ++ |
resource |
+ finding_aid_author |
+ YES | +text | ++ |
resource |
+ finding_aid_language_note |
+ YES | +varchar(255) | ++ |
resource |
+ finding_aid_sponsor |
+ YES | +text | ++ |
resource |
+ finding_aid_edition_statement |
+ YES | +text | ++ |
resource |
+ finding_aid_series_statement |
+ YES | +text | ++ |
resource |
+ finding_aid_note |
+ YES | +text | ++ |
resource |
+ finding_aid_subtitle |
+ YES | +text | ++ |
Like the main record types listed above, supporting records can also be created independently of other records, and are addressable in the staff interface and API via their own URI. However, they are primarily meaningful via their many-to-many linkages to the main record types (and, sometimes, other supporting record types). These records typically provide additional information about, or otherwise enhance, the primary record types. A few supporting record types - for instance those in the term table - are used to enhance other supporting record types.
| Supporting module tables | +Linked to | +
|---|---|
agent_corporate_entity |
+ + |
agent_family |
+ + |
agent_person |
+ + |
agent_software |
+ + |
assessment |
+ + |
classification |
+ accession, resource |
+
classification_term |
+ classification, accession, resource |
+
container_profile |
+ top_container |
+
event |
+ + |
location |
+ + |
location_profile |
+ location |
+
subject |
+ resource, archival_object |
+
term |
+ subject |
+
top_container |
+ + |
vocabulary |
+ subject, term |
+
assessment_attribute_definition |
+ assessment_attribute, assessment_attribute_note |
+
Subrecords must be associated with a main or supporting record - they cannot be created independently. As such, they do not have their own URIs, and can only be accessed via the API by retrieving the top-level record with which they are associated. In the staff interface these records are embedded within main or supporting record views. In the API subrecord data is contained in arrays within main or supporting records.
+ +The various subrecord types do have their own database tables. In addition to data specific to the subrecord type, the tables also contain foreign key columns which hold the database identifiers of main or supporting records. Subrecord tables must have a value in one of the foreign key fields. Some subrecords can have another subrecord as parent (for instance, the sub_container subrecord has instance_id as its foreign key column).
Subrecords exist in a one-to-many relationship with their parent records, so a record’s id may appear multiple times in a subrecord table (i.e. when there are two dates associated with a resource record).
It is important to note that subrecords are deleted and recreated upon each save of the main or supporting record with which they are associated, regardless of whether the subrecord itself is modified. This means that the database identifier is deleted and reassigned upon each save.
+ +| Subrecord tables | +Foreign keys | +
|---|---|
agent_contact |
+ agent_person_id, agent_family_id, agent_corporate_entity_id, agent_software_id |
+
date |
+ accession_id, deaccession_id, archival_object_id, resource_id, event_id, digital_object_id, digital_object_component_id, related_agents_rlshp_id, agent_person_id, agent_family_id, agent_corporate_entity_id, agent_software_id, name_person_id, name_family_id, name_corporate_entity_id, name_software_id |
+
extent |
+ accession_id, deaccession_id, archival_object_id, resource_id, digital_object_id, digital_object_component_id |
+
external_document |
+ accession_id, archival_object_id, resource_id, subject_id, agent_person_id, agent_family_id, agent_corporate_entity_id, agent_software_id, rights_statement_id, digital_object_id, digital_object_component_id, event_id |
+
external_id |
+ subject_id, accession_id, archival_object_id, collection_management_id, digital_object_id, digital_object_component_id, event_id, location_id, resource_id |
+
file_version |
+ digital_object_id, digital_object_component_id |
+
instance |
+ resource_id, archival_object_id, accession_id |
+
name_authority_id |
+ name_person_id, name_family_id, name_software_id, name_corporate_entity_id |
+
name_corporate_entity |
+ agent_corporate_entity_id |
+
name_family |
+ agent_family_id |
+
name_person |
+ agent_person_id |
+
name_software |
+ agent_software_id |
+
note |
+ resource_id, archival_object_id, digital_object_id, digital_object_component_id, agent_person_id, agent_corporate_entity_id, agent_family_id, agent_software_id, rights_statement_act_id, rights_statement_id |
+
note_persistent_id |
+ note_id, parent_id |
+
revision_statement |
+ resource_id |
+
rights_restriction |
+ resource_id, archival_object_id |
+
rights_restriction_type |
+ rights_restriction_id |
+
rights_statement |
+ accession_id, archival_object_id, resource_id, digital_object_id, digital_object_component_id, repo_id |
+
rights_statement_act |
+ rights_statement_id |
+
sub_container |
+ instance_id |
+
telephone |
+ agent_contact_id |
+
user_defined |
+ accession_id, resource_id, digital_object_id |
+
ark_name |
+ archival_object_id, resource_id |
+
assessment_attribute_note |
+ assessment_id |
+
assessment_attribute |
+ assessment_id |
+
lang_material |
+ archival_object_id, resource_id, digital_object_id, digital_object_component_id |
+
language_and_script |
+ lang_material_id |
+
collection_management |
+ accession_id, resource_id, digital_object_id |
+
location_function |
+ location_id |
+
These tables exist to enable linking between main records and supporting records. Relationship tables are necessary because, unlike subrecord tables, supporting record tables do not include foreign keys which link them to the main record tables.
+ +Most relationship tables have the _rlshp suffix in their names. They typically contain just the primary keys for the tables that are being linked, though a few tables also include fields that are specific to the relationship between the two record types.
| Relationship/linking tables | +Tables linked | +
|---|---|
assessment_reviewer_rlshp |
+ assessment to agent_person |
+
assessment_rlshp |
+ assessment to accession, archival_object, resource, or digital_object |
+
classification_creator_rlshp |
+ classification to agent_person, agent_family, agent_corporate_entity, or agent_software |
+
classification_rlshp |
+ classification or classification_term to resource or accession |
+
classification_term_creator_rlshp |
+ classification_term to agent_person, agent_family, agent_corporate_entity, or agent_software |
+
event_link_rlshp |
+ event to accession, resource, archival_object, digital_object, digital_object_component, agent_person, agent_family, agent_corporate_entity, agent_software, or top_container. Also includes the role_id table, which can be joined with the enumeration_value table to return the event role (source, outcome, transfer, context) |
+
instance_do_link_rlshp |
+ digital_object to instance |
+
linked_agents_rlshp |
+ agent_person, agent_software, agent_family, or agent_corporate_entity to accession, archival_object, digital_object, digital_object_component, event, or resource. Also includes the role_id and relator_id tables, which can be joined with the enumeration_value table |
+
location_profile_rlshp |
+ location to location_profile |
+
owner_repo_rlshp |
+ location to repository |
+
related_accession_rlshp |
+ Links a row in the accession table to another row in the accession table. Also includes fields for relator and relationship type. |
+
related_agents_rlshp |
+ agent_person, agent_corporate_entity, agent_software, or agent_family to other agent tables, or two rows in the same agent table. Also includes fields for relator and description, and the type of relationship. |
+
spawned_rlshp |
+ accession to resource. This contains all linked accession data, even if the resource was not spawned from the accession record. |
+
subject_rlshp |
+ subject to accession, archival_object, resource, digital_object, or digital_object_component |
+
surveyed_by_rlshp |
+ assessment to agent_person |
+
top_container_housed_at_rlshp |
+ top_container to location. Also includes fields for start_date, end_date, status, and a free-text note. |
+
top_container_link_rlshp |
+ top_container to sub_container |
+
top_container_profile_rlshp |
+ top_container to container_profile |
+
subject_term |
+ subject to term |
+
linked_agent_term |
+ linked_agents_rlshp to term |
+
It is not always obvious which relationship tables will provide the desired results. For instance, to get a box list for a given resource record, enter the following query into a MySQL editor:
+ + SELECT DISTINCT CONCAT('/repositories/', resource.repo_id, '/resources/', resource.id) as resource_uri
+ , resource.identifier
+ , resource.title
+ , tc.barcode as barcode
+ , tc.indicator as box_number
+ FROM sub_container sc
+ JOIN top_container_link_rlshp tclr on tclr.sub_container_id = sc.id
+ JOIN top_container tc on tclr.top_container_id = tc.id
+ JOIN instance on sc.instance_id = instance.id
+ JOIN archival_object ao on instance.archival_object_id = ao.id
+ JOIN resource on ao.root_record_id = resource.id
+ #change to your desired resource id
+ WHERE resource.id = 4556
+Sometimes numerous relationship tables must be joined to retrieve the desired results. For instance, to get all boxes and folders for a given resource record, including any container profiles and locations, enter the following query into a MySQL editor:
+ + SELECT CONCAT('/repositories/', tc.repo_id, '/top_containers/', tc.id) as tc_uri
+ , CONCAT('/repositories/', resource.repo_id, '/resources/', resource.id) as resource_uri
+ , CONCAT('/repositories/', resource.repo_id) as repo_uri
+ , CONCAT('/repositories/', ao.repo_id, '/archival_objects/', ao.id) as ao_uri
+ , resource.identifier AS resource_identifier
+ , resource.title AS resource_title
+ , ao.display_string AS ao_title
+ , ev2.value AS level
+ , tc.barcode AS barcode
+ , cp.name AS container_profile
+ , tc.indicator AS container_num
+ , ev.value AS sc_type
+ , sc.indicator_2 AS sc_num
+ from sub_container sc
+ JOIN top_container_link_rlshp tclr on tclr.sub_container_id = sc.id
+ JOIN top_container tc on tclr.top_container_id = tc.id
+ LEFT JOIN top_container_profile_rlshp tcpr on tcpr.top_container_id = tc.id
+ LEFT JOIN container_profile cp on cp.id = tcpr.container_profile_id
+ LEFT JOIN top_container_housed_at_rlshp tchar on tchar.top_container_id = tc.id
+ JOIN instance on sc.instance_id = instance.id
+ JOIN archival_object ao on instance.archival_object_id = ao.id
+ JOIN resource on ao.root_record_id = resource.id
+ LEFT JOIN enumeration_value ev on ev.id = sc.type_2_id
+ LEFT JOIN enumeration_value ev2 on ev2.id = ao.level_id
+ #change to your desired resource id
+ WHERE resource.id = 4223
+
+All controlled values used by the application - excluding tool-tips and frontend/public display values and the values that are stored a few of the supporting record tables (see below) - are stored in a table called enumeration_values. Controlled values are organized into a variety of parent enumerations (akin to a set of distinct controlled value lists) which are utilized by different record and subrecord types. Parent enumeration data is stored in the enumeration table and is linked by foreign key in the enumeration_id field in the enumeration_value table. In the record and subrecord tables, enumeration values appear as foreign keys in a variety of foreign key columns, usually identified by an _id suffix.
ArchivesSpace comes with a standard set of controlled values, but most of these are modifiable by end-users via the staff interface and API. However, some values in the enumeration_value table are read-only - these values define the terminology and data types used in different parts of the application (i.e. the various note types).
Enumeration IDs appear as foreign keys in a variety of database tables:
+ +| table_name | +column_name | +enumeration_name | +
|---|---|---|
accession |
+ acquisition_type_id |
+ accession_acquisition_type | +
accession |
+ resource_type_id |
+ accession_resource_type | +
agent_contact |
+ salutation_id |
+ agent_contact_salutation | +
archival_object |
+ level_id |
+ archival_record_level | +
collection_management |
+ processing_priority_id |
+ collection_management_processing_priority | +
collection_management |
+ processing_status_id |
+ collection_management_processing_status | +
collection_management |
+ processing_total_extent_type_id |
+ extent_extent_type_id | +
container_profile |
+ dimension_units_id |
+ dimension_units | +
date |
+ calendar_id |
+ date_calendar | +
date |
+ certainty_id |
+ date_certainty | +
date |
+ date_type_id |
+ date_type | +
date |
+ era_id |
+ date_era | +
date |
+ label_id |
+ date_label | +
deaccession |
+ scope_id |
+ deaccession_scope | +
digital_object |
+ digital_oject_type_id |
+ digital_object_digital_object_type | +
digital_object |
+ level_id |
+ digital_object_level | +
event |
+ event_type_id |
+ event_event_type | +
event |
+ outcome_id |
+ event_outcome | +
extent |
+ extent_type_id |
+ extent_extent_type | +
extent |
+ portion_id |
+ extent_portion | +
external_document |
+ identifier_type_id |
+ rights_statement_external_document_identifier_type | +
file_version |
+ checksum_method_id |
+ file_version_checksum_methods | +
file_version |
+ file_format_name_id |
+ file_version_file_format_name | +
file_version |
+ use_statement_id |
+ file_version_use_statement | +
file_version |
+ xlink_actuate_attribute_id |
+ file_version_xlink_actuate_attribute | +
file_version |
+ xlink_show_attribute_id |
+ file_version_xlink_show_attribute | +
instance |
+ instance_type_id |
+ instance_instance_type | +
language_and_script |
+ language_id |
+ + |
language_and_script |
+ script_id |
+ + |
location |
+ temporary_id |
+ location_temporary | +
location_function |
+ location_function_type_id |
+ location_function_type | +
location_profile |
+ dimension_units_id |
+ dimension_units | +
name_corporate_entity |
+ rules_id |
+ name_rule | +
name_corporate_entity |
+ source_id |
+ name_source | +
name_family |
+ rules_id |
+ name_rule | +
name_family |
+ source_id |
+ name_source | +
name_person |
+ name_order_id |
+ name_person_name_order | +
name_person |
+ rules_id |
+ name_rule | +
name_person |
+ source_id |
+ name_source | +
name_software |
+ rules_id |
+ name_rule | +
name_software |
+ source_id |
+ name_source | +
repository |
+ country_id |
+ country_iso_3166 | +
resource |
+ finding_aid_description_rules_id |
+ resource_finding_aid_description_rules | +
resource |
+ finding_aid_language_id |
+ + |
resource |
+ finding_aid_script_id |
+ + |
resource |
+ finding_aid_status_id |
+ resource_finding_aid_status | +
resource |
+ level_id |
+ archival_record_level | +
resource |
+ resource_type_id |
+ resource_resource_type | +
rights_restriction_type |
+ restriction_type_id |
+ restriction_type | +
rights_statement |
+ jurisdiction_id |
+ + |
rights_statement |
+ other_rights_basis_id |
+ rights_statement_other_rights_basis | +
rights_statement |
+ rights_type_id |
+ rights_statement_rights_type | +
rights_statement |
+ status_id |
+ + |
rights_statement_act |
+ act_type_id |
+ rights_statement_act_type | +
rights_statement_act |
+ restriction_id |
+ rights_statement_act_restriction | +
rights_statement_pre_088 |
+ ip_status_id |
+ rights_statement_ip_status | +
rights_statement_pre_088 |
+ jurisdiction_id |
+ + |
rights_statement_pre_088 |
+ rights_type_id |
+ rights_statement_rights_type | +
sub_container |
+ type_2_id |
+ container_type | +
sub_container |
+ type_3_id |
+ container_type | +
subject |
+ source_id |
+ subject_source | +
telephone |
+ number_type_id |
+ telephone_number_type | +
term |
+ term_type_id |
+ subject_term_type | +
top_container |
+ type_id |
+ container_type | +
To translate the enumeration ID that appears in the record and subrecord tables, join the enumeration_value table. The table can be joined multiple times if there are multiple values to translate, but you must use an alias for each table. For example:
SELECT CONCAT('/repositories/', ao.repo_id, '/archival_objects/', ao.id) as ao_uri
+ , ao.display_string as ao_title
+ , date.begin
+ , date.end
+ , ev.value as date_label
+ , ev2.value as date_type
+ , ev3.value as date_calendar
+FROM archival_object ao
+LEFT JOIN date on date.archival_object_id = ao.id
+LEFT JOIN enumeration_value ev on ev.id = date.label_id
+LEFT JOIN enumeration_value ev2 on ev2.id = date.date_type_id
+LEFT JOIN enumeration_value ev3 on ev3.id = date.calendar_id
+NOTE: container_profile, location_profile, and assessment_attribute_definition records are similar to the records in the enumeration_value table in that they store controlled values which are referenced by other parts of the system. However, they differ in that they have their own tables and are addressable via their own URIs.
These tables store user and permissions information, user/repository/global preferences, and RDE and custom report templates.
+ +| Table name | +Description | +
|---|---|
custom_report_template |
+ Custom report templates | +
default_values |
+ Default values settings | +
group |
+ Data about permission groups created by each repository | +
group_permission |
+ Links the permission table to the group table | +
group_user |
+ Links the group table to the user table | +
oai_config |
+ Configuration data for OAI-PMH harvesting | +
permission |
+ All permission types that can be assigned to users | +
preference |
+ User preference data | +
rde_template |
+ RDE templates | +
required_fields |
+ Contains repository-defined required fields | +
user |
+ User data | +
These tables store data related to background jobs, including imports.
+ +| Table name | +Description | +
|---|---|
job |
+ All jobs which have been run in an ArchivesSpace instance. | +
job_created_record |
+ Records created via background jobs | +
job_input_file |
+ Data about input files used in background jobs | +
job_modified_record |
+ Data about records modified via background jobs | +
These tables track actions taken against the database (i.e. edits and deletes), system events, session and authorization data, and database information. These tables are typically not referenced by any other table.
+ +| Table name | +Description | +
|---|---|
active_edit |
+ Records being actively edited by a user. Read-only system table | +
auth_db |
+ Authentication data for users. Read-only system table | +
deleted_records |
+ Records deleted in the past 24 hours. Read-only system table | +
notification |
+ Notifications stream. Read-only system table | +
schema_info |
+ Contains the database schema version. Read-only system table. | +
sequence |
+ The value corresponds to the number of children the archival object has - 1. Read-only system table | +
session |
+ Recent session data. Read-only system table | +
system_event |
+ System event data. Read-only system table | +
Many main and supporting records are scoped to a particular repository. In these tables the parent repository is identified by a foreign key which corresponds to the database identifier in the repository table:
| Column name | +Description | +Example | +Found in | +
|---|---|---|---|
repo_id |
+ The database ID of the parent repository | +12 |
+ accession, archival_object, assessment, assessment_attribute_definition, classification, classification_term, custom_report_template, default_values, digital_object, digital_object_component, event, group, job, preference, required_fields, resource, rights_statement, top_container |
+
Hierarchical relationships between other records are also expressed through foreign keys:
+ +| Column name | +Description | +Example | +PK Tables | +Found in | +
|---|---|---|---|---|
root_record_id |
+ The database ID of the root parent record | +4566 |
+ resource, digital_object, classification |
+ archival_object, digital_object_component, classification_term |
+
parent_id |
+ The database ID of the immediate parent record. This is used to identify parent records which are of the same type as the child record (i.e. two archival object records). The value will be NULL if the only parent is the root record. | +1748121 |
+ archival_object, classification_term, digital_object_component |
+ archival_object, classification_term, digital_object_component, note_persistent_id |
+
parent_name |
+ The database ID or URI, and the record type of the immediate parent | +144@archival_object, root@/repositories/2/resources/2 |
+ resource, archival_object, classification, classification_term, digital_object, digital_object_component |
+ archival_object, classification_term, digital_object_component |
+
Beginning with MySQL 8, you can recursively retrieve all parents of an archival object (or all archival objects linked to a resource) by running the following query:
+ +WITH RECURSIVE ao_path AS
+ (SELECT ao1.id
+ , ao1.display_string
+ , ao1.component_id
+ , ao1.parent_id
+ , ev.value as `ao_level`
+ , 1 as level
+ FROM archival_object ao1
+ LEFT JOIN enumeration_value ev on ev.id = ao1.level_id
+ WHERE ao1.id = <your ao id>
+ <!-- to get all trees for a resource change to: WHERE ao1.root_record_id = <your root_record_id> -->
+ UNION ALL
+ SELECT ao2.id
+ , ao2.display_string
+ , ao2.component_id
+ , ao2.parent_id
+ , ev.value as `ao_level`
+ , ao_path.level + 1 as level
+ FROM ao_path
+ JOIN archival_object ao2 on ao_path.parent_id = ao2.id
+ LEFT JOIN enumeration_value ev on ev.id = ao2.level_id)
+ SELECT GROUP_CONCAT(CONCAT(display_string, ' ', ' (', CONCAT(UPPER(SUBSTRING(ao_level,1,1)),LOWER(SUBSTRING(ao_level,2))), ' ', IF(component_id is not NULL, CAST(component_id as CHAR), "N/A"), ')') ORDER BY level DESC SEPARATOR ' > ') as tree
+ FROM ao_path;
+
+To retrieve all children (MySQL 8+):
+ +To retrieve both parents and children (MySQL 8+):
+ +To retrieve all parents of a record in MySQL 5.7 and below, run the following query:
+ +SELECT (SELECT GROUP_CONCAT(CONCAT(display_string, ' (', ao_level, ')') SEPARATOR ' < ') as parent_path
+ FROM (SELECT T2.display_string as display_string
+ , ev.value as ao_level
+ FROM (SELECT @r AS _id
+ , @p := @r AS previous
+ , (SELECT @r := parent_id FROM archival_object WHERE id = _id) AS parent_id
+ , @l := @l + 1 AS lvl
+ FROM ((SELECT @r := 1749840, @p := 0, @l := 0) AS vars,
+ archival_object h)
+ WHERE @r <> 0 AND @r <> @p) AS T1
+ JOIN archival_object T2 ON T1._id = T2.id
+ LEFT JOIN enumeration_value ev on ev.id = T2.level_id
+ WHERE T2.id != 1749840
+ ORDER BY T1.lvl DESC) as all_parents) as p_path
+ , ao.display_string
+ , CONCAT('/repositories/', ao.repo_id, '/archival_objects/', ao.id) as uri
+FROM archival_object ao
+WHERE ao.id = 1749840
+To retrieve all children of a record (MysQL 5.7 and below):
+ +The ordering of records in a resource, classification, or digital_object tree is determined by the position field. The position field is also used to order values in the enumeration_value and assessment_attribute_definition tables:
| Column name | +Description | +Example | +Found in | +
|---|---|---|---|
position |
+ The position of the archival object under the immediate parent | +168000 |
+ enumeration_value, assessment_attribute_definition, classification_term, digital_object_component, archival_object |
+
Many records and subrecords include fields which contain integers (0 or 1) corresponding to boolean values.
| Boolean fields | +Description | +Found in | +
|---|---|---|
publish |
+ + | subnote_metadata, file_version, external_document, accession, classification, agent_person, agent_family, agent_software, agent_corporate_entity, classification_term, revision_statement, repository, note, digital_object, digital_object_component, archival_object, resource |
+
suppressed |
+ + | accession, archival_object, assessment_reviewer_rlshp, assessment_rlshp, classification, classification_creator_rlshp, classification_rlshp, classification_term, classification_term_creator_rlshp, digital_object, digital_object_component, enumeration_value, event, event_link_rlshp, instance_do_link_rlshp, linked_agents_rlshp, location_profile_rlshp, owner_repo_rlshp, related_accession_rlshp, related_agents_rlshp, resource, spawned_rlshp, surveyed_by_rlshp, top_container_housed_at_rlshp, top_container_link_rlshp, top_container_profile_rlshp |
+
restrictions_apply |
+ + | accession, archival_object |
+
Several system generated, read-only fields appear across many tables. These include database identifiers, timestamps that track record creation and modification, and fields that record the username of the user that created and last modified the each record.
+ +| Most common read-only fields | +Description | +
|---|---|
id (primary key) |
+ Database identifier for each record | +
system_mtime |
+ The last time the record was modified by the system | +
created_by |
+ The user that created a record | +
last_modified_by |
+ The user that last modified a record | +
user_mtime |
+ The time that a record was last modified by a user | +
create_time |
+ The time that a record was created | +
lock_version |
+ This field is incrementally updated each time a record is updated. This provides a method of tracking updates and managing near-simultaneous edits by different users. | +
json_schema_version |
+ The JSON schema version | +
aspace_relationship_position |
+ The position of a linked record in a list of other linked records | +
is_slug_auto |
+ A boolean value that indicates whether a slug was auto-generated | +
system_generated |
+ A boolean value that indicates whether a field was system-generated | +
display_string |
+ A system-generated field which concatenates the title and date fields of an archival object record | +
NOTE: for subrecord tables these fields may hold unexpected data - because subrecords are deleted and recreated upon each save of a main or supporting record, their create and modification times will also be recreated and will not reflect the original creation date of the subrecord itself. For resource records, the timestamp only records the time that the resource itself was modified, not the last time any of its components were modified.
+ + + + +Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + architecture/backend/README.md + +
+ ++ + Report issue on Jira + architecture/backend/README.md + +
+The backend is responsible for implementing the ArchivesSpace API, and +supports the sort of access patterns shown in the previous section. +We’ve seen that the backend must support CRUD operations against a +number of different record types, and those records as expressed as +JSON documents produced from instances of JSONModel classes.
+ +The following sections describe how the backend fits together.
+ +The main.rb program is responsible for starting the ArchivesSpace
+system: loading all controllers and models, creating
+users/groups/permissions as needed, and preparing the system to handle
+requests.
When the system starts up, the main.rb program performs the
+following actions:
In addition to handling the system startup, main.rb also provides
+the following facilities:
X-ArchivesSpace-Session request header.The rest.rb module provides the mechanism used to define the API’s
+REST endpoints. Each endpoint definition includes:
Each controller in the system consists of one or more of these
+endpoint definitions. By using the endpoint syntax provided by
+rest.rb, the controllers can declare the interface they provide, and
+are freed of having to perform the sort of boilerplate associated
+with request handling–check parameter types, coerce values from
+strings into other types, and so on.
The main.rb and rest.rb components work together to insulate the
+controllers from much of the complexity of request handling. By the
+time a request reaches the body of an endpoint:
As touched upon in the previous section, controllers implement the +functionality of the ArchivesSpace API by registering one or more +endpoints. Each endpoint accepts a HTTP request for a given URI, +carries out the request and returns a JSON response (if successful) or +throws an exception (if something goes wrong).
+ +Each controller lives in its own file, and these can be found in the
+backend/app/controllers directory. Since most of the request
+handling logic is captured by the rest.rb module, controllers
+generally don’t do much more than coordinate the classes from the
+model layer and send a response back to the client.
Even though controllers are quite thin, there’s still a lot of overlap +in their behaviour. Each record type in the system supports the same +set of CRUD operations, and from the controller’s point of view +there’s not much difference between an update request for an accession +and an update request for a digital object (for example).
+ +The crud_helpers.rb module pulls this commonality into a set of
+helper methods that are invoked by each controller, providing methods
+for the standard operations of the system.
The backend’s model layer is where the action is. The model layer’s +role is to bridge the gap between the high-level JSONModel objects +(complete with their properties, nested records, references to other +records, etc.) and the underlying relational database (via the Sequel +database toolkit). As such, the model layer is mainly concerned with +mapping JSONModel instances to database tables in a way that preserves +everything and allows them to be queried efficiently.
+ +Each record type has a corresponding model class, but the individual +model definitions are often quite sparse. This is because the +different record types differ in the following ways:
+ +The first of these–the set of allowable properties–is already +captured by the JSONModel schema definitions, so the model layer +doesn’t have to enforce these restrictions. Each model can simply +take the values supplied by the JSONModel object it is passed and +assume that everything that needs to be there is there, and that +validation has already happened.
+ +The remaining two aspects are enforced by the model layer, but
+generally don’t pertain to just a single record type. For example, an
+accession may be linked to zero or more subjects, but so can several
+other record types, so it doesn’t make sense for the Accession model
+to contain the logic for handling subjects.
In practice we tend to see very little functionality that belongs +exclusively to a single record type, and as a result there’s not much +to put in each corresponding model. Instead, models are generally +constructed by combining a number of mix-ins (Ruby modules) to satisfy +the requirements of the given record type. Features à la carte!
+ +At a minimum, every model includes the ASModel mix-in, which provides
+base versions of the following methods:
Model.create_from_json – Take a JSONModel instance and create a
+model instance (a subclass of Sequel::Model) from it. Returns the
+instance.model.update_from_json – Update the target model instance with
+the values from a given JSONModel instance.Model.sequel_to_json – Return a JSONModel instance of the appropriate
+type whose values are taken from the target model instance.
+Model classes are declared to correspond to a particular JSONModel
+instance when created, so this method can automatically return a
+JSONModel instance of the appropriate type.These methods comprise the primary interface of the model layer: +virtually every mix-in in the model layer overrides one or all of +these to add behaviour in a modular way.
+ +For example, the ‘notes’ mix-in adds support for multiple notes to be +added to a record type–by mixing this module into a model class, that +class will automatically accept a JSONModel property called ‘notes’ +that will be stored and retrieved to and from the database as needed. +This works by overriding the three methods as follows:
+ +Model.create_from_json – Call ‘super’ to delegate the creation to
+the next mix-in in the chain. When it returns the newly created
+object, extract the notes from the JSONModel instance and attach
+them to the model instance (saving them in the database).model.update_from_json – Call ‘super’ to save the other updates
+to the database, then replace any existing notes entries for the
+record with the ones provided by the JSONModel.Model.sequel_to_json – Call ‘super’ to have the next mix-in in
+the chain create a JSONModel instance, then pull the stored notes
+from the database and poke them into it.All of the mix-ins follow this pattern: call ‘super’ to delegate the +call to the next mix-in in the chain (eventually reaching ASModel), +then manipulate the result to implement the desired behaviour.
+ +Some record types, like accessions, digital objects, and subjects, are +top-level records, in the sense that they are created independently +of any other record and are addressable via their own URI. However, +there are a number of records that can’t exist in isolation, and only +exist in the context of another record. When one record can contain +instances of another record, we call them nested records.
+ +To give an example, the date record type is nested within an
+accession record (among others). When the model layer is asked to
+save a JSONModel instance containing nested records, it must pluck out
+those records, save them in the appropriate database table, and ensure
+that linkages are created within the database to allow them to be
+retrieved later.
This happens often enough that it would be tedious to write code for
+each model to handle its nested records, so the ASModel mix-in
+provides a declaration to handle this automatically. For example, the
+accession model uses a definition like:
base.def_nested_record(:the_property => :dates,
+ :contains_records_of_type => :date,
+ :corresponding_to_association => :date)
+When creating an accession, this declaration instructs the Accession
+model to create a database record for each date listed in the “dates”
+property of the incoming record. Each of these date records will be
+automatically linked to the created accession.
A relationship is a link between two top-level records, where the link +is a separate, dynamically generated, model with zero or more +properties of its own.
+ +For example, the Event model can be related to several different
+types of records:
define_relationship(:name => :event_link,
+ :json_property => 'linked_records',
+ :contains_references_to_types => proc {[Accession, Resource, ArchivalObject]})
+This declaration generates a custom class that models the relationship
+between events and the other record types. The corresponding JSON
+schema declaration for the linked_records property looks like this:
"linked_records" => {
+ "type" => "array",
+ "ifmissing" => "error",
+ "minItems" => 1,
+ "items" => {
+ "type" => "object",
+ "subtype" => "ref",
+ "properties" => {
+ "role" => {
+ "type" => "string",
+ "dynamic_enum" => "linked_event_archival_record_roles",
+ "ifmissing" => "error",
+ },
+ "ref" => {
+ "type" => [{"type" => "JSONModel(:accession) uri"},
+ {"type" => "JSONModel(:resource) uri"},
+ {"type" => "JSONModel(:archival_object) uri"},
+ ...],
+ "ifmissing" => "error"
+ },
+ ...
+That is, the property includes URI references to other records, plus +an additional “role” property to indicate the nature of the +relationship. The corresponding JSON might then be:
+ +linked_records: [{ref: '/repositories/123/accessions/456', role: 'authorizer'}, ...]
+The define_relationship definition automatically makes use of the
+appropriate join tables in the database to store this relationship and
+retrieve it later as needed.
agent_manager.rbAgents present a bit of a representational challenge. There are four
+types of agents (person, family, corporate entity, software), and at a
+high-level they are structured in the same way: each type can contain
+one or more name records, zero or more contact records, and a number
+of properties. Records that link to agents (via a relationship, for
+example) can link to any of the four types so, in some sense, each
+agent type implements a common Agent interface.
However, the agent types differ in their details. Agents contain name +records, but the types of those name records correspond to the type of +the agent: a person agent contains a person name record, for example. +So, in spite of their similarities, the different agents need to be +modelled as separate record types.
+ +The agent_manager module captures the high-level similarities
+between agents. Each agent model includes the agent manager mix-in:
include AgentManager::Mixin
+and then defines itself declaratively by the provided class method:
+ + register_agent_type(:jsonmodel => :agent_person,
+ :name_type => :name_person,
+ :name_model => NamePerson)
+This definition sets up the properties of that agent. It creates:
+ +As records are added to and updated within the ArchivesSpace system, +they are validated against a number of rules to make sure they are +well-formed and don’t conflict with other records. There are two +types of record validation:
+ +Record-level validations can be performed in isolation, while +system-level records require comparing the record to others in the +database.
+ +System-level validations need to be implemented in the database itself +(as integrity constraints), but record-level validations are often too +complex to be expressed this way. As a result, validations in +ArchivesSpace can appear in one or both of the following layers:
+ +common/validations.rb file, allowing validation
+logic to be expressed using arbitrary Ruby code.As a general rule, record-level validations are handled by the +JSONModel validations (either through the JSON schema or custom +validations), while system-level validations are handled by the model +and the database schema.
+ +Updating a record using the ArchivesSpace API is a two part process:
+ + # Perform a `GET` against the desired record to fetch its JSON
+ # representation:
+
+ GET /repositories/5/accessions/2
+
+ # Manipulate the JSON representation as required, and then `POST`
+ # it back to replace the original:
+
+ POST /repositories/5/accessions/2
+If two people do this simultaneously, there’s a risk that one person
+would silently overwrite the changes made by the other. To prevent
+this, every record is marked with a version number that it carries in
+the lock_version property. When the system receives the updated
+copy of a record, it checks that the version it carries is still
+current; if the version number doesn’t match the one stored in the
+database, the update request is rejected and the user must re-fetch
+the latest version before applying their update.
The ArchivesSpace backend enforces access control, defining which +users are allowed to create, read, update, suppress and delete the +records in the system. The major actors in the permissions model are:
+ +update_accession_record permission is allowed to
+update accessions for a repository.To summarize, a user can perform an action within a repository if they +are a member of a group that has been assigned permission to perform +that action.
+ +Since they’re repository-scoped, groups govern access to repositories. +However, there are several record types that exist at the top-level of +the system (such as the repositories themselves, subjects and agents), +and the permissions model must be able to accommodate these.
+ +To get around this, we invent a concept: the “global” repository +conceptually contains the whole ArchivesSpace universe. As with other +repositories, the global repository contains groups, and users can be +made members of these groups to grant them permissions across the +entire system. One example of this is the “admin” user, which is +granted all permissions by the “administrators” group of the global +repository; another is the “search indexer” user, which can read (but +not update or delete) any record in the system.
+ + +Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + architecture/directories.md + +
+ + +ArchivesSpace is made up of several components that are kept in separate directories.
+ +This directory contains the code for the documentation tool used to generate the github io pages here: http://archivesspace.github.io/archivesspace/
+ +This directory contains the code that handles the database and the API.
+ +This directory contains the code used to build the application. It includes the commands that are used to run the development servers, the test suites, and to build the releases. ArchivesSpace is a JRuby application and Apache Ant is used to build it.
+ +This directory contains code that can be used when clustering an ArchivesSpace installation.
+ +This directory contains code that is used across two or more of the components. It includes configuration options, database schemas and migrations, and translation files.
+ +This directory contains the documentation and PDFs of the license agreement files.
+ +This directory contains documentation files that are included in a release.
+ +This directory contains the staff interface Ruby on Rails application.
+ +This directory contains the indexer Sinatra application.
+ +This directory contains an example that can be used to set up Apache JMeter to load test functional behavior and measure performance.
+ +This directory contains the code that launches (starts, restarts, and stops) an ArchivesSpace application.
+ +This directory contains the OAI-PMH Sinatra application.
+ +This directory contains ArchivesSpace Program Team supported plugins.
+ +This directory contains the Docker proxy code.
+ +This directory contains the public interface Ruby on Rails application.
+ +This directory contains the reports code.
+ +This directory contains scripts necessary for building, deploying, and other ArchivesSpace tasks.
+ +This directory contains the selenium tests.
+ +This directory contains the solr code.
+ +This directory contains XSL stylesheets used by ArchivesSpace.
+ +This directory contains a tool that can be used to run the development servers.
+ + +Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + architecture/frontend/README.md + +
+ ++ + Report issue on Jira + architecture/frontend/README.md + +
+This document provides an overview of the parts of the ArchivesSpace codebase which control the frontend/staff interface. For guidance on using the ArchivesSpace staff interface, consult the ArchivesSpace Help Center (ArchivesSpace members only).
+ +++ + +Additional documentation needed
+
Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + + + + +ArchivesSpace is divided into several components: the backend, which +exposes the major workflows and data types of the system via a +REST API, a staff interface, a public interface, and a search system, +consisting of Solr and an indexer application.
+ +These components interact by exchanging JSON data. The format of this +data is defined by a class called JSONModel.
+ +ArchivesSpace components are constructed using several programming languages, platforms, and additional open source projects.
+ +The languages used are Java, JRuby, Ruby, JavaScript, and CSS.
+ +The backend, OAI harvester, and indexer are Sinatra apps. The staff and public user interfaces are Ruby on Rails apps.
+ +The database used out of the box and for testing is Apache Derby. The database suggested for production is MySQL. The index platform is Apache Solr.
+ +ArchivesSpace is made up of several components that are kept in separate directories.
+ +This directory contains the code for the documentation tool used to generate the github io pages here: https://archivesspace.github.io/archivesspace/api/ and https://archivesspace.github.io/archivesspace/doc/
+ +This directory contains the code that handles the database and the API.
+ +This directory contains the code used to build the application. It includes the commands that are used to run the development servers, the test suites, and to build the releases. ArchivesSpace is a JRuby application and Apache Ant is used to build it.
+ +This directory contains code that can be used when clustering an ArchivesSpace installation.
+ +This directory contains code that is used across two or more of the components. It includes configuration options, database schemas and migrations, and translation files.
+ +This directory contains the documentation and PDFs of the license agreement files.
+ +This directory contains documentation files that are included in a release.
+ +This directory contains the staff interface Ruby on Rails application.
+ +This directory contains the indexer Sinatra application.
+ +This directory contains an example that can be used to set up Apache JMeter to load test functional behavior and measure performance.
+ +This directory contains the code that launches (starts, restarts, and stops) an ArchivesSpace application.
+ +This directory contains the OAI-PMH Sinatra application.
+ +This directory contains ArchivesSpace Program Team supported plugins.
+ +This directory contains the Docker proxy code.
+ +This directory contains the public interface Ruby on Rails application.
+ +This directory contains the reports code.
+ +This directory contains scripts necessary for building, deploying, and other ArchivesSpace tasks.
+ +This directory contains the selenium tests.
+ +This directory contains the solr code.
+ +This directory contains XSL stylesheets used by ArchivesSpace.
+ +This directory contains a tool that can be used to run the development servers.
+ + +Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + architecture/jobs/README.md + +
+ + +ArchivesSpace provides a mechanism for long running processes to run
+asynchronously. These processes are called Background Jobs.
The Create menu has a Background Job option which shows a submenu of job
+types that the current user has permission to create. (See below for more
+ information about job permissions and hidden jobs.) Selecting one of these
+ options will take the user to a form to enter any parameters required for the
+ job and then to create it.
When a job is created it is placed in the Background Job Queue. Jobs in the
+queue will be run in the order they were created. (See below for more
+ information about multiple threads and concurrent jobs.)
The Browse menu has a Background Jobs option. This takes the user to a list
+of jobs arranged by their status. The user can then view the details of a job,
+and cancel it if they have permission.
A user must have the create_job permission to create a job. By default, this
+permission is included in the repository_basic_data_entry group.
A user must have the cancel_job permission to cancel a job. By default, this
+permission is included in the repository_managers group.
When a JobRunner registers it can specify additional create and cancel +permissions. (See below for more information)
+ +Each job has a type, and each type has a registered runner to run jobs of that +type and JSONModel schema to define its parameters.
+ +All jobs of a type are handled by a registered JobRunner. The job runner
+classes are located here:
backend/app/lib/job_runners/
+It is possible to define additional job runners from a plugin. (See below for + more information about plugins.)
+ +A job runner class must subclass JobRunner, register to run one or more job
+types, and implement a #run method for jobs that it handles.
When a job runner registers for a job type, it can set some options:
+ +:hidden
+ false:run_concurrently
+ false:create_permissions
+ []create_job, to create jobs of this type.:cancel_permissions
+ []cancel_job, to cancel jobs of this type.For more information about defining a job runner, see the JobRunner superclass:
backend/app/lib/job_runner.rb
+A job type also requires a JSONModel schema that defines the parameters to run a +job of the type. The schema name must be the same as the type that the runner +registers for. For example:
+ + common/schemas/import_job.rb
+This schema, for JSONModel(:import_job), defines the parameters for running a
+job of type import_job.
ArchivesSpace can be configured to run more than one background job at a time. +By default, there will be two threads available to run background jobs. +The configuration looks like this:
+ + AppConfig[:job_thread_count] = 2
+The BackgroundJobQueue will start this number of threads at start up. Those
+threads will then poll for queued jobs and run them.
When a job runner registers, it can set an option called :run_concurrently.
+This is false by default. When set to false a job thread will not run a job
+if there is already a job of that type running. The job will remain on the queue
+and will be run when there are no longer any jobs of its type running.
When set to true a job will be run when it comes to the front of the queue
+regardless of whether there is a job of the same type running.
It is possible to add a new job type from a plugin. ArchivesSpace includes a +plugin that demonstrates how to do this:
+ + plugins/jobs_example
+Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + architecture/jsonmodel.md + +
+ + +The ArchivesSpace system is concerned with managing a number of +different archival record types. Each record can be expressed as a +set of nested key/value pairs, and associated with each record type is +a number of rules that describe what it means for a record of that +type to be valid:
+ +The JSONModel class provides a common language for expressing these +rules that all parts of the application can share. There is a +JSONModel class instance for each type of record in the system, so:
+ +JSONModel(:digital_object)
+is a class that knows how to take a hash of properties and make sure +those properties conform to the specification of a Digital Object:
+ +JSONModel(:digital_object).from_hash(myhash)
+If it passes validation, a new JSONModel(:digital_object) instance is +returned, which provides accessors for accessing its values, and +facilities for round-tripping between JSON documents and regular Ruby +hashes:
+ + obj = JSONModel(:digital_object).from_hash(myhash)
+
+ obj.title # or obj['title']
+ obj.title = 'a new title' # or obj['title'] = 'a new title'
+
+ obj.\_exceptions # Validates the object and reports any issues
+
+ obj.to_hash # Turn the JSONModel object back into a regular hash
+ obj.to_json # Serialize the JSONModel object into JSON
+Much of the validation performed by JSONModel is provided by the JSON
+schema definitions listed in the common/schemas directory. JSON
+schemas provide a standard way of declaring which properties a record
+may and may not contain, along with their types and other
+restrictions. ArchivesSpace uses these schemas to capture the
+validation rules defining each record type in a declarative and
+relatively self-documenting fashion.
JSONModel instances are the primary data interchange mechanism for the +ArchivesSpace system: the API consumes and produces JSONModel +instances (in JSON format), and much of the user interface’s life is +spent turning forms into JSONModel instances and shipping them off to +the backend.
+ +To save the need for a lot of HTTP request wrangling, ArchivesSpace +ships with a module called JSONModel::Client that simplifies the +common CRUD-style operations. Including this module just requires +passing an additional parameter when initializing JSONModel:
+ + JSONModel::init(:client_mode => true, :url => @backend_url)
+ include JSONModel
+If you’ll be working against a single repository, it’s convenient to +set it as the default for subsequent actions:
+ + JSONModel.set_repository(123)
+Then, several additional JSONModel methods are available:
+ + # As before, get a paginated list of accessions (GET)
+ JSONModel(:accession).all(:page => 1)
+
+ # Create a new accession (POST)
+ obj = JSONModel(:accession).from_hash(:title => "A new accession", ...)
+ obj.save
+
+ # Get a single accession by ID (GET)
+ obj = JSONModel(:accession).find(123)
+
+ # Update an existing accession (POST)
+ obj = JSONModel(:accession).find(123)
+ obj.title = "Updated title"
+ obj.save
+Technical documentation for ArchivesSpace
+ + +View the Project on GitHub archivesspace/tech-docs
+ + + + + + ++ + Edit this page on GitHub + architecture/languages.md + +
+ + +ArchivesSpace components are constructed using several programming languages, platforms, and additional open source projects.
+ +The languages used are Java, JRuby, Ruby, JavaScript, and CSS.
+ +The backend, OAI harvester, and indexer are Sinatra apps. The staff and public user interfaces are Ruby on Rails apps.
+ +The database used out of the box and for testing is Apache Derby. The database suggested for production is MySQL. The index platform is Apache Solr.
+ + +`, ``, ``, ` Technical documentation for ArchivesSpace View the Project on GitHub archivesspace/tech-docs
+
+ Edit this page on GitHub
+ architecture/oai-pmh/README.md
+
+
+
+ Report issue on Jira
+ architecture/oai-pmh/README.md
+
+ A starter OAI-PMH interface for ArchivesSpace allowing other systems to harvest
+your records is included in version 2.1.0. Additional features and functionality
+will be added in later releases. By default, the OAI-PMH interface runs on port 8082. A sample request page is
+available at http://localhost:8082/sample. (To access it, make sure that you
+have set the AppConfig[:oai_proxy_url] appropriately.) The system provides responses to a number of standard OAI-PMH requests,
+including GetRecord, Identify, ListIdentifiers, ListMetadataFormats,
+ListRecords, and ListSets. Unpublished and suppressed records and elements are
+not included in any of the OAI-PMH responses. Some responses require the URL parameter metadataPrefix. There are five
+different metadata responses available: The EAD response for resources and MARC response for resources and archival
+objects use the mappings from the built-in exporter for resources. The DC,
+DCMI terms, and MODS responses for resources and archival objects use mappings
+suggested by the community. Here are some example URLs and other information for these requests: GetRecord – needs a record identifier and metadataPrefix
+ Up to ArchivesSpace v3.5.1 OAI identifiers are in this format: Starting with ArchivesSpace v4.0.0 OAI identifiers are in the new format (notice the colon after the see also: https://github.com/code4lib/ruby-oai/releases/tag/v1.0.0 Identify ListIdentifiers – needs a metadataPrefix ListMetadataFormats ListRecords – needs a metadataPrefix ListSets Harvesting the ArchivesSpace OAI-PMH server without specifying a set will yield
+all published records across all repositories.
+Predefined sets can be accessed using the set parameter. In order to retrieve
+records from sets include a set parameter in the URL and the DC metadataPrefix,
+such as “&set=collection&metadataPrefix=oai_dc”. These sets can be from
+configured sets as shown above or from the following levels of description: In addition to the sets based on level of description, you can define sets
+based on repository codes and/or sponsors in the config/config.rb file: The interface implements resumption tokens for pagination of results. As an
+example, the following URL format should be used to page through the results
+from a ListRecords request: using the resumption token: Note: you do not use the metadataPrefix when you use the resumptionToken The ArchivesSpace OAI-PMH server supports persistent deletes, so harvesters
+will be notified of any records that were deleted since
+they last harvested. Mixed content is removed from Dublin Core, dcterms, MARC, and MODS field outputs
+in the OAI-PMH response (e.g., a scope note mapped to a DC description field
+ would not include The component level records include inherited data from superior hierarchical
+levels of the finding aid. Element inheritance is determined by institutional
+system configuration (editable in the config/config.rb file) as implemented for
+the Public User Interface. ARKs have not yet been implemented, pending more discussion of how they should
+be formulated.`,
- `
`, `
`,
- `
`, `
+
+
+
+
+
+
+
+
+tech-docs
+
+
+
+ OAI-PMH interface
+
+
+
+
+http://localhost:8082/oai?verb=GetRecord&identifier=oai:archivesspace//repositories/2/resources/138&metadataPrefix=oai_eadoai:archivesspace namespace part of the identifier):http://localhost:8082/oai?verb=GetRecord&identifier=oai:archivesspace:/repositories/2/resources/138&metadataPrefix=oai_eadhttp://localhost:8082/oai?verb=Identifyhttp://localhost:8082/oai?verb=ListIdentifiers&metadataPrefix=oai_dchttp://localhost:8082/oai?verb=ListMetadataFormatshttp://localhost:8082/oai?verb=ListRecords&metadataPrefix=oai_dctermshttp://localhost:8082/oai?verb=ListSets
+
+
+
+
+AppConfig[:oai_sets] = {
+'repository_set' => {
+ :repo_codes => ['hello626'],
+ :description => "A set of one or more repositories",
+},
+'sponsor_set' => {
+ :sponsors => ['The_Sponsor'],
+ :description => "A set of one or more sponsors",
+}
+}
+
+
+`http://localhost:8082/oai?verb=ListRecords&metadataPrefix=oai_ead`
+
+
+`http://localhost:8082/oai?verb=ListRecords&resumptionToken=eyJtZXRhZGF0YV9wcmVmaXgiOiJvYWlfZWFkIiwiZnJvbSI6IjE5NzAtMDEtMDEgMDA6MDA6MDAgVVRDIiwidW50aWwiOiIyMDE3LTA3LTA2IDE3OjEwOjQxIFVUQyIsInN0YXRlIjoicHJvZHVjaW5nX3JlY29yZHMiLCJsYXN0X2RlbGV0ZV9pZCI6MCwicmVtYWluaW5nX3R5cGVzIjp7IlJlc291cmNlIjoxfSwiaXNzdWVfdGltZSI6MTQ5OTM2MTA0Mjc0OX0=`
+<p>, <abbr>, <address>, <archref>, <bibref>, <blockquote>,
+ <chronlist>, <corpname>, <date>, <emph>, <expan>, <extptr>, <extref>,
+ <famname>, <function>, <genreform>, <geogname>, <lb>, <linkgrp>, <list>,
+ <name>, <note>, <num>, <occupation>, <origination>, <persname>, <ptr>, <ref>, <repository>, <subject>, <table>, <title>, <unitdate>, <unittitle>).