v1.14.0

1.14.0 - 2021-01-07

Added

• Added a new /api/reports/metrics/extractors report for summarizing extractor usage by user. Database administrators
  can use scripts/updates/UpdateUserId.js to assign user IDs to older extraction event records based on resource ownership
  in order to improve the accuracy of the report for older data.

Changed

• api/reports/storage/spaces endpoint now accepts a space parameter for ID rather than requiring a space filter.
• Datasets and collections in the trash are no longer indexed for discovery in search services.

v1.13.0

1.13.0 - 2020-12-02

Added

• Ability to submit multiple selected files within a dataset to an extractor.
• Support for Amplitude clickstream tracking. See Admin -> Customize to configure Amplitude apikey.
• UpdateUserId.js to scripts/updates. This code adds user_id to each document in extractions collection in mongodb.
  user_id is taken from author id in uploads.files if exists, else it taken from author id in datasets collection.

Fixed

• An extractor with file matching set to */* (all file types) would incorrectly send out dataset events.
• Space Editors can now delete tags on files, datasets and sections.
• GeospatialViewer previewer no longer shows if file does not contain geospatial layers.

v1.12.2

1.12.2 - 2020-11-19

Changed

• /api/reindex admin endpoint no longer deletes and swaps a temporary index, but reindexes in-place.

v1.12.1

1.12.1 - 2020-11-05

Fixed

• Error uploading to spaces that did not have extractors enabled/disabled (personel spaces).
• If extractor does not have any parameters, there would be an error message in the console of the browser.
• If the extractor did not have a user_id it would create an error and not record the event.

Changed

• Docker Images are now pushed to github container registry

v1.12.0

1.12.0 - 2020-10-19

Warning:

• This update modifies the MongoDB schema. Make sure to start the application with -DMONGOUPDATE=1.
• This update modifies information stored in Elasticsearch used for text based searching. Make sure to initiate a reindex
  of Elasticsearch from the Admin menu or by POST /api/reindex.

Added

• Global extractors page now shows more information, including submission metrics, logs (using Graylog), job history and
  extractors maturity. Extractors can be grouped using labels. User can filter list of extractors by labels, space, trigger
  and metadata key.
• Users have more refined options to set extractors triggers at the space level. They can now follow global settings,
  disable and enable triggers.
• Ability to set chunksize when downloading files. Set defult to 1MB from 8KB. This will result in faster downloads and
  less CPU usage at the cost of slightly more memory use.
• Support for parsing of Date and Numeric data in new metadata fields. New search operators <, >, <=, >= have been
  added to search API now that they can be compared properly.
• Track user_id with every extraction event. #94
• Added a new storage report at GET api/reports/storage/spaces/:id for auditing user storage usage on a space basis.
• The file and dataset metrics reports also have support for since and until ISO8601 date parameters.
• Added viewer_hop a 3D models previewer for *.ply and *.nxz files. Added mimetype.nxz=model/nxz and
  mimetype.NXZ=model/nxz as new mimetypes in conf/mimetypes.conf

Fixed

• Ignore the update field when posting to /api/extractors. #89
• Search results were hardcoded to be in batches of 2.
• Fixed permissions checks on search results for search interfaces that would cause misleading counts. #60

v1.11.2

1.11.2 - 2020-10-13

Fixed

• Clowder healthcheck was not correct, resulting in docker-compose never thinking it was healthy. This could also result in traefik not setting up the routes.

v1.11.1

1.11.1 - 2020-09-29

Added

• Added healtz endpoint that is cheap and quick to return, useful for kubernetes live/ready checks.

Fixed

• Fixed health check script when using custom path prefix.
• Proxy will no correctly handle paths that end with a / at the end.
• Submitting an extraction will always return a 500 error, see #84
• Added MongoDB index for folders.files.

Changed

• Updated update-clowder script to work with migration to github. Has the ability now to push a message to MSTEAMS as well as influxdb.

v1.11.0

1.11.0 - 2020-08-31

Added

• Downloaded datasets now include DataCite v4 XML files in the output /metadata folder.
• Script to clean extractors' tmp files scripts/clean-extractors-tmpfiles/.
• Script for RabbitMQ error queue cleanup scripts/rmq-error-shovel/.
• Ability to use HTML formatting in the welcome message on the home page. #51
• Expose a read-only list of extractors to all users.

Changed

• Improved test script scripts/tester/tester.sh to report successes once a day.

Fixed

• Escape colon characters on search values for search box and advanced search to allow those values in a search.
• Typesafe now only offers https access. #49
• If uploading files by url > 2147483647 it would fail. #54

v1.10.1

Fixed

• Queue threads (e.g. Elasticsearch indexer) will no longer crash permanently if the queue connection to Mongo is lost temporarily.
• Docker images would not build correctly on GitHub.
• If monitor HTTP server would crash, it would not restart correctly.
• Don't call server side twice when rendering list of files on dataset page. #7
• Fixed Sphinx build errors and switched to using pipenv. Now building docs on readthedocs.

Added

• GitHub artifacts can be uploaded using SCP to remote server.

v1.10.0

Added

• Ability to mark multiple files in a dataset and perform bulk operations (download, tag, delete) on them at once.

Fixed

• Return thumbnail as part of the file information.
  #8
• Datasets layout on space page would sometimes have overlapping tiles.

Changed

• mongo-init script with users would return with exit code -1 if user exists, now returns exit code 0.