Releases: 4dn-dcic/snovault
1.2.8
Fixed an issue with MPIndexer.update_objects
that would cause some processes to be lost when counter did not increment but indexing was deferred. Also added a fix to Indexer.update_objects
to actually return indexing errors, though this will only matter if single-process indexing is used.
1.2.7
Add indexing_stats
dictionary to document returned from /index-data
. It times various parts of the indexing progress.
1.2.6
Use multiprocessing.cpu_count - 2
processes for the MPIndexer. Minimum number of processes is 1.
1.2.5
Fixed incorrect initialization of IndexerQueue.target_queues
so now they are correctly ordered. Slight test update.
1.2.4
Up SEARCH_MAX to support more pages.
1.2.3
Small release to fix ES SEARCH_MAX
discrepancy between create_mapping and rev link searching function.
1.2.2
Release that improves indexing performance.
- Sid checking now happens in indexer before
/index-data
view is called - Bugfix for the indexer that would catch KeyError/MissingIndexItemException before SidException
- Bugfix for Indexer_queue.queue_targets not being ordered
- es_index_listener now checks queue counts before calling
/index
- MPIndexer now always uses
multiprocessing.cpu_count() - 1
for number if processes in pool - Some other random refactoring
1.2.1
Small release adding mimetype_map
to ItemWithAttachment
class to allow handling of mimetypes outside of the python-magic
scope.
1.2.0
Merge pull request #101 from 4dn-dcic/single_txn_indexing 1.2.0 -- Remove deferred queue