From e3b21d399d072ac2ae56ec40f488daadd92e2110 Mon Sep 17 00:00:00 2001 From: Alan Protasio Date: Wed, 10 Aug 2022 20:01:25 -0700 Subject: [PATCH] Remove support for chunks storage entirely: Take 2 (#4812) * Fix updateCachedShippedBlocks - new thanos Signed-off-by: Alan Protasio * Remove support for chunks ingestion Signed-off-by: Andrew Bloomgarden * Remove blocksconvert Signed-off-by: Andrew Bloomgarden * Remove most all of chunks storage Signed-off-by: Andrew Bloomgarden * Remove old legacy object store client and uses Signed-off-by: Andrew Bloomgarden * Start removing chunks docs Signed-off-by: Andrew Bloomgarden * Remove HTTP APIs that were chunks-only Signed-off-by: Andrew Bloomgarden * Remove more chunks docs Signed-off-by: Andrew Bloomgarden * Remove deprecated and chunks configs Signed-off-by: Andrew Bloomgarden * Draft changelog Signed-off-by: Andrew Bloomgarden * Fix up integration tests Signed-off-by: Andrew Bloomgarden * Clean up lint warnings Signed-off-by: Andrew Bloomgarden * go mod tidy && go mod vendor Signed-off-by: Andrew Bloomgarden * Remove chunks storage from bug report template Signed-off-by: Andrew Bloomgarden * Remove most chunk encoding/decoding code Signed-off-by: Andrew Bloomgarden * Remove unused cardinality limit flag Signed-off-by: Andrew Bloomgarden * Document removed flags in the CHANGELOG Signed-off-by: Andrew Bloomgarden * rebasing Signed-off-by: Alan Protasio * Fix Ingester Proto Signed-off-by: Alan Protasio * Removing chunk storage config Signed-off-by: Alan Protasio * Removing tsdb_test.go Signed-off-by: Alan Protasio * go mod Signed-off-by: Alan Protasio * Removing cassandra referenves from the integ tests Signed-off-by: Alan Protasio * Keep MIgrating Storage Documentation Signed-off-by: Alan Protasio * Removing dangling chunks config Signed-off-by: Alan Protasio * Update Changelog Signed-off-by: Alan Protasio * Removing blocks convert from gitignore and makfile local Signed-off-by: Alan Protasio * Remove dead code Signed-off-by: Alan Protasio Signed-off-by: Alan Protasio Signed-off-by: Andrew Bloomgarden Co-authored-by: Andrew Bloomgarden Co-authored-by: Alvin Lin --- .github/ISSUE_TEMPLATE/bug_report.md | 4 - .github/workflows/test-build-deploy.yml | 10 +- .gitignore | 1 - CHANGELOG.md | 38 +- Makefile | 7 +- Makefile.local.example | 3 - cmd/blocksconvert/Dockerfile | 9 - cmd/blocksconvert/main.go | 107 - docs/api/_index.md | 80 +- docs/architecture.md | 37 +- docs/blocks-storage/_index.md | 15 +- docs/blocks-storage/querier.md | 11 +- docs/blocks-storage/querier.template | 2 +- docs/chunks-storage/_index.md | 46 - docs/chunks-storage/aws-tips.md | 91 - docs/chunks-storage/caching.md | 201 - .../chunks-storage-getting-started.md | 200 - docs/chunks-storage/ingesters-with-wal.md | 117 - .../running-chunks-storage-in-production.md | 85 - .../running-chunks-storage-with-cassandra.md | 182 - docs/chunks-storage/schema-config.md | 151 - docs/chunks-storage/table-manager.md | 71 - docs/configuration/arguments.md | 71 +- docs/configuration/config-file-reference.md | 1712 +- .../single-process-config-blocks-local.yaml} | 73 +- docs/configuration/v1-guarantees.md | 3 +- docs/guides/capacity-planning.md | 2 + docs/guides/deleting-series.md | 73 - docs/guides/encryption-at-rest.md | 10 +- docs/guides/encryption-at-rest.template | 4 - docs/guides/glossary.md | 27 +- docs/guides/grpc-storage-plugin.md | 44 - docs/guides/ingesters-rolling-updates.md | 77 - docs/guides/ingesters-scaling-up-and-down.md | 30 +- docs/guides/limitations.md | 4 - docs/proposals/ingesters-migration.md | 4 +- go.mod | 10 - go.sum | 8 - integration/alertmanager_test.go | 249 +- integration/api_endpoints_test.go | 5 +- integration/asserts.go | 13 +- integration/backward_compatibility_test.go | 112 +- integration/certs.go | 51 - integration/configs.go | 132 +- integration/e2ecortex/services.go | 35 +- ...tegration_memberlist_single_binary_test.go | 14 +- integration/querier_remote_read_test.go | 21 +- .../querier_streaming_mixed_ingester_test.go | 164 - integration/querier_test.go | 18 +- integration/ruler_test.go | 173 +- integration/s3_storage_client_test.go | 136 - pkg/alertmanager/alertstore/config.go | 45 - .../alertstore/objectclient/store.go | 163 - .../alertstore/objectclient/store_test.go | 4 - pkg/alertmanager/alertstore/store.go | 40 - pkg/alertmanager/alertstore/store_test.go | 18 - pkg/alertmanager/multitenant.go | 12 +- pkg/alertmanager/multitenant_test.go | 7 - pkg/api/api.go | 20 - pkg/api/handlers.go | 2 +- pkg/api/middlewares.go | 2 +- pkg/chunk/aws/dynamodb_index_reader.go | 237 - pkg/chunk/aws/dynamodb_metrics.go | 60 - pkg/chunk/aws/dynamodb_storage_client.go | 819 - pkg/chunk/aws/dynamodb_storage_client_test.go | 43 - pkg/chunk/aws/dynamodb_table_client.go | 387 - pkg/chunk/aws/fixtures.go | 96 - pkg/chunk/aws/metrics_autoscaling.go | 378 - pkg/chunk/aws/metrics_autoscaling_test.go | 554 - pkg/chunk/aws/mock.go | 424 - pkg/chunk/aws/retryer.go | 52 - pkg/chunk/aws/s3_storage_client.go | 418 - pkg/chunk/aws/s3_storage_client_test.go | 79 - pkg/chunk/aws/sse_config.go | 66 - pkg/chunk/aws/sse_config_test.go | 92 - pkg/chunk/azure/blob_storage_client.go | 284 - pkg/chunk/bucket_client.go | 11 - pkg/chunk/cache/cache_test.go | 106 +- pkg/chunk/cassandra/authenticator.go | 43 - pkg/chunk/cassandra/fixtures.go | 76 - pkg/chunk/cassandra/instrumentation.go | 38 - pkg/chunk/cassandra/storage_client.go | 565 - pkg/chunk/cassandra/storage_client_test.go | 155 - pkg/chunk/cassandra/table_client.go | 81 - pkg/chunk/cassandra/table_client_test.go | 48 - .../cassandra/testdata/example.com-key.pem | 27 - .../cassandra/testdata/example.com.ca.pem | 25 - pkg/chunk/cassandra/testdata/example.com.pem | 27 - .../password-with-trailing-newline.txt | 1 - .../password-without-trailing-newline.txt | 1 - pkg/chunk/chunk.go | 331 +- pkg/chunk/chunk_store.go | 714 - pkg/chunk/chunk_store_test.go | 1150 - pkg/chunk/chunk_store_utils.go | 273 - pkg/chunk/chunk_test.go | 295 +- pkg/chunk/composite_store.go | 271 - pkg/chunk/composite_store_test.go | 291 - pkg/chunk/encoding/bigchunk.go | 352 - pkg/chunk/encoding/bigchunk_test.go | 96 - pkg/chunk/encoding/chunk.go | 191 +- pkg/chunk/encoding/chunk_test.go | 84 +- pkg/chunk/encoding/delta_helpers.go | 87 - pkg/chunk/encoding/doubledelta.go | 546 - pkg/chunk/encoding/factory.go | 102 +- pkg/chunk/encoding/prometheus_chunk.go | 29 +- pkg/chunk/encoding/varbit.go | 1229 - pkg/chunk/encoding/varbit_helpers.go | 78 - pkg/chunk/encoding/varbit_test.go | 55 - pkg/chunk/fixtures.go | 30 +- pkg/chunk/gcp/bigtable_index_client.go | 413 - pkg/chunk/gcp/bigtable_object_client.go | 182 - pkg/chunk/gcp/fixtures.go | 111 - pkg/chunk/gcp/fnv.go | 36 - pkg/chunk/gcp/gcs_object_client.go | 183 - pkg/chunk/gcp/instrumentation.go | 87 - pkg/chunk/gcp/table_client.go | 126 - pkg/chunk/grpc/grpc.pb.go | 6481 --- pkg/chunk/grpc/grpc.proto | 142 - pkg/chunk/grpc/grpc_client.go | 35 - pkg/chunk/grpc/grpc_client_test.go | 180 - pkg/chunk/grpc/grpc_server_mock_test.go | 185 - pkg/chunk/grpc/index_client.go | 107 - pkg/chunk/grpc/storage_client.go | 118 - pkg/chunk/grpc/table_client.go | 107 - pkg/chunk/index_reader.go | 33 - pkg/chunk/inmemory_storage_client.go | 551 - pkg/chunk/local/boltdb_index_client.go | 366 - pkg/chunk/local/boltdb_index_client_test.go | 261 - pkg/chunk/local/boltdb_table_client.go | 61 - pkg/chunk/local/fixtures.go | 80 - pkg/chunk/local/fs_object_client.go | 211 - pkg/chunk/local/fs_object_client_test.go | 202 - pkg/chunk/objectclient/client.go | 119 - pkg/chunk/openstack/swift_object_client.go | 164 - pkg/chunk/purger/delete_plan.pb.go | 1353 - pkg/chunk/purger/delete_plan.proto | 34 - pkg/chunk/purger/delete_requests_store.go | 394 - pkg/chunk/purger/purger.go | 828 - pkg/chunk/purger/purger_test.go | 532 - pkg/chunk/purger/request_handler.go | 183 - pkg/chunk/purger/table_provisioning.go | 30 - pkg/chunk/purger/tombstones.go | 450 +- pkg/chunk/purger/tombstones_test.go | 230 - pkg/chunk/schema.go | 966 - pkg/chunk/schema_caching.go | 73 - pkg/chunk/schema_caching_test.go | 70 - pkg/chunk/schema_config.go | 448 - pkg/chunk/schema_config_test.go | 714 - pkg/chunk/schema_test.go | 456 - pkg/chunk/schema_util.go | 257 - pkg/chunk/schema_util_test.go | 149 - pkg/chunk/series_store.go | 588 - pkg/chunk/storage/by_key_test.go | 12 - pkg/chunk/storage/bytes.go | 39 - pkg/chunk/storage/caching_fixtures.go | 48 - pkg/chunk/storage/caching_index_client.go | 308 - pkg/chunk/storage/caching_index_client.pb.go | 843 - pkg/chunk/storage/caching_index_client.proto | 25 - .../storage/caching_index_client_test.go | 264 - pkg/chunk/storage/chunk_client_test.go | 61 - pkg/chunk/storage/factory.go | 373 - pkg/chunk/storage/factory_test.go | 197 - pkg/chunk/storage/index_client_test.go | 230 - pkg/chunk/storage/metrics.go | 110 - pkg/chunk/storage/utils_test.go | 39 - pkg/chunk/storage_client.go | 90 - pkg/chunk/strings.go | 88 - pkg/chunk/table_client.go | 62 - pkg/chunk/table_manager.go | 574 - pkg/chunk/table_manager_test.go | 769 - pkg/chunk/table_provisioning.go | 111 - pkg/chunk/tags.go | 58 - pkg/chunk/testutils/testutils.go | 154 - pkg/chunk/util/parallel_chunk_fetch.go | 79 - pkg/chunk/util/parallel_chunk_fetch_test.go | 26 - pkg/chunk/util/util.go | 122 - pkg/cortex/cortex.go | 40 +- pkg/cortex/cortex_test.go | 27 +- pkg/cortex/modules.go | 252 +- pkg/cortex/modules_test.go | 10 - pkg/cortex/runtime_config_test.go | 2 - pkg/cortex/storage/config.go | 27 + pkg/distributor/distributor_test.go | 5 +- pkg/flusher/flusher.go | 21 +- pkg/ingester/active_series_test.go | 17 - pkg/ingester/client/compat.go | 12 - pkg/ingester/client/compat_test.go | 59 - pkg/ingester/client/cortex_mock_test.go | 5 - pkg/ingester/client/cortex_util.go | 8 - pkg/ingester/client/fnv.go | 31 - pkg/ingester/client/ingester.pb.go | 404 +- pkg/ingester/client/ingester.proto | 6 - pkg/ingester/errors.go | 26 - pkg/ingester/flush.go | 417 +- pkg/ingester/flush_test.go | 253 - pkg/ingester/index/index.go | 324 - pkg/ingester/index/index_test.go | 201 - pkg/ingester/ingester.go | 817 +- pkg/ingester/ingester_test.go | 804 +- pkg/ingester/ingester_v2.go | 56 +- pkg/ingester/ingester_v2_test.go | 18 +- pkg/ingester/label_pairs.go | 90 - pkg/ingester/label_pairs_test.go | 106 - pkg/ingester/lifecycle_test.go | 269 +- pkg/ingester/locker.go | 58 - pkg/ingester/locker_test.go | 45 - pkg/ingester/mapper.go | 155 - pkg/ingester/mapper_test.go | 134 - pkg/ingester/series.go | 253 - pkg/ingester/series_map.go | 110 - pkg/ingester/transfer.go | 377 +- pkg/ingester/user_state.go | 338 - pkg/ingester/user_state_test.go | 147 - pkg/ingester/wal.go | 1134 - pkg/ingester/wal.pb.go | 607 - pkg/ingester/wal.proto | 16 - pkg/ingester/wal_test.go | 313 - pkg/querier/batch/batch_test.go | 6 - pkg/querier/batch/chunk_test.go | 8 +- pkg/querier/chunk_store_queryable.go | 88 - pkg/querier/chunk_store_queryable_test.go | 6 +- pkg/querier/chunk_tar_test.go | 139 - pkg/querier/chunks_handler.go | 93 - pkg/querier/chunkstore/chunkstore.go | 16 - pkg/querier/distributor_queryable.go | 8 +- pkg/querier/distributor_queryable_test.go | 10 +- pkg/querier/error_translate_queryable.go | 7 +- pkg/querier/error_translate_queryable_test.go | 7 - .../iterators/chunk_merge_iterator_test.go | 29 +- pkg/querier/lazyquery/lazyquery.go | 37 - pkg/querier/querier.go | 43 +- pkg/querier/querier_benchmark_test.go | 38 - pkg/querier/querier_test.go | 103 +- pkg/querier/queryrange/queryable.go | 154 - pkg/querier/queryrange/queryable_test.go | 270 - pkg/querier/queryrange/querysharding.go | 262 - pkg/querier/queryrange/querysharding_test.go | 665 - pkg/querier/queryrange/roundtrip.go | 37 - pkg/querier/queryrange/roundtrip_test.go | 29 - pkg/querier/series/series_set.go | 4 +- pkg/ring/lifecycler.go | 12 - pkg/ruler/api_test.go | 35 +- pkg/ruler/lifecycle_test.go | 10 +- pkg/ruler/ruler.go | 6 - pkg/ruler/ruler_test.go | 99 +- .../bucketclient/bucket_client_test.go | 10 - .../rulestore/objectclient/rule_store.go | 289 - .../rulestore/objectclient/rule_store_test.go | 4 - pkg/ruler/storage.go | 100 - pkg/storegateway/gateway_test.go | 2 +- pkg/util/chunkcompat/compat.go | 20 +- pkg/util/validation/exporter.go | 1 - pkg/util/validation/limits.go | 35 +- pkg/util/validation/limits_test.go | 38 +- tools/blocksconvert/allowed_users.go | 67 - tools/blocksconvert/builder/builder.go | 482 - tools/blocksconvert/builder/builder_test.go | 62 - tools/blocksconvert/builder/fetcher.go | 52 - tools/blocksconvert/builder/heap.go | 30 - tools/blocksconvert/builder/series.go | 224 - .../blocksconvert/builder/series_iterator.go | 163 - tools/blocksconvert/builder/series_test.go | 119 - .../blocksconvert/builder/symbols_iterator.go | 172 - tools/blocksconvert/builder/tsdb.go | 427 - tools/blocksconvert/builder/tsdb_test.go | 237 - tools/blocksconvert/cleaner/cleaner.go | 357 - tools/blocksconvert/plan_file.go | 112 - tools/blocksconvert/plan_file_test.go | 27 - .../blocksconvert/planprocessor/heartbeat.go | 98 - tools/blocksconvert/planprocessor/service.go | 412 - .../scanner/bigtable_index_reader.go | 242 - .../scanner/bigtable_index_reader_test.go | 182 - .../scanner/cassandra_index_reader.go | 172 - tools/blocksconvert/scanner/files.go | 111 - tools/blocksconvert/scanner/index_entry.go | 76 - tools/blocksconvert/scanner/scanner.go | 649 - .../scanner/scanner_processor.go | 149 - .../scanner/scanner_processor_test.go | 118 - tools/blocksconvert/scanner/scanner_test.go | 115 - tools/blocksconvert/scheduler.pb.go | 772 - tools/blocksconvert/scheduler.proto | 25 - tools/blocksconvert/scheduler/plan_status.go | 60 - tools/blocksconvert/scheduler/scheduler.go | 508 - .../blocksconvert/scheduler/scheduler_test.go | 122 - tools/blocksconvert/shared_config.go | 44 - tools/doc-generator/main.go | 21 +- .../cloud.google.com/go/bigtable/CHANGES.md | 33 - vendor/cloud.google.com/go/bigtable/LICENSE | 202 - vendor/cloud.google.com/go/bigtable/admin.go | 1397 - .../cloud.google.com/go/bigtable/bigtable.go | 925 - .../go/bigtable/bttest/inmem.go | 1427 - .../go/bigtable/bttest/instance_server.go | 133 - .../go/bigtable/bttest/validation.go | 87 - vendor/cloud.google.com/go/bigtable/doc.go | 125 - vendor/cloud.google.com/go/bigtable/filter.go | 352 - vendor/cloud.google.com/go/bigtable/gc.go | 167 - vendor/cloud.google.com/go/bigtable/go.mod | 20 - vendor/cloud.google.com/go/bigtable/go.sum | 313 - .../go/bigtable/go_mod_tidy_hack.go | 22 - .../go/bigtable/internal/option/option.go | 112 - vendor/cloud.google.com/go/bigtable/reader.go | 249 - .../go/longrunning/autogen/doc.go | 156 - .../go/longrunning/autogen/from_conn.go | 30 - .../longrunning/autogen/gapic_metadata.json | 43 - .../go/longrunning/autogen/info.go | 24 - .../longrunning/autogen/operations_client.go | 468 - .../go/longrunning/longrunning.go | 179 - .../github.com/aws/aws-sdk-go/aws/arn/arn.go | 93 - .../aws/aws-sdk-go/aws/crr/cache.go | 122 - .../aws/aws-sdk-go/aws/crr/endpoint.go | 132 - .../aws/aws-sdk-go/aws/crr/sync_map.go | 30 - .../aws/aws-sdk-go/aws/crr/sync_map_1_8.go | 49 - .../internal/s3shared/arn/accesspoint_arn.go | 50 - .../aws-sdk-go/internal/s3shared/arn/arn.go | 94 - .../internal/s3shared/arn/outpost_arn.go | 126 - .../s3shared/arn/s3_object_lambda_arn.go | 15 - .../internal/s3shared/endpoint_errors.go | 202 - .../internal/s3shared/resource_request.go | 45 - .../internal/s3shared/s3err/error.go | 57 - .../private/checksum/content_md5.go | 53 - .../private/protocol/eventstream/debug.go | 144 - .../private/protocol/eventstream/decode.go | 216 - .../private/protocol/eventstream/encode.go | 162 - .../private/protocol/eventstream/error.go | 23 - .../eventstream/eventstreamapi/error.go | 81 - .../eventstream/eventstreamapi/reader.go | 173 - .../eventstream/eventstreamapi/shared.go | 23 - .../eventstream/eventstreamapi/signer.go | 123 - .../eventstreamapi/stream_writer.go | 129 - .../eventstream/eventstreamapi/transport.go | 10 - .../eventstreamapi/transport_go1.17.go | 19 - .../eventstream/eventstreamapi/writer.go | 63 - .../private/protocol/eventstream/header.go | 175 - .../protocol/eventstream/header_value.go | 506 - .../private/protocol/eventstream/message.go | 117 - .../private/protocol/restxml/restxml.go | 79 - .../aws/aws-sdk-go/service/dynamodb/api.go | 25395 ---------- .../service/dynamodb/customizations.go | 98 - .../aws/aws-sdk-go/service/dynamodb/doc.go | 45 - .../aws-sdk-go/service/dynamodb/doc_custom.go | 27 - .../dynamodb/dynamodbiface/interface.go | 288 - .../aws/aws-sdk-go/service/dynamodb/errors.go | 327 - .../aws-sdk-go/service/dynamodb/service.go | 111 - .../aws-sdk-go/service/dynamodb/waiters.go | 107 - .../aws/aws-sdk-go/service/s3/api.go | 42213 ---------------- .../aws/aws-sdk-go/service/s3/body_hash.go | 202 - .../aws-sdk-go/service/s3/bucket_location.go | 107 - .../aws-sdk-go/service/s3/customizations.go | 89 - .../aws/aws-sdk-go/service/s3/doc.go | 26 - .../aws/aws-sdk-go/service/s3/doc_custom.go | 110 - .../aws/aws-sdk-go/service/s3/endpoint.go | 299 - .../aws-sdk-go/service/s3/endpoint_builder.go | 242 - .../aws/aws-sdk-go/service/s3/errors.go | 60 - .../service/s3/host_style_bucket.go | 136 - .../service/s3/platform_handlers.go | 9 - .../service/s3/platform_handlers_go1.6.go | 29 - .../service/s3/s3iface/interface.go | 479 - .../aws/aws-sdk-go/service/s3/service.go | 107 - .../aws/aws-sdk-go/service/s3/sse.go | 84 - .../aws-sdk-go/service/s3/statusok_error.go | 47 - .../aws-sdk-go/service/s3/unmarshal_error.go | 114 - .../aws/aws-sdk-go/service/s3/waiters.go | 214 - .../github.com/fsouza/fake-gcs-server/LICENSE | 22 - .../fake-gcs-server/fakestorage/bucket.go | 54 - .../fsouza/fake-gcs-server/fakestorage/doc.go | 10 - .../fakestorage/mux_tranport.go | 22 - .../fake-gcs-server/fakestorage/object.go | 237 - .../fake-gcs-server/fakestorage/response.go | 120 - .../fake-gcs-server/fakestorage/server.go | 172 - .../fake-gcs-server/fakestorage/upload.go | 268 - .../fake-gcs-server/internal/backend/fs.go | 133 - .../internal/backend/memory.go | 124 - .../internal/backend/object.go | 19 - .../internal/backend/storage.go | 16 - vendor/github.com/gocql/gocql/.gitignore | 5 - vendor/github.com/gocql/gocql/.travis.yml | 49 - vendor/github.com/gocql/gocql/AUTHORS | 117 - vendor/github.com/gocql/gocql/CONTRIBUTING.md | 78 - vendor/github.com/gocql/gocql/LICENSE | 27 - vendor/github.com/gocql/gocql/README.md | 231 - .../gocql/gocql/address_translators.go | 26 - vendor/github.com/gocql/gocql/cluster.go | 233 - vendor/github.com/gocql/gocql/compressor.go | 28 - vendor/github.com/gocql/gocql/conn.go | 1474 - .../github.com/gocql/gocql/connectionpool.go | 661 - vendor/github.com/gocql/gocql/control.go | 497 - vendor/github.com/gocql/gocql/cqltypes.go | 11 - vendor/github.com/gocql/gocql/debug_off.go | 5 - vendor/github.com/gocql/gocql/debug_on.go | 5 - vendor/github.com/gocql/gocql/doc.go | 9 - vendor/github.com/gocql/gocql/errors.go | 125 - vendor/github.com/gocql/gocql/events.go | 285 - vendor/github.com/gocql/gocql/filters.go | 81 - vendor/github.com/gocql/gocql/frame.go | 2106 - vendor/github.com/gocql/gocql/fuzz.go | 33 - vendor/github.com/gocql/gocql/go.mod | 13 - vendor/github.com/gocql/gocql/go.sum | 406 - vendor/github.com/gocql/gocql/helpers.go | 437 - vendor/github.com/gocql/gocql/host_source.go | 716 - .../github.com/gocql/gocql/host_source_gen.go | 45 - .../gocql/gocql/install_test_deps.sh | 16 - vendor/github.com/gocql/gocql/integration.sh | 95 - .../gocql/gocql/internal/lru/lru.go | 127 - .../gocql/gocql/internal/murmur/murmur.go | 135 - .../gocql/internal/murmur/murmur_appengine.go | 11 - .../gocql/internal/murmur/murmur_unsafe.go | 16 - .../gocql/gocql/internal/streams/streams.go | 140 - vendor/github.com/gocql/gocql/marshal.go | 2452 - vendor/github.com/gocql/gocql/metadata.go | 1464 - vendor/github.com/gocql/gocql/policies.go | 967 - .../github.com/gocql/gocql/prepared_cache.go | 89 - .../github.com/gocql/gocql/query_executor.go | 161 - vendor/github.com/gocql/gocql/ring.go | 152 - vendor/github.com/gocql/gocql/session.go | 2075 - vendor/github.com/gocql/gocql/token.go | 222 - vendor/github.com/gocql/gocql/topology.go | 277 - vendor/github.com/gocql/gocql/uuid.go | 315 - .../protobuf/ptypes/wrappers/wrappers.pb.go | 71 - vendor/github.com/google/go-cmp/LICENSE | 27 - .../github.com/google/go-cmp/cmp/compare.go | 667 - .../google/go-cmp/cmp/export_panic.go | 16 - .../google/go-cmp/cmp/export_unsafe.go | 36 - .../go-cmp/cmp/internal/diff/debug_disable.go | 18 - .../go-cmp/cmp/internal/diff/debug_enable.go | 123 - .../google/go-cmp/cmp/internal/diff/diff.go | 398 - .../google/go-cmp/cmp/internal/flags/flags.go | 9 - .../go-cmp/cmp/internal/function/func.go | 99 - .../google/go-cmp/cmp/internal/value/name.go | 164 - .../cmp/internal/value/pointer_purego.go | 34 - .../cmp/internal/value/pointer_unsafe.go | 37 - .../google/go-cmp/cmp/internal/value/sort.go | 106 - .../google/go-cmp/cmp/internal/value/zero.go | 48 - .../github.com/google/go-cmp/cmp/options.go | 552 - vendor/github.com/google/go-cmp/cmp/path.go | 378 - vendor/github.com/google/go-cmp/cmp/report.go | 54 - .../google/go-cmp/cmp/report_compare.go | 435 - .../google/go-cmp/cmp/report_references.go | 264 - .../google/go-cmp/cmp/report_reflect.go | 407 - .../google/go-cmp/cmp/report_slices.go | 613 - .../google/go-cmp/cmp/report_text.go | 431 - .../google/go-cmp/cmp/report_value.go | 121 - .../hailocab/go-hostpool/.gitignore | 22 - .../hailocab/go-hostpool/.travis.yml | 0 .../github.com/hailocab/go-hostpool/LICENSE | 21 - .../github.com/hailocab/go-hostpool/README.md | 17 - .../hailocab/go-hostpool/epsilon_greedy.go | 220 - .../go-hostpool/epsilon_value_calculators.go | 40 - .../hailocab/go-hostpool/host_entry.go | 62 - .../hailocab/go-hostpool/hostpool.go | 243 - .../weaveworks/common/aws/config.go | 65 - .../weaveworks/common/mtime/mtime.go | 16 - .../github.com/weaveworks/common/test/diff.go | 24 - vendor/go.etcd.io/bbolt/.gitignore | 7 - vendor/go.etcd.io/bbolt/.travis.yml | 18 - vendor/go.etcd.io/bbolt/LICENSE | 20 - vendor/go.etcd.io/bbolt/Makefile | 36 - vendor/go.etcd.io/bbolt/README.md | 958 - vendor/go.etcd.io/bbolt/bolt_386.go | 7 - vendor/go.etcd.io/bbolt/bolt_amd64.go | 7 - vendor/go.etcd.io/bbolt/bolt_arm.go | 7 - vendor/go.etcd.io/bbolt/bolt_arm64.go | 9 - vendor/go.etcd.io/bbolt/bolt_linux.go | 10 - vendor/go.etcd.io/bbolt/bolt_mips64x.go | 9 - vendor/go.etcd.io/bbolt/bolt_mipsx.go | 9 - vendor/go.etcd.io/bbolt/bolt_openbsd.go | 27 - vendor/go.etcd.io/bbolt/bolt_ppc.go | 9 - vendor/go.etcd.io/bbolt/bolt_ppc64.go | 9 - vendor/go.etcd.io/bbolt/bolt_ppc64le.go | 9 - vendor/go.etcd.io/bbolt/bolt_riscv64.go | 9 - vendor/go.etcd.io/bbolt/bolt_s390x.go | 9 - vendor/go.etcd.io/bbolt/bolt_unix.go | 86 - vendor/go.etcd.io/bbolt/bolt_unix_aix.go | 90 - vendor/go.etcd.io/bbolt/bolt_unix_solaris.go | 88 - vendor/go.etcd.io/bbolt/bolt_windows.go | 141 - vendor/go.etcd.io/bbolt/boltsync_unix.go | 8 - vendor/go.etcd.io/bbolt/bucket.go | 777 - vendor/go.etcd.io/bbolt/compact.go | 114 - vendor/go.etcd.io/bbolt/cursor.go | 396 - vendor/go.etcd.io/bbolt/db.go | 1232 - vendor/go.etcd.io/bbolt/doc.go | 44 - vendor/go.etcd.io/bbolt/errors.go | 71 - vendor/go.etcd.io/bbolt/freelist.go | 404 - vendor/go.etcd.io/bbolt/freelist_hmap.go | 178 - vendor/go.etcd.io/bbolt/go.mod | 5 - vendor/go.etcd.io/bbolt/go.sum | 2 - vendor/go.etcd.io/bbolt/mlock_unix.go | 36 - vendor/go.etcd.io/bbolt/mlock_windows.go | 11 - vendor/go.etcd.io/bbolt/node.go | 602 - vendor/go.etcd.io/bbolt/page.go | 204 - vendor/go.etcd.io/bbolt/tx.go | 723 - vendor/go.etcd.io/bbolt/unsafe.go | 39 - .../go.opencensus.io/plugin/ocgrpc/client.go | 56 - .../plugin/ocgrpc/client_metrics.go | 109 - .../plugin/ocgrpc/client_stats_handler.go | 49 - vendor/go.opencensus.io/plugin/ocgrpc/doc.go | 19 - .../go.opencensus.io/plugin/ocgrpc/server.go | 81 - .../plugin/ocgrpc/server_metrics.go | 99 - .../plugin/ocgrpc/server_stats_handler.go | 63 - .../plugin/ocgrpc/stats_common.go | 227 - .../plugin/ocgrpc/trace_common.go | 107 - .../golang.org/x/sync/semaphore/semaphore.go | 136 - .../v1/cloudresourcemanager-api.json | 2294 - .../v1/cloudresourcemanager-gen.go | 8253 --- .../api/transport/grpc/dial.go | 319 - .../api/transport/grpc/dial_appengine.go | 32 - .../api/transport/grpc/dial_socketopt.go | 50 - .../api/transport/grpc/pool.go | 92 - .../internal/socket/socket_service.pb.go | 2822 -- .../internal/socket/socket_service.proto | 460 - .../google.golang.org/appengine/socket/doc.go | 10 - .../appengine/socket/socket_classic.go | 290 - .../appengine/socket/socket_vm.go | 64 - .../admin/v2/bigtable_instance_admin.pb.go | 3915 -- .../admin/v2/bigtable_table_admin.pb.go | 4544 -- .../googleapis/bigtable/admin/v2/common.pb.go | 270 - .../bigtable/admin/v2/instance.pb.go | 1541 - .../googleapis/bigtable/admin/v2/table.pb.go | 1878 - .../googleapis/bigtable/v2/bigtable.pb.go | 2368 - .../googleapis/bigtable/v2/data.pb.go | 2594 - .../googleapis/longrunning/operations.pb.go | 1229 - .../protobuf/field_mask/field_mask.go | 23 - .../grpclb/grpc_lb_v1/load_balancer.pb.go | 960 - .../grpc_lb_v1/load_balancer_grpc.pb.go | 137 - .../grpc/balancer/grpclb/grpclb.go | 518 - .../grpc/balancer/grpclb/grpclb_config.go | 67 - .../grpc/balancer/grpclb/grpclb_picker.go | 202 - .../balancer/grpclb/grpclb_remote_balancer.go | 452 - .../grpc/balancer/grpclb/grpclb_util.go | 208 - .../grpc/credentials/alts/alts.go | 332 - .../alts/internal/authinfo/authinfo.go | 95 - .../grpc/credentials/alts/internal/common.go | 67 - .../alts/internal/conn/aeadrekey.go | 131 - .../alts/internal/conn/aes128gcm.go | 105 - .../alts/internal/conn/aes128gcmrekey.go | 116 - .../credentials/alts/internal/conn/common.go | 70 - .../credentials/alts/internal/conn/counter.go | 62 - .../credentials/alts/internal/conn/record.go | 275 - .../credentials/alts/internal/conn/utils.go | 63 - .../alts/internal/handshaker/handshaker.go | 375 - .../internal/handshaker/service/service.go | 60 - .../internal/proto/grpc_gcp/altscontext.pb.go | 264 - .../internal/proto/grpc_gcp/handshaker.pb.go | 1426 - .../proto/grpc_gcp/handshaker_grpc.pb.go | 149 - .../grpc_gcp/transport_security_common.pb.go | 326 - .../grpc/credentials/alts/utils.go | 70 - .../grpc/credentials/google/google.go | 145 - .../grpc/credentials/google/xds.go | 91 - .../grpc/credentials/oauth/oauth.go | 242 - .../grpc/internal/googlecloud/googlecloud.go | 128 - .../types/known/wrapperspb/wrappers.pb.go | 760 - vendor/gopkg.in/inf.v0/LICENSE | 28 - vendor/gopkg.in/inf.v0/dec.go | 615 - vendor/gopkg.in/inf.v0/rounder.go | 145 - vendor/modules.txt | 79 - vendor/rsc.io/binaryregexp/LICENSE | 27 - vendor/rsc.io/binaryregexp/backtrack.go | 367 - vendor/rsc.io/binaryregexp/exec.go | 555 - vendor/rsc.io/binaryregexp/go.mod | 3 - vendor/rsc.io/binaryregexp/onepass.go | 507 - vendor/rsc.io/binaryregexp/regexp.go | 1217 - vendor/rsc.io/binaryregexp/syntax/compile.go | 289 - vendor/rsc.io/binaryregexp/syntax/doc.go | 131 - .../rsc.io/binaryregexp/syntax/op_string.go | 26 - vendor/rsc.io/binaryregexp/syntax/parse.go | 1918 - .../rsc.io/binaryregexp/syntax/perl_groups.go | 134 - vendor/rsc.io/binaryregexp/syntax/prog.go | 346 - vendor/rsc.io/binaryregexp/syntax/regexp.go | 321 - vendor/rsc.io/binaryregexp/syntax/simplify.go | 151 - 568 files changed, 1020 insertions(+), 214693 deletions(-) delete mode 100644 cmd/blocksconvert/Dockerfile delete mode 100644 cmd/blocksconvert/main.go delete mode 100644 docs/chunks-storage/_index.md delete mode 100644 docs/chunks-storage/aws-tips.md delete mode 100644 docs/chunks-storage/caching.md delete mode 100644 docs/chunks-storage/chunks-storage-getting-started.md delete mode 100644 docs/chunks-storage/ingesters-with-wal.md delete mode 100644 docs/chunks-storage/running-chunks-storage-in-production.md delete mode 100644 docs/chunks-storage/running-chunks-storage-with-cassandra.md delete mode 100644 docs/chunks-storage/schema-config.md delete mode 100644 docs/chunks-storage/table-manager.md rename docs/{chunks-storage/single-process-config.yaml => configuration/single-process-config-blocks-local.yaml} (54%) delete mode 100644 docs/guides/deleting-series.md delete mode 100644 docs/guides/grpc-storage-plugin.md delete mode 100644 integration/certs.go delete mode 100644 integration/querier_streaming_mixed_ingester_test.go delete mode 100644 integration/s3_storage_client_test.go delete mode 100644 pkg/alertmanager/alertstore/objectclient/store.go delete mode 100644 pkg/alertmanager/alertstore/objectclient/store_test.go delete mode 100644 pkg/chunk/aws/dynamodb_index_reader.go delete mode 100644 pkg/chunk/aws/dynamodb_metrics.go delete mode 100644 pkg/chunk/aws/dynamodb_storage_client.go delete mode 100644 pkg/chunk/aws/dynamodb_storage_client_test.go delete mode 100644 pkg/chunk/aws/dynamodb_table_client.go delete mode 100644 pkg/chunk/aws/fixtures.go delete mode 100644 pkg/chunk/aws/metrics_autoscaling.go delete mode 100644 pkg/chunk/aws/metrics_autoscaling_test.go delete mode 100644 pkg/chunk/aws/mock.go delete mode 100644 pkg/chunk/aws/retryer.go delete mode 100644 pkg/chunk/aws/s3_storage_client.go delete mode 100644 pkg/chunk/aws/s3_storage_client_test.go delete mode 100644 pkg/chunk/aws/sse_config.go delete mode 100644 pkg/chunk/aws/sse_config_test.go delete mode 100644 pkg/chunk/azure/blob_storage_client.go delete mode 100644 pkg/chunk/bucket_client.go delete mode 100644 pkg/chunk/cassandra/authenticator.go delete mode 100644 pkg/chunk/cassandra/fixtures.go delete mode 100644 pkg/chunk/cassandra/instrumentation.go delete mode 100644 pkg/chunk/cassandra/storage_client.go delete mode 100644 pkg/chunk/cassandra/storage_client_test.go delete mode 100644 pkg/chunk/cassandra/table_client.go delete mode 100644 pkg/chunk/cassandra/table_client_test.go delete mode 100644 pkg/chunk/cassandra/testdata/example.com-key.pem delete mode 100644 pkg/chunk/cassandra/testdata/example.com.ca.pem delete mode 100644 pkg/chunk/cassandra/testdata/example.com.pem delete mode 100644 pkg/chunk/cassandra/testdata/password-with-trailing-newline.txt delete mode 100644 pkg/chunk/cassandra/testdata/password-without-trailing-newline.txt delete mode 100644 pkg/chunk/chunk_store.go delete mode 100644 pkg/chunk/chunk_store_test.go delete mode 100644 pkg/chunk/chunk_store_utils.go delete mode 100644 pkg/chunk/composite_store.go delete mode 100644 pkg/chunk/composite_store_test.go delete mode 100644 pkg/chunk/encoding/bigchunk.go delete mode 100644 pkg/chunk/encoding/bigchunk_test.go delete mode 100644 pkg/chunk/encoding/delta_helpers.go delete mode 100644 pkg/chunk/encoding/doubledelta.go delete mode 100644 pkg/chunk/encoding/varbit.go delete mode 100644 pkg/chunk/encoding/varbit_helpers.go delete mode 100644 pkg/chunk/encoding/varbit_test.go delete mode 100644 pkg/chunk/gcp/bigtable_index_client.go delete mode 100644 pkg/chunk/gcp/bigtable_object_client.go delete mode 100644 pkg/chunk/gcp/fixtures.go delete mode 100644 pkg/chunk/gcp/fnv.go delete mode 100644 pkg/chunk/gcp/gcs_object_client.go delete mode 100644 pkg/chunk/gcp/instrumentation.go delete mode 100644 pkg/chunk/gcp/table_client.go delete mode 100644 pkg/chunk/grpc/grpc.pb.go delete mode 100644 pkg/chunk/grpc/grpc.proto delete mode 100644 pkg/chunk/grpc/grpc_client.go delete mode 100644 pkg/chunk/grpc/grpc_client_test.go delete mode 100644 pkg/chunk/grpc/grpc_server_mock_test.go delete mode 100644 pkg/chunk/grpc/index_client.go delete mode 100644 pkg/chunk/grpc/storage_client.go delete mode 100644 pkg/chunk/grpc/table_client.go delete mode 100644 pkg/chunk/index_reader.go delete mode 100644 pkg/chunk/inmemory_storage_client.go delete mode 100644 pkg/chunk/local/boltdb_index_client.go delete mode 100644 pkg/chunk/local/boltdb_index_client_test.go delete mode 100644 pkg/chunk/local/boltdb_table_client.go delete mode 100644 pkg/chunk/local/fixtures.go delete mode 100644 pkg/chunk/local/fs_object_client.go delete mode 100644 pkg/chunk/local/fs_object_client_test.go delete mode 100644 pkg/chunk/objectclient/client.go delete mode 100644 pkg/chunk/openstack/swift_object_client.go delete mode 100644 pkg/chunk/purger/delete_plan.pb.go delete mode 100644 pkg/chunk/purger/delete_plan.proto delete mode 100644 pkg/chunk/purger/delete_requests_store.go delete mode 100644 pkg/chunk/purger/purger.go delete mode 100644 pkg/chunk/purger/purger_test.go delete mode 100644 pkg/chunk/purger/request_handler.go delete mode 100644 pkg/chunk/purger/table_provisioning.go delete mode 100644 pkg/chunk/purger/tombstones_test.go delete mode 100644 pkg/chunk/schema.go delete mode 100644 pkg/chunk/schema_caching.go delete mode 100644 pkg/chunk/schema_caching_test.go delete mode 100644 pkg/chunk/schema_config.go delete mode 100644 pkg/chunk/schema_config_test.go delete mode 100644 pkg/chunk/schema_test.go delete mode 100644 pkg/chunk/schema_util.go delete mode 100644 pkg/chunk/schema_util_test.go delete mode 100644 pkg/chunk/series_store.go delete mode 100644 pkg/chunk/storage/by_key_test.go delete mode 100644 pkg/chunk/storage/bytes.go delete mode 100644 pkg/chunk/storage/caching_fixtures.go delete mode 100644 pkg/chunk/storage/caching_index_client.go delete mode 100644 pkg/chunk/storage/caching_index_client.pb.go delete mode 100644 pkg/chunk/storage/caching_index_client.proto delete mode 100644 pkg/chunk/storage/caching_index_client_test.go delete mode 100644 pkg/chunk/storage/chunk_client_test.go delete mode 100644 pkg/chunk/storage/factory.go delete mode 100644 pkg/chunk/storage/factory_test.go delete mode 100644 pkg/chunk/storage/index_client_test.go delete mode 100644 pkg/chunk/storage/metrics.go delete mode 100644 pkg/chunk/storage/utils_test.go delete mode 100644 pkg/chunk/storage_client.go delete mode 100644 pkg/chunk/strings.go delete mode 100644 pkg/chunk/table_client.go delete mode 100644 pkg/chunk/table_manager.go delete mode 100644 pkg/chunk/table_manager_test.go delete mode 100644 pkg/chunk/table_provisioning.go delete mode 100644 pkg/chunk/tags.go delete mode 100644 pkg/chunk/testutils/testutils.go delete mode 100644 pkg/chunk/util/parallel_chunk_fetch.go delete mode 100644 pkg/chunk/util/parallel_chunk_fetch_test.go create mode 100644 pkg/cortex/storage/config.go delete mode 100644 pkg/ingester/flush_test.go delete mode 100644 pkg/ingester/index/index.go delete mode 100644 pkg/ingester/index/index_test.go delete mode 100644 pkg/ingester/label_pairs.go delete mode 100644 pkg/ingester/label_pairs_test.go delete mode 100644 pkg/ingester/locker.go delete mode 100644 pkg/ingester/locker_test.go delete mode 100644 pkg/ingester/mapper.go delete mode 100644 pkg/ingester/mapper_test.go delete mode 100644 pkg/ingester/series_map.go delete mode 100644 pkg/ingester/wal.go delete mode 100644 pkg/ingester/wal.pb.go delete mode 100644 pkg/ingester/wal.proto delete mode 100644 pkg/ingester/wal_test.go delete mode 100644 pkg/querier/chunk_tar_test.go delete mode 100644 pkg/querier/chunks_handler.go delete mode 100644 pkg/querier/chunkstore/chunkstore.go delete mode 100644 pkg/querier/querier_benchmark_test.go delete mode 100644 pkg/querier/queryrange/queryable.go delete mode 100644 pkg/querier/queryrange/queryable_test.go delete mode 100644 pkg/querier/queryrange/querysharding.go delete mode 100644 pkg/querier/queryrange/querysharding_test.go delete mode 100644 pkg/ruler/rulestore/objectclient/rule_store.go delete mode 100644 pkg/ruler/rulestore/objectclient/rule_store_test.go delete mode 100644 tools/blocksconvert/allowed_users.go delete mode 100644 tools/blocksconvert/builder/builder.go delete mode 100644 tools/blocksconvert/builder/builder_test.go delete mode 100644 tools/blocksconvert/builder/fetcher.go delete mode 100644 tools/blocksconvert/builder/heap.go delete mode 100644 tools/blocksconvert/builder/series.go delete mode 100644 tools/blocksconvert/builder/series_iterator.go delete mode 100644 tools/blocksconvert/builder/series_test.go delete mode 100644 tools/blocksconvert/builder/symbols_iterator.go delete mode 100644 tools/blocksconvert/builder/tsdb.go delete mode 100644 tools/blocksconvert/builder/tsdb_test.go delete mode 100644 tools/blocksconvert/cleaner/cleaner.go delete mode 100644 tools/blocksconvert/plan_file.go delete mode 100644 tools/blocksconvert/plan_file_test.go delete mode 100644 tools/blocksconvert/planprocessor/heartbeat.go delete mode 100644 tools/blocksconvert/planprocessor/service.go delete mode 100644 tools/blocksconvert/scanner/bigtable_index_reader.go delete mode 100644 tools/blocksconvert/scanner/bigtable_index_reader_test.go delete mode 100644 tools/blocksconvert/scanner/cassandra_index_reader.go delete mode 100644 tools/blocksconvert/scanner/files.go delete mode 100644 tools/blocksconvert/scanner/index_entry.go delete mode 100644 tools/blocksconvert/scanner/scanner.go delete mode 100644 tools/blocksconvert/scanner/scanner_processor.go delete mode 100644 tools/blocksconvert/scanner/scanner_processor_test.go delete mode 100644 tools/blocksconvert/scanner/scanner_test.go delete mode 100644 tools/blocksconvert/scheduler.pb.go delete mode 100644 tools/blocksconvert/scheduler.proto delete mode 100644 tools/blocksconvert/scheduler/plan_status.go delete mode 100644 tools/blocksconvert/scheduler/scheduler.go delete mode 100644 tools/blocksconvert/scheduler/scheduler_test.go delete mode 100644 tools/blocksconvert/shared_config.go delete mode 100644 vendor/cloud.google.com/go/bigtable/CHANGES.md delete mode 100644 vendor/cloud.google.com/go/bigtable/LICENSE delete mode 100644 vendor/cloud.google.com/go/bigtable/admin.go delete mode 100644 vendor/cloud.google.com/go/bigtable/bigtable.go delete mode 100644 vendor/cloud.google.com/go/bigtable/bttest/inmem.go delete mode 100644 vendor/cloud.google.com/go/bigtable/bttest/instance_server.go delete mode 100644 vendor/cloud.google.com/go/bigtable/bttest/validation.go delete mode 100644 vendor/cloud.google.com/go/bigtable/doc.go delete mode 100644 vendor/cloud.google.com/go/bigtable/filter.go delete mode 100644 vendor/cloud.google.com/go/bigtable/gc.go delete mode 100644 vendor/cloud.google.com/go/bigtable/go.mod delete mode 100644 vendor/cloud.google.com/go/bigtable/go.sum delete mode 100644 vendor/cloud.google.com/go/bigtable/go_mod_tidy_hack.go delete mode 100644 vendor/cloud.google.com/go/bigtable/internal/option/option.go delete mode 100644 vendor/cloud.google.com/go/bigtable/reader.go delete mode 100644 vendor/cloud.google.com/go/longrunning/autogen/doc.go delete mode 100644 vendor/cloud.google.com/go/longrunning/autogen/from_conn.go delete mode 100644 vendor/cloud.google.com/go/longrunning/autogen/gapic_metadata.json delete mode 100644 vendor/cloud.google.com/go/longrunning/autogen/info.go delete mode 100644 vendor/cloud.google.com/go/longrunning/autogen/operations_client.go delete mode 100644 vendor/cloud.google.com/go/longrunning/longrunning.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/aws/arn/arn.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/aws/crr/cache.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/aws/crr/endpoint.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/aws/crr/sync_map.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/aws/crr/sync_map_1_8.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/internal/s3shared/arn/accesspoint_arn.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/internal/s3shared/arn/arn.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/internal/s3shared/arn/outpost_arn.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/internal/s3shared/arn/s3_object_lambda_arn.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/internal/s3shared/endpoint_errors.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/internal/s3shared/resource_request.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/internal/s3shared/s3err/error.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/checksum/content_md5.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/debug.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/decode.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/encode.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/error.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/eventstreamapi/error.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/eventstreamapi/reader.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/eventstreamapi/shared.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/eventstreamapi/signer.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/eventstreamapi/stream_writer.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/eventstreamapi/transport.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/eventstreamapi/transport_go1.17.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/eventstreamapi/writer.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/header.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/header_value.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/eventstream/message.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/private/protocol/restxml/restxml.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/dynamodb/api.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/dynamodb/customizations.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/dynamodb/doc.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/dynamodb/doc_custom.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/dynamodb/dynamodbiface/interface.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/dynamodb/errors.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/dynamodb/service.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/dynamodb/waiters.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/api.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/body_hash.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/bucket_location.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/customizations.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/doc.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/doc_custom.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/endpoint.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/endpoint_builder.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/errors.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/host_style_bucket.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/platform_handlers.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/platform_handlers_go1.6.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/s3iface/interface.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/service.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/sse.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/statusok_error.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/unmarshal_error.go delete mode 100644 vendor/github.com/aws/aws-sdk-go/service/s3/waiters.go delete mode 100644 vendor/github.com/fsouza/fake-gcs-server/LICENSE delete mode 100644 vendor/github.com/fsouza/fake-gcs-server/fakestorage/bucket.go delete mode 100644 vendor/github.com/fsouza/fake-gcs-server/fakestorage/doc.go delete mode 100644 vendor/github.com/fsouza/fake-gcs-server/fakestorage/mux_tranport.go delete mode 100644 vendor/github.com/fsouza/fake-gcs-server/fakestorage/object.go delete mode 100644 vendor/github.com/fsouza/fake-gcs-server/fakestorage/response.go delete mode 100644 vendor/github.com/fsouza/fake-gcs-server/fakestorage/server.go delete mode 100644 vendor/github.com/fsouza/fake-gcs-server/fakestorage/upload.go delete mode 100644 vendor/github.com/fsouza/fake-gcs-server/internal/backend/fs.go delete mode 100644 vendor/github.com/fsouza/fake-gcs-server/internal/backend/memory.go delete mode 100644 vendor/github.com/fsouza/fake-gcs-server/internal/backend/object.go delete mode 100644 vendor/github.com/fsouza/fake-gcs-server/internal/backend/storage.go delete mode 100644 vendor/github.com/gocql/gocql/.gitignore delete mode 100644 vendor/github.com/gocql/gocql/.travis.yml delete mode 100644 vendor/github.com/gocql/gocql/AUTHORS delete mode 100644 vendor/github.com/gocql/gocql/CONTRIBUTING.md delete mode 100644 vendor/github.com/gocql/gocql/LICENSE delete mode 100644 vendor/github.com/gocql/gocql/README.md delete mode 100644 vendor/github.com/gocql/gocql/address_translators.go delete mode 100644 vendor/github.com/gocql/gocql/cluster.go delete mode 100644 vendor/github.com/gocql/gocql/compressor.go delete mode 100644 vendor/github.com/gocql/gocql/conn.go delete mode 100644 vendor/github.com/gocql/gocql/connectionpool.go delete mode 100644 vendor/github.com/gocql/gocql/control.go delete mode 100644 vendor/github.com/gocql/gocql/cqltypes.go delete mode 100644 vendor/github.com/gocql/gocql/debug_off.go delete mode 100644 vendor/github.com/gocql/gocql/debug_on.go delete mode 100644 vendor/github.com/gocql/gocql/doc.go delete mode 100644 vendor/github.com/gocql/gocql/errors.go delete mode 100644 vendor/github.com/gocql/gocql/events.go delete mode 100644 vendor/github.com/gocql/gocql/filters.go delete mode 100644 vendor/github.com/gocql/gocql/frame.go delete mode 100644 vendor/github.com/gocql/gocql/fuzz.go delete mode 100644 vendor/github.com/gocql/gocql/go.mod delete mode 100644 vendor/github.com/gocql/gocql/go.sum delete mode 100644 vendor/github.com/gocql/gocql/helpers.go delete mode 100644 vendor/github.com/gocql/gocql/host_source.go delete mode 100644 vendor/github.com/gocql/gocql/host_source_gen.go delete mode 100644 vendor/github.com/gocql/gocql/install_test_deps.sh delete mode 100644 vendor/github.com/gocql/gocql/integration.sh delete mode 100644 vendor/github.com/gocql/gocql/internal/lru/lru.go delete mode 100644 vendor/github.com/gocql/gocql/internal/murmur/murmur.go delete mode 100644 vendor/github.com/gocql/gocql/internal/murmur/murmur_appengine.go delete mode 100644 vendor/github.com/gocql/gocql/internal/murmur/murmur_unsafe.go delete mode 100644 vendor/github.com/gocql/gocql/internal/streams/streams.go delete mode 100644 vendor/github.com/gocql/gocql/marshal.go delete mode 100644 vendor/github.com/gocql/gocql/metadata.go delete mode 100644 vendor/github.com/gocql/gocql/policies.go delete mode 100644 vendor/github.com/gocql/gocql/prepared_cache.go delete mode 100644 vendor/github.com/gocql/gocql/query_executor.go delete mode 100644 vendor/github.com/gocql/gocql/ring.go delete mode 100644 vendor/github.com/gocql/gocql/session.go delete mode 100644 vendor/github.com/gocql/gocql/token.go delete mode 100644 vendor/github.com/gocql/gocql/topology.go delete mode 100644 vendor/github.com/gocql/gocql/uuid.go delete mode 100644 vendor/github.com/golang/protobuf/ptypes/wrappers/wrappers.pb.go delete mode 100644 vendor/github.com/google/go-cmp/LICENSE delete mode 100644 vendor/github.com/google/go-cmp/cmp/compare.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/export_panic.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/export_unsafe.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/internal/diff/debug_disable.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/internal/diff/debug_enable.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/internal/diff/diff.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/internal/flags/flags.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/internal/function/func.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/internal/value/name.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/internal/value/pointer_purego.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/internal/value/pointer_unsafe.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/internal/value/sort.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/internal/value/zero.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/options.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/path.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/report.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/report_compare.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/report_references.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/report_reflect.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/report_slices.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/report_text.go delete mode 100644 vendor/github.com/google/go-cmp/cmp/report_value.go delete mode 100644 vendor/github.com/hailocab/go-hostpool/.gitignore delete mode 100644 vendor/github.com/hailocab/go-hostpool/.travis.yml delete mode 100644 vendor/github.com/hailocab/go-hostpool/LICENSE delete mode 100644 vendor/github.com/hailocab/go-hostpool/README.md delete mode 100644 vendor/github.com/hailocab/go-hostpool/epsilon_greedy.go delete mode 100644 vendor/github.com/hailocab/go-hostpool/epsilon_value_calculators.go delete mode 100644 vendor/github.com/hailocab/go-hostpool/host_entry.go delete mode 100644 vendor/github.com/hailocab/go-hostpool/hostpool.go delete mode 100644 vendor/github.com/weaveworks/common/aws/config.go delete mode 100644 vendor/github.com/weaveworks/common/mtime/mtime.go delete mode 100644 vendor/github.com/weaveworks/common/test/diff.go delete mode 100644 vendor/go.etcd.io/bbolt/.gitignore delete mode 100644 vendor/go.etcd.io/bbolt/.travis.yml delete mode 100644 vendor/go.etcd.io/bbolt/LICENSE delete mode 100644 vendor/go.etcd.io/bbolt/Makefile delete mode 100644 vendor/go.etcd.io/bbolt/README.md delete mode 100644 vendor/go.etcd.io/bbolt/bolt_386.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_amd64.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_arm.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_arm64.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_linux.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_mips64x.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_mipsx.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_openbsd.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_ppc.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_ppc64.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_ppc64le.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_riscv64.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_s390x.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_unix.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_unix_aix.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_unix_solaris.go delete mode 100644 vendor/go.etcd.io/bbolt/bolt_windows.go delete mode 100644 vendor/go.etcd.io/bbolt/boltsync_unix.go delete mode 100644 vendor/go.etcd.io/bbolt/bucket.go delete mode 100644 vendor/go.etcd.io/bbolt/compact.go delete mode 100644 vendor/go.etcd.io/bbolt/cursor.go delete mode 100644 vendor/go.etcd.io/bbolt/db.go delete mode 100644 vendor/go.etcd.io/bbolt/doc.go delete mode 100644 vendor/go.etcd.io/bbolt/errors.go delete mode 100644 vendor/go.etcd.io/bbolt/freelist.go delete mode 100644 vendor/go.etcd.io/bbolt/freelist_hmap.go delete mode 100644 vendor/go.etcd.io/bbolt/go.mod delete mode 100644 vendor/go.etcd.io/bbolt/go.sum delete mode 100644 vendor/go.etcd.io/bbolt/mlock_unix.go delete mode 100644 vendor/go.etcd.io/bbolt/mlock_windows.go delete mode 100644 vendor/go.etcd.io/bbolt/node.go delete mode 100644 vendor/go.etcd.io/bbolt/page.go delete mode 100644 vendor/go.etcd.io/bbolt/tx.go delete mode 100644 vendor/go.etcd.io/bbolt/unsafe.go delete mode 100644 vendor/go.opencensus.io/plugin/ocgrpc/client.go delete mode 100644 vendor/go.opencensus.io/plugin/ocgrpc/client_metrics.go delete mode 100644 vendor/go.opencensus.io/plugin/ocgrpc/client_stats_handler.go delete mode 100644 vendor/go.opencensus.io/plugin/ocgrpc/doc.go delete mode 100644 vendor/go.opencensus.io/plugin/ocgrpc/server.go delete mode 100644 vendor/go.opencensus.io/plugin/ocgrpc/server_metrics.go delete mode 100644 vendor/go.opencensus.io/plugin/ocgrpc/server_stats_handler.go delete mode 100644 vendor/go.opencensus.io/plugin/ocgrpc/stats_common.go delete mode 100644 vendor/go.opencensus.io/plugin/ocgrpc/trace_common.go delete mode 100644 vendor/golang.org/x/sync/semaphore/semaphore.go delete mode 100644 vendor/google.golang.org/api/cloudresourcemanager/v1/cloudresourcemanager-api.json delete mode 100644 vendor/google.golang.org/api/cloudresourcemanager/v1/cloudresourcemanager-gen.go delete mode 100644 vendor/google.golang.org/api/transport/grpc/dial.go delete mode 100644 vendor/google.golang.org/api/transport/grpc/dial_appengine.go delete mode 100644 vendor/google.golang.org/api/transport/grpc/dial_socketopt.go delete mode 100644 vendor/google.golang.org/api/transport/grpc/pool.go delete mode 100644 vendor/google.golang.org/appengine/internal/socket/socket_service.pb.go delete mode 100644 vendor/google.golang.org/appengine/internal/socket/socket_service.proto delete mode 100644 vendor/google.golang.org/appengine/socket/doc.go delete mode 100644 vendor/google.golang.org/appengine/socket/socket_classic.go delete mode 100644 vendor/google.golang.org/appengine/socket/socket_vm.go delete mode 100644 vendor/google.golang.org/genproto/googleapis/bigtable/admin/v2/bigtable_instance_admin.pb.go delete mode 100644 vendor/google.golang.org/genproto/googleapis/bigtable/admin/v2/bigtable_table_admin.pb.go delete mode 100644 vendor/google.golang.org/genproto/googleapis/bigtable/admin/v2/common.pb.go delete mode 100644 vendor/google.golang.org/genproto/googleapis/bigtable/admin/v2/instance.pb.go delete mode 100644 vendor/google.golang.org/genproto/googleapis/bigtable/admin/v2/table.pb.go delete mode 100644 vendor/google.golang.org/genproto/googleapis/bigtable/v2/bigtable.pb.go delete mode 100644 vendor/google.golang.org/genproto/googleapis/bigtable/v2/data.pb.go delete mode 100644 vendor/google.golang.org/genproto/googleapis/longrunning/operations.pb.go delete mode 100644 vendor/google.golang.org/genproto/protobuf/field_mask/field_mask.go delete mode 100644 vendor/google.golang.org/grpc/balancer/grpclb/grpc_lb_v1/load_balancer.pb.go delete mode 100644 vendor/google.golang.org/grpc/balancer/grpclb/grpc_lb_v1/load_balancer_grpc.pb.go delete mode 100644 vendor/google.golang.org/grpc/balancer/grpclb/grpclb.go delete mode 100644 vendor/google.golang.org/grpc/balancer/grpclb/grpclb_config.go delete mode 100644 vendor/google.golang.org/grpc/balancer/grpclb/grpclb_picker.go delete mode 100644 vendor/google.golang.org/grpc/balancer/grpclb/grpclb_remote_balancer.go delete mode 100644 vendor/google.golang.org/grpc/balancer/grpclb/grpclb_util.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/alts.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/internal/authinfo/authinfo.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/internal/common.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/internal/conn/aeadrekey.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/internal/conn/aes128gcm.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/internal/conn/aes128gcmrekey.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/internal/conn/common.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/internal/conn/counter.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/internal/conn/record.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/internal/conn/utils.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/internal/handshaker/handshaker.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/internal/handshaker/service/service.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/internal/proto/grpc_gcp/altscontext.pb.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/internal/proto/grpc_gcp/handshaker.pb.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/internal/proto/grpc_gcp/handshaker_grpc.pb.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/internal/proto/grpc_gcp/transport_security_common.pb.go delete mode 100644 vendor/google.golang.org/grpc/credentials/alts/utils.go delete mode 100644 vendor/google.golang.org/grpc/credentials/google/google.go delete mode 100644 vendor/google.golang.org/grpc/credentials/google/xds.go delete mode 100644 vendor/google.golang.org/grpc/credentials/oauth/oauth.go delete mode 100644 vendor/google.golang.org/grpc/internal/googlecloud/googlecloud.go delete mode 100644 vendor/google.golang.org/protobuf/types/known/wrapperspb/wrappers.pb.go delete mode 100644 vendor/gopkg.in/inf.v0/LICENSE delete mode 100644 vendor/gopkg.in/inf.v0/dec.go delete mode 100644 vendor/gopkg.in/inf.v0/rounder.go delete mode 100644 vendor/rsc.io/binaryregexp/LICENSE delete mode 100644 vendor/rsc.io/binaryregexp/backtrack.go delete mode 100644 vendor/rsc.io/binaryregexp/exec.go delete mode 100644 vendor/rsc.io/binaryregexp/go.mod delete mode 100644 vendor/rsc.io/binaryregexp/onepass.go delete mode 100644 vendor/rsc.io/binaryregexp/regexp.go delete mode 100644 vendor/rsc.io/binaryregexp/syntax/compile.go delete mode 100644 vendor/rsc.io/binaryregexp/syntax/doc.go delete mode 100644 vendor/rsc.io/binaryregexp/syntax/op_string.go delete mode 100644 vendor/rsc.io/binaryregexp/syntax/parse.go delete mode 100644 vendor/rsc.io/binaryregexp/syntax/perl_groups.go delete mode 100644 vendor/rsc.io/binaryregexp/syntax/prog.go delete mode 100644 vendor/rsc.io/binaryregexp/syntax/regexp.go delete mode 100644 vendor/rsc.io/binaryregexp/syntax/simplify.go diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 0e1b47780e..234056c931 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -22,9 +22,5 @@ A clear and concise description of what you expected to happen. - Infrastructure: [e.g., Kubernetes, bare-metal, laptop] - Deployment tool: [e.g., helm, jsonnet] -**Storage Engine** -- [ ] Blocks -- [ ] Chunks - **Additional Context** diff --git a/.github/workflows/test-build-deploy.yml b/.github/workflows/test-build-deploy.yml index ff684b916d..302551e73c 100644 --- a/.github/workflows/test-build-deploy.yml +++ b/.github/workflows/test-build-deploy.yml @@ -35,13 +35,6 @@ jobs: runs-on: ubuntu-20.04 container: image: quay.io/cortexproject/build-image:update-hugo-0.94.x-96d1d7cdd - services: - cassandra: - image: cassandra:3.11 - env: - JVM_OPTS: "-Xms1024M -Xmx1024M" - ports: - - 9042:9042 steps: - name: Checkout Repo uses: actions/checkout@v2 @@ -50,7 +43,7 @@ jobs: mkdir -p /go/src/github.com/cortexproject/cortex ln -s $GITHUB_WORKSPACE/* /go/src/github.com/cortexproject/cortex - name: Run Tests - run: CASSANDRA_TEST_ADDRESSES=cassandra:9042 make BUILD_IN_CONTAINER=false test + run: make BUILD_IN_CONTAINER=false test build: runs-on: ubuntu-20.04 @@ -138,7 +131,6 @@ jobs: docker pull quay.io/cortexproject/cortex:v1.9.0 docker pull quay.io/cortexproject/cortex:v1.10.0 docker pull shopify/bigtable-emulator:0.1.0 - docker pull rinscy/cassandra:3.11.0 docker pull memcached:1.6.1 docker pull bouncestorage/swift-aio:55ba4331 - name: Integration Tests diff --git a/.gitignore b/.gitignore index ccb1f57ebe..c610d30757 100644 --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,6 @@ cmd/test-exporter/test-exporter cmd/cortex/cortex cmd/query-tee/query-tee -cmd/blocksconvert/blocksconvert cmd/thanosconvert/thanosconvert .uptodate .pkg diff --git a/CHANGELOG.md b/CHANGELOG.md index 4dae8d9b36..00b0d0e727 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,42 @@ # Changelog ## master / unreleased + **This release removes support for chunks storage. See below for more.** +* [CHANGE] Remove support for chunks storage entirely. If you are using chunks storage on a previous version, you must [migrate your data](https://github.com/cortexproject/cortex/blob/v1.11.1/docs/blocks-storage/migrate-from-chunks-to-blocks.md) on version 1.12 or earlier. Before upgrading to this release, you should also remove any deprecated chunks-related configuration, as this release will no longer accept that. The following flags are gone: + - `-dynamodb.*` + - `-metrics.*` + - `-s3.*` + - `-azure.*` + - `-bigtable.*` + - `-gcs.*` + - `-cassandra.*` + - `-boltdb.*` + - `-local.*` + - some `-ingester` flags: + - `-ingester.wal-enabled` + - `-ingester.checkpoint-enabled` + - `-ingester.recover-from-wal` + - `-ingester.wal-dir` + - `-ingester.checkpoint-duration` + - `-ingester.flush-on-shutdown-with-wal-enabled` + - `-ingester.max-transfer-retries` + - `-ingester.max-samples-per-query` + - `-ingester.min-chunk-length` + - `-ingester.flush-period` + - `-ingester.retain-period` + - `-ingester.max-chunk-idle` + - `-ingester.max-stale-chunk-idle` + - `-ingester.flush-op-timeout` + - `-ingester.max-chunk-age` + - `-ingester.chunk-age-jitter` + - `-ingester.concurrent-flushes` + - `-ingester.spread-flushes` + - `-store.*` except `-store.engine` and `-store.max-query-length` + - `-store.query-chunk-limit` was deprecated and replaced by `-querier.max-fetched-chunks-per-query` + - `-deletes.*` + - `-grpc-store.*` + - `-flusher.wal-dir`, `-flusher.concurrent-flushes`, `-flusher.flush-op-timeout` +* [CHANGE] Remove support for alertmanager and ruler legacy store configuration. Before upgrading, you need to convert your configuration to use the `alertmanager-storage` and `ruler-storage` configuration on the version that you're already running, then upgrade. * [ENHANCEMENT] Querier/Ruler: Retry store-gateway in case of unexpected failure, instead of failing the query. #4532 * [ENHANCEMENT] Ring: DoBatch prioritize 4xx errors when failing. #4783 * [FEATURE] Compactor: Added `-compactor.block-files-concurrency` allowing to configure number of go routines for download/upload block files during compaction. #4784 @@ -8,8 +44,6 @@ * [FEATURE] Compactor: Added configurations for Azure MSI in blocks-storage, ruler-storage and alertmanager-storage. #4818 * [BUGFIX] Memberlist: Add join with no retrying when starting service. #4804 - - ## 1.13.0 2022-07-14 * [CHANGE] Changed default for `-ingester.min-ready-duration` from 1 minute to 15 seconds. #4539 diff --git a/Makefile b/Makefile index bbab71dd0a..2a658b8ffe 100644 --- a/Makefile +++ b/Makefile @@ -96,15 +96,12 @@ pkg/frontend/v1/frontendv1pb/frontend.pb.go: pkg/frontend/v1/frontendv1pb/fronte pkg/frontend/v2/frontendv2pb/frontend.pb.go: pkg/frontend/v2/frontendv2pb/frontend.proto pkg/querier/queryrange/queryrange.pb.go: pkg/querier/queryrange/queryrange.proto pkg/querier/stats/stats.pb.go: pkg/querier/stats/stats.proto -pkg/chunk/storage/caching_index_client.pb.go: pkg/chunk/storage/caching_index_client.proto pkg/distributor/ha_tracker.pb.go: pkg/distributor/ha_tracker.proto pkg/ruler/rulespb/rules.pb.go: pkg/ruler/rulespb/rules.proto pkg/ruler/ruler.pb.go: pkg/ruler/ruler.proto pkg/ring/kv/memberlist/kv.pb.go: pkg/ring/kv/memberlist/kv.proto pkg/scheduler/schedulerpb/scheduler.pb.go: pkg/scheduler/schedulerpb/scheduler.proto pkg/storegateway/storegatewaypb/gateway.pb.go: pkg/storegateway/storegatewaypb/gateway.proto -pkg/chunk/grpc/grpc.pb.go: pkg/chunk/grpc/grpc.proto -tools/blocksconvert/scheduler.pb.go: tools/blocksconvert/scheduler.proto pkg/alertmanager/alertmanagerpb/alertmanager.pb.go: pkg/alertmanager/alertmanagerpb/alertmanager.proto pkg/alertmanager/alertspb/alerts.pb.go: pkg/alertmanager/alertspb/alerts.proto @@ -359,7 +356,7 @@ dist/$(UPTODATE)-packages: dist $(wildcard packaging/deb/**) $(wildcard packagin --before-remove packaging/deb/control/prerm \ --package dist/cortex-$(VERSION)_$$arch.deb \ dist/cortex-linux-$$arch=/usr/local/bin/cortex \ - docs/chunks-storage/single-process-config.yaml=/etc/cortex/single-process-config.yaml \ + docs/configuration/single-process-config-blocks.yaml=/etc/cortex/single-process-config.yaml \ packaging/deb/default/cortex=/etc/default/cortex \ packaging/deb/systemd/cortex.service=/etc/systemd/system/cortex.service; \ $(FPM_OPTS) -t rpm \ @@ -368,7 +365,7 @@ dist/$(UPTODATE)-packages: dist $(wildcard packaging/deb/**) $(wildcard packagin --before-remove packaging/rpm/control/preun \ --package dist/cortex-$(VERSION)_$$arch.rpm \ dist/cortex-linux-$$arch=/usr/local/bin/cortex \ - docs/chunks-storage/single-process-config.yaml=/etc/cortex/single-process-config.yaml \ + docs/configuration/single-process-config-blocks.yaml=/etc/cortex/single-process-config.yaml \ packaging/rpm/sysconfig/cortex=/etc/sysconfig/cortex \ packaging/rpm/systemd/cortex.service=/etc/systemd/system/cortex.service; \ done diff --git a/Makefile.local.example b/Makefile.local.example index 7faa0bb13c..7ea7ace91f 100644 --- a/Makefile.local.example +++ b/Makefile.local.example @@ -3,8 +3,5 @@ BUILD_IMAGE ?= quay.io/cortexproject/build-image IMAGE_PREFIX ?= custom-prefix/ -blocksconvert-push: cmd/blocksconvert/.uptodate - docker push $(IMAGE_PREFIX)blocksconvert:$(IMAGE_TAG) - cortex-push: cmd/cortex/.uptodate docker push $(IMAGE_PREFIX)cortex:$(IMAGE_TAG) diff --git a/cmd/blocksconvert/Dockerfile b/cmd/blocksconvert/Dockerfile deleted file mode 100644 index 100ee9aa3b..0000000000 --- a/cmd/blocksconvert/Dockerfile +++ /dev/null @@ -1,9 +0,0 @@ -FROM alpine:3.14 -RUN apk add --no-cache ca-certificates -COPY blocksconvert / -ENTRYPOINT ["/blocksconvert"] - -ARG revision -LABEL org.opencontainers.image.title="blocksconvert" \ - org.opencontainers.image.source="https://github.com/cortexproject/cortex/tree/master/tools/blocksconvert" \ - org.opencontainers.image.revision="${revision}" diff --git a/cmd/blocksconvert/main.go b/cmd/blocksconvert/main.go deleted file mode 100644 index f3f2b8a5e0..0000000000 --- a/cmd/blocksconvert/main.go +++ /dev/null @@ -1,107 +0,0 @@ -package main - -import ( - "context" - "flag" - "fmt" - "os" - "strings" - - "github.com/go-kit/log/level" - "github.com/prometheus/client_golang/prometheus" - "github.com/weaveworks/common/server" - "github.com/weaveworks/common/signals" - - "github.com/cortexproject/cortex/pkg/cortex" - util_log "github.com/cortexproject/cortex/pkg/util/log" - "github.com/cortexproject/cortex/pkg/util/services" - "github.com/cortexproject/cortex/tools/blocksconvert" - "github.com/cortexproject/cortex/tools/blocksconvert/builder" - "github.com/cortexproject/cortex/tools/blocksconvert/cleaner" - "github.com/cortexproject/cortex/tools/blocksconvert/scanner" - "github.com/cortexproject/cortex/tools/blocksconvert/scheduler" -) - -type Config struct { - Target string - ServerConfig server.Config - - SharedConfig blocksconvert.SharedConfig - ScannerConfig scanner.Config - BuilderConfig builder.Config - SchedulerConfig scheduler.Config - CleanerConfig cleaner.Config -} - -func main() { - cfg := Config{} - flag.StringVar(&cfg.Target, "target", "", "Module to run: Scanner, Scheduler, Builder") - cfg.SharedConfig.RegisterFlags(flag.CommandLine) - cfg.ScannerConfig.RegisterFlags(flag.CommandLine) - cfg.BuilderConfig.RegisterFlags(flag.CommandLine) - cfg.SchedulerConfig.RegisterFlags(flag.CommandLine) - cfg.CleanerConfig.RegisterFlags(flag.CommandLine) - cfg.ServerConfig.RegisterFlags(flag.CommandLine) - flag.Parse() - - util_log.InitLogger(&cfg.ServerConfig) - - cortex.DisableSignalHandling(&cfg.ServerConfig) - serv, err := server.New(cfg.ServerConfig) - if err != nil { - level.Error(util_log.Logger).Log("msg", "Unable to initialize server", "err", err.Error()) - os.Exit(1) - } - - cfg.Target = strings.ToLower(cfg.Target) - - registry := prometheus.DefaultRegisterer - - var targetService services.Service - switch cfg.Target { - case "scanner": - targetService, err = scanner.NewScanner(cfg.ScannerConfig, cfg.SharedConfig, util_log.Logger, registry) - case "builder": - targetService, err = builder.NewBuilder(cfg.BuilderConfig, cfg.SharedConfig, util_log.Logger, registry) - case "scheduler": - targetService, err = scheduler.NewScheduler(cfg.SchedulerConfig, cfg.SharedConfig, util_log.Logger, registry, serv.HTTP, serv.GRPC) - case "cleaner": - targetService, err = cleaner.NewCleaner(cfg.CleanerConfig, cfg.SharedConfig, util_log.Logger, registry) - default: - err = fmt.Errorf("unknown target") - } - - if err != nil { - level.Error(util_log.Logger).Log("msg", "failed to initialize", "err", err) - os.Exit(1) - } - - servService := cortex.NewServerService(serv, func() []services.Service { - return []services.Service{targetService} - }) - servManager, err := services.NewManager(servService, targetService) - if err == nil { - servManager.AddListener(services.NewManagerListener(nil, nil, func(service services.Service) { - servManager.StopAsync() - })) - - err = services.StartManagerAndAwaitHealthy(context.Background(), servManager) - } - if err != nil { - level.Error(util_log.Logger).Log("msg", "Unable to start", "err", err.Error()) - os.Exit(1) - } - - // Setup signal handler and ask service maanger to stop when signal arrives. - handler := signals.NewHandler(serv.Log) - go func() { - handler.Loop() - servManager.StopAsync() - }() - - // We only wait for target service. If any other service fails, listener will stop it (via manager) - if err := targetService.AwaitTerminated(context.Background()); err != nil { - level.Error(util_log.Logger).Log("msg", cfg.Target+" failed", "err", targetService.FailureCase()) - os.Exit(1) - } -} diff --git a/docs/api/_index.md b/docs/api/_index.md index c20975ec95..32dce280eb 100644 --- a/docs/api/_index.md +++ b/docs/api/_index.md @@ -28,7 +28,7 @@ For the sake of clarity, in this document we have grouped API endpoints by servi | [Remote write](#remote-write) | Distributor | `POST /api/v1/push` | | [Tenants stats](#tenants-stats) | Distributor | `GET /distributor/all_user_stats` | | [HA tracker status](#ha-tracker-status) | Distributor | `GET /distributor/ha_tracker` | -| [Flush chunks / blocks](#flush-chunks--blocks) | Ingester | `GET,POST /ingester/flush` | +| [Flush blocks](#flush-blocks) | Ingester | `GET,POST /ingester/flush` | | [Shutdown](#shutdown) | Ingester | `GET,POST /ingester/shutdown` | | [Ingesters ring status](#ingesters-ring-status) | Ingester | `GET /ingester/ring` | | [Instant query](#instant-query) | Querier, Query-frontend | `GET,POST /api/v1/query` | @@ -40,7 +40,6 @@ For the sake of clarity, in this document we have grouped API endpoints by servi | [Get metric metadata](#get-metric-metadata) | Querier, Query-frontend | `GET /api/v1/metadata` | | [Remote read](#remote-read) | Querier, Query-frontend | `POST /api/v1/read` | | [Get tenant ingestion stats](#get-tenant-ingestion-stats) | Querier | `GET /api/v1/user_stats` | -| [Get tenant chunks](#get-tenant-chunks) | Querier | `GET /api/v1/chunks` | | [Ruler ring status](#ruler-ring-status) | Ruler | `GET /ruler/ring` | | [Ruler rules ](#ruler-rule-groups) | Ruler | `GET /ruler/rule_groups` | | [List rules](#list-rules) | Ruler | `GET /api/v1/rules` | @@ -60,9 +59,6 @@ For the sake of clarity, in this document we have grouped API endpoints by servi | [Get Alertmanager configuration](#get-alertmanager-configuration) | Alertmanager | `GET /api/v1/alerts` | | [Set Alertmanager configuration](#set-alertmanager-configuration) | Alertmanager | `POST /api/v1/alerts` | | [Delete Alertmanager configuration](#delete-alertmanager-configuration) | Alertmanager | `DELETE /api/v1/alerts` | -| [Delete series](#delete-series) | Purger | `PUT,POST /api/v1/admin/tsdb/delete_series` | -| [List delete requests](#list-delete-requests) | Purger | `GET /api/v1/admin/tsdb/delete_series` | -| [Cancel delete request](#cancel-delete-request) | Purger | `PUT,POST /api/v1/admin/tsdb/cancel_delete_request` | | [Tenant delete request](#tenant-delete-request) | Purger | `POST /purger/delete_tenant` | | [Tenant delete status](#tenant-delete-status) | Purger | `GET /purger/delete_tenant_status` | | [Store-gateway ring status](#store-gateway-ring-status) | Store-gateway | `GET /store-gateway/ring` | @@ -247,7 +243,7 @@ Displays a web page with the current status of the HA tracker, including the ele ## Ingester -### Flush chunks / blocks +### Flush blocks ``` GET,POST /ingester/flush @@ -256,11 +252,11 @@ GET,POST /ingester/flush GET,POST /flush ``` -Triggers a flush of the in-memory time series data (chunks or blocks) to the long-term storage. This endpoint triggers the flush also when `-ingester.flush-on-shutdown-with-wal-enabled` or `-blocks-storage.tsdb.flush-blocks-on-shutdown` are disabled. +Triggers a flush of the in-memory time series data to the long-term storage. This endpoint triggers the flush also when `-ingester.flush-on-shutdown-with-wal-enabled` or `-blocks-storage.tsdb.flush-blocks-on-shutdown` are disabled. -When using blocks storage, this endpoint accepts `tenant` parameter to specify tenant whose blocks are compacted and shipped. This parameter may be specified multiple times to select more tenants. If no tenant is specified, all tenants are flushed. +This endpoint accepts `tenant` parameter to specify tenant whose blocks are compacted and shipped. This parameter may be specified multiple times to select more tenants. If no tenant is specified, all tenants are flushed. -Flush endpoint now also accepts `wait=true` parameter, which makes the call synchronous – it will only return after flushing has finished. Note that returned status code does not reflect the result of flush operation. This parameter is only available when using blocks storage. +Flush endpoint now also accepts `wait=true` parameter, which makes the call synchronous – it will only return after flushing has finished. Note that returned status code does not reflect the result of flush operation. ### Shutdown @@ -427,25 +423,6 @@ Returns realtime ingestion rate, for the authenticated tenant, in `JSON` format. _Requires [authentication](#authentication)._ -### Get tenant chunks - -``` -GET /api/v1/chunks - -# Legacy -GET /chunks -``` - -Fetch a compressed tar of all the chunks containing samples for the given time range and label matchers. This endpoint is supported only by the **chunks storage**, requires `-querier.ingester-streaming=true` and should **not be exposed to users** but just used for debugging purposes. - -| URL query parameter | Description | -| ------------------- | ----------- | -| `start` | Start timestamp, in RFC3339 format or unix epoch. | -| `end` | End timestamp, in RFC3339 format or unix epoch. | -| `matcher` | Label matcher that selects the series for which chunks should be fetched. | - -_Requires [authentication](#authentication)._ - ## Ruler The ruler API endpoints require to configure a backend object storage to store the recording rules and alerts. The ruler API uses the concept of a "namespace" when creating rule groups. This is a stand in for the name of the rule file in Prometheus and rule groups must be named uniquely within a namespace. @@ -810,52 +787,7 @@ _Requires [authentication](#authentication)._ ## Purger -The Purger service provides APIs for requesting deletion of series in chunks storage and managing delete requests. For more information about it, please read the [Delete series Guide](../guides/deleting-series.md). - -### Delete series - -``` -PUT,POST /api/v1/admin/tsdb/delete_series - -# Legacy -PUT,POST /api/v1/admin/tsdb/delete_series -``` - -Prometheus-compatible delete series endpoint. - -_For more information, please check out the Prometheus [delete series](https://prometheus.io/docs/prometheus/latest/querying/api/#delete-series) documentation._ - -_Requires [authentication](#authentication)._ - -### List delete requests - -``` -GET /api/v1/admin/tsdb/delete_series - -# Legacy -GET /api/v1/admin/tsdb/delete_series -``` - -List all the delete requests. - -_Requires [authentication](#authentication)._ - -### Cancel delete request - -``` -PUT,POST /api/v1/admin/tsdb/cancel_delete_request - -# Legacy -PUT,POST /api/v1/admin/tsdb/cancel_delete_request -``` - -Cancel a delete request while the request is still in the grace period (before the request is effectively processed by the purger and time series data is hard-deleted from the storage). - -| URL query parameter | Description | -| ------------------- | ----------- | -| `request_id` | Deletion request ID to cancel. Can be obtained by the [List delete requests](#list-delete-requests) endpoint. | - -_Requires [authentication](#authentication)._ +The Purger service provides APIs for requesting deletion of tenants. ### Tenant Delete Request diff --git a/docs/architecture.md b/docs/architecture.md index 2249e29c25..4114170fbd 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -19,32 +19,7 @@ Incoming samples (writes from Prometheus) are handled by the [distributor](#dist ## Storage -Cortex currently supports two storage engines to store and query the time series: - -- Chunks (deprecated) -- Blocks - -The two engines mostly share the same Cortex architecture with few differences outlined in the rest of the document. - -### Chunks storage (deprecated) - -The chunks storage stores each single time series into a separate object called _Chunk_. Each Chunk contains the samples for a given period (defaults to 12 hours). Chunks are then indexed by time range and labels, in order to provide a fast lookup across many (over millions) Chunks. - -For this reason, the chunks storage consists of: - -* An index for the Chunks. This index can be backed by: - * [Amazon DynamoDB](https://aws.amazon.com/dynamodb) - * [Google Bigtable](https://cloud.google.com/bigtable) - * [Apache Cassandra](https://cassandra.apache.org) -* An object store for the Chunk data itself, which can be: - * [Amazon DynamoDB](https://aws.amazon.com/dynamodb) - * [Google Bigtable](https://cloud.google.com/bigtable) - * [Apache Cassandra](https://cassandra.apache.org) - * [Amazon S3](https://aws.amazon.com/s3) - * [Google Cloud Storage](https://cloud.google.com/storage/) - * [Microsoft Azure Storage](https://azure.microsoft.com/en-us/services/storage/) - -For more information, please check out the [Chunks storage](./chunks-storage/_index.md) documentation. +Cortex currently supports the `blocks` storage engine to store and query time series. It used to support `chunks` storage in the past. ### Blocks storage @@ -151,14 +126,14 @@ We recommend randomly load balancing write requests across distributor instances The **ingester** service is responsible for writing incoming series to a [long-term storage backend](#storage) on the write path and returning in-memory series samples for queries on the read path. -Incoming series are not immediately written to the storage but kept in memory and periodically flushed to the storage (by default, 12 hours for the chunks storage and 2 hours for the blocks storage). For this reason, the [queriers](#querier) may need to fetch samples both from ingesters and long-term storage while executing a query on the read path. +Incoming series are not immediately written to the storage but kept in memory and periodically flushed to the storage (by default, 2 hours). For this reason, the [queriers](#querier) may need to fetch samples both from ingesters and long-term storage while executing a query on the read path. Ingesters contain a **lifecycler** which manages the lifecycle of an ingester and stores the **ingester state** in the [hash ring](#the-hash-ring). Each ingester could be in one of the following states: - **`PENDING`**
- The ingester has just started. While in this state, the ingester doesn't receive neither write and read requests, and could be waiting for time series data transfer from another ingester if running the chunks storage and the [hand-over](guides/ingesters-rolling-updates.md#chunks-storage-with-wal-disabled-hand-over) is enabled. + The ingester has just started. While in this state, the ingester doesn't receive neither write and read requests. - **`JOINING`**
- The ingester is starting up and joining the ring. While in this state the ingester doesn't receive neither write and read requests. The ingester will join the ring using tokens received by a leaving ingester as part of the [hand-over](guides/ingesters-rolling-updates.md#chunks-storage-with-wal-disabled-hand-over) process (if enabled), otherwise it could load tokens from disk (if `-ingester.tokens-file-path` is configured) or generate a set of new random ones. Finally, the ingester optionally observes the ring for tokens conflicts and then, once any conflict is resolved, will move to `ACTIVE` state. + The ingester is starting up and joining the ring. While in this state the ingester doesn't receive neither write and read requests. The ingester will join the ring using tokens loaded from disk (if `-ingester.tokens-file-path` is configured) or generate a set of new random ones. Finally, the ingester optionally observes the ring for tokens conflicts and then, once any conflict is resolved, will move to `ACTIVE` state. - **`ACTIVE`**
The ingester is up and running. While in this state the ingester can receive both write and read requests. - **`LEAVING`**
@@ -166,8 +141,6 @@ Ingesters contain a **lifecycler** which manages the lifecycle of an ingester an - **`UNHEALTHY`**
The ingester has failed to heartbeat to the ring's KV Store. While in this state, distributors skip the ingester while building the replication set for incoming series and the ingester does not receive write or read requests. -_The ingester states are internally used for different purposes, including the series hand-over process supported by the chunks storage. For more information about it, please check out the [Ingester hand-over](guides/ingesters-rolling-updates.md#chunks-storage-with-wal-disabled-hand-over) documentation._ - Ingesters are **semi-stateful**. #### Ingesters failure and data loss @@ -183,8 +156,6 @@ The **write-ahead log** (WAL) is used to write to a persistent disk all incoming Contrary to the sole replication and given the persistent disk data is not lost, in the event of multiple ingesters failure each ingester will recover the in-memory series samples from WAL upon subsequent restart. The replication is still recommended in order to ensure no temporary failures on the read path in the event of a single ingester failure. -The WAL for the chunks storage is disabled by default, while it's always enabled for the blocks storage. - #### Ingesters write de-amplification Ingesters store recently received samples in-memory in order to perform write de-amplification. If the ingesters would immediately write received samples to the long-term storage, the system would be very difficult to scale due to the very high pressure on the storage. For this reason, the ingesters batch and compress samples in-memory and periodically flush them out to the storage. diff --git a/docs/blocks-storage/_index.md b/docs/blocks-storage/_index.md index 97e5997b5d..32619b8994 100644 --- a/docs/blocks-storage/_index.md +++ b/docs/blocks-storage/_index.md @@ -33,8 +33,6 @@ The **[compactor](./compactor.md)** is responsible to merge and deduplicate smal The `alertmanager` and `ruler` components can also use object storage to store its configurations and rules uploaded by users. In that case a separate bucket should be created to store alertmanager configurations and rules: using the same bucket between ruler/alertmanager and blocks will cause issue with the **[compactor](./compactor.md)**. -Finally, the [**table-manager**](../chunks-storage/table-manager.md) and the [**schema config**](../chunks-storage/schema-config.md) are **not used** by the blocks storage. - ### The write path **Ingesters** receive incoming samples from the distributors. Each push request belongs to a tenant, and the ingester appends the received samples to the specific per-tenant TSDB stored on the local disk. The received samples are both kept in-memory and written to a write-ahead log (WAL) and used to recover the in-memory series in case the ingester abruptly terminates. The per-tenant TSDB is lazily created in each ingester as soon as the first samples are received for that tenant. @@ -47,7 +45,7 @@ In order to effectively use the **WAL** and being able to recover the in-memory The series sharding and replication done by the distributor doesn't change based on the storage engine. -It's important to note that - differently than the [chunks storage](../chunks-storage/_index.md) - due to the replication factor N (typically 3), each time series is stored by N ingesters. Since each ingester writes its own block to the long-term storage, this leads a storage utilization N times more than the chunks storage. [Compactor](./compactor.md) solves this problem by merging blocks from multiple ingesters into a single block, and removing duplicated samples. After blocks compaction, the storage utilization is significantly smaller compared to the chunks storage for the same exact series and samples. +It's important to note that due to the replication factor N (typically 3), each time series is stored by N ingesters. Since each ingester writes its own block to the long-term storage, this leads a storage utilization N times more. [Compactor](./compactor.md) solves this problem by merging blocks from multiple ingesters into a single block, and removing duplicated samples. After blocks compaction, the storage utilization is significantly smaller. For more information, please refer to the following dedicated sections: @@ -68,17 +66,6 @@ For more information, please refer to the following dedicated sections: - [Store-gateway](./store-gateway.md) - [Production tips](./production-tips.md) -## Configuration - -The general [configuration documentation](../configuration/config-file-reference.md) also applies to a Cortex cluster running the blocks storage. The blocks storage can be enabled switching the storage `engine` to `blocks`: - -```yaml -storage: - # The storage engine to use. Use "blocks" for the blocks storage. - # CLI flag: -store.engine - engine: blocks -``` - ## Known issues GitHub issues tagged with the [`storage/blocks`](https://github.com/cortexproject/cortex/issues?q=is%3Aopen+is%3Aissue+label%3Astorage%2Fblocks) label are the best source of currently known issues affecting the blocks storage. diff --git a/docs/blocks-storage/querier.md b/docs/blocks-storage/querier.md index e98785feb1..ee0029547d 100644 --- a/docs/blocks-storage/querier.md +++ b/docs/blocks-storage/querier.md @@ -7,7 +7,7 @@ slug: querier -The **querier** service handles queries using the [PromQL](https://prometheus.io/docs/prometheus/latest/querying/basics/) query language. The querier service is used both by the chunks and blocks storage, and the [general architecture documentation](../architecture.md#querier) applies to the blocks storage too, except for the differences described in this document. +The **querier** service handles queries using the [PromQL](https://prometheus.io/docs/prometheus/latest/querying/basics/) query language. This document dives into the storage-specific details of the querier service. The [general architecture documentation](../architecture.md#querier) applies too. The querier is **stateless**. @@ -207,15 +207,6 @@ querier: # CLI flag: -querier.store-gateway-client.tls-insecure-skip-verify [tls_insecure_skip_verify: | default = false] - # Second store engine to use for querying. Empty = disabled. - # CLI flag: -querier.second-store-engine - [second_store_engine: | default = ""] - - # If specified, second store is only used for queries before this timestamp. - # Default value 0 means secondary store is always queried. - # CLI flag: -querier.use-second-store-before-time - [use_second_store_before_time: