Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new bi sample benchmark tests #1995

Closed
wants to merge 1,187 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1187 commits
Select commit Hold shift + click to select a range
bb506d4
Enhancement/515/support metadata in finalize staged data (#1525)
alexowens90 Apr 25, 2024
86350d6
Add empty index type and feature flag for it (#1524)
vasil-pashov Apr 25, 2024
3bd39c5
Bugfix 1507+1509: dynamic schema different index append fixes (#1529)
alexowens90 Apr 26, 2024
cb91286
Should speed up ASV benchmarks significantly
IvoDD Apr 25, 2024
2cc90c3
Add CODEOWNERS for cpp/ and python/
poodlewars Apr 24, 2024
547cd6b
list_snapshots: Add option to skip loading metadata (#1531)
muhammadhamzasajjad Apr 26, 2024
9858652
LibraryTool add read_index (#1533)
muhammadhamzasajjad Apr 26, 2024
7e58881
Docs: Improve Landing Page (#1520)
DrNickClarke Apr 30, 2024
3228ab8
Fix short_wide asv modification benchmarks
IvoDD May 1, 2024
bb0294a
Fix crash in calling Library::tail when the tail contains empty typed…
vasil-pashov May 2, 2024
e6c81e9
Bugfix/944: Include field index in error message (#1536)
muhammadhamzasajjad May 2, 2024
43137db
Updates the asv json file after #1544
IvoDD May 2, 2024
7ddb618
Revert destructor speed regression from #1152
IvoDD May 1, 2024
571af72
Revert to C++17 build
willdealtry Apr 30, 2024
c337af1
Release GIL on update and append
willdealtry May 8, 2024
cf8393f
Added Data Organisation Guide to Docs (#1553)
DrNickClarke May 9, 2024
4073298
Completely disable Pandas consolidation (#1561)
vasil-pashov May 10, 2024
d9d847b
maint: Remove use of folly/ThreadCachedInt (#1486)
ianthomas23 May 14, 2024
807bda0
Improve SlabAlloc cpp tests speed
IvoDD May 14, 2024
4d916e0
Only write ref key once when writing with prune_previous_versions (#1…
poodlewars May 15, 2024
a8cc787
Update openssl to v3 on pypi
phoebusm Apr 11, 2024
0954588
Add prune_previous to various functions
phoebusm May 3, 2024
bdb6da4
Correct last_update_time type hint (#1573)
jamesmunro May 17, 2024
1578935
Fix #1148: Show clearer error message for mismatched types in QueryBu…
muhammadhamzasajjad May 21, 2024
6bc43bd
Store naming (#1528)
G-D-Petrov May 21, 2024
dbc1058
maint: Update to openssl 3.3.0 (#1575)
jjerphan May 21, 2024
223768f
conda-build: Unpin libcurl (#1581)
jjerphan May 23, 2024
0823b57
-Set ca path and directory automatically
Sep 14, 2023
53ff421
Add more FAQs (#1567)
ms041223 May 29, 2024
81ee498
Resampling MVP (#1495)
alexowens90 May 30, 2024
d601836
Billion row challenge speedup (#1584)
alexowens90 May 31, 2024
34adf36
Update docstring: added section to clarify that an empty df does not …
DrNickClarke Jun 3, 2024
bdca564
Adds a failing stress test which writes a large dataframe to storage
IvoDD May 28, 2024
28df1cd
Fix overflow bug on append
IvoDD May 28, 2024
e93d604
Skips the large_df stress tests and adds a lighter version
IvoDD May 31, 2024
fa396f4
build: Use C++20 and dynamically link the C++ standard library (#1572)
jjerphan Jun 3, 2024
5c0f2c3
Enhancement 1404: query builder with sparse data (#1595)
alexowens90 Jun 4, 2024
3b371c9
Add tests reading/writing/appending/updating unicode strings (#1599)
G-D-Petrov Jun 5, 2024
c0e5a72
Extend memory tracing for programatic use and to report on max RSS (#…
poodlewars Jun 5, 2024
688fd55
maint: Update to boost 1.84.0 (#1574)
jjerphan Jun 5, 2024
3931c63
maint: Update to folly 2023.09.25.00 (#1576)
jjerphan Jun 5, 2024
869d42c
Pin Windows CI to VS 17.9.7 toolset as runner image update has build …
muhammadhamzasajjad Jun 10, 2024
963125c
Refactor 1608: Re-introduce C++20 features (#1614)
alexowens90 Jun 11, 2024
4ff88af
Bugfix 1615: Fix segfault in benchmark_clause.cpp (#1616)
alexowens90 Jun 11, 2024
8646050
Bugfix 1612: Raise a meaningful exception type/message when an empty …
alexowens90 Jun 11, 2024
2a0a1c5
Make service alive checking more universal
phoebusm Jun 10, 2024
3bcae7c
Bugfix 1609: Allow parallel appends when incomplete index values matc…
alexowens90 Jun 12, 2024
d3a0d7c
Compile and run C++ benchmarks in CI (results currently unused) (#1620)
alexowens90 Jun 12, 2024
b8f9cfe
Aws permissions guide (#1622)
jamesmunro Jun 13, 2024
b73f7d0
Remove test mark dependency in storage fixtures
phoebusm Jun 11, 2024
968a206
Additional warning for prometheus
phoebusm Jun 13, 2024
a3ce2d5
Add docs link (#1625)
jamesmunro Jun 18, 2024
b929f5f
Adds a unit test for lib.version_store.sort_index
IvoDD Jun 19, 2024
f14d6c3
Fix bullet point lists in query builder docstrings (#1594)
DrNickClarke Jun 21, 2024
f44aa62
Added resample demo notebook (#1634)
DrNickClarke Jun 21, 2024
bb7f2ff
Fix sort merge and add basic tests
willdealtry Apr 24, 2024
5992b77
Adapt Raise to use format string as message directly if no args are g…
muhammadhamzasajjad Jun 21, 2024
662fa1b
Enhancement/1610/method to force compact symbol list (#1624)
alexowens90 Jun 21, 2024
5787f83
build: Update `azure-core-cpp` to `1.12.0` (#1639)
jjerphan Jun 25, 2024
fa4fb1f
Set numpy<2 for older arcticdb (#1644)
muhammadhamzasajjad Jun 25, 2024
f29c3d9
Implement support for very old style symbol list entries. (#1630)
poodlewars Jun 25, 2024
488f44a
Add binary encoding format
willdealtry Feb 9, 2024
4901dcc
Fix tests
willdealtry Jun 25, 2024
298dce3
Reduce memory usage during conda build (#1649)
muhammadhamzasajjad Jun 26, 2024
7c51644
Fix ArcticDB reading streaming data (#1647)
poodlewars Jun 28, 2024
f444d28
conda-build: Replace `boost-cpp` with `libboost-devel` (#1653)
jjerphan Jul 1, 2024
add35c9
Upgrade to Protobuf 5 (#1635)
muhammadhamzasajjad Jul 3, 2024
e3b03fc
Try fix gh action checkout issue
phoebusm Jul 4, 2024
932a1c2
Update benchmark_commits.yml and fix the checkout there
G-D-Petrov Jul 5, 2024
00ca0ab
Refactor log_event function and add retry logic (#1660)
G-D-Petrov Jul 8, 2024
fb1cb6d
Add note about staged/incomplete data and delete (#1669)
jamesmunro Jul 11, 2024
183ab31
update docs: map_size is per library (#1651)
jamesmunro Jul 11, 2024
a8ebdfe
Pin libcxx for conda to last known working versions (#1680)
G-D-Petrov Jul 12, 2024
1094b14
Refactor/1643/reduce prune previous memory footprint (#1657)
alexowens90 Jul 15, 2024
d8e9989
Bugfix 1637: append does not respect prune previous (#1640)
alexowens90 Jul 15, 2024
f8c1c75
Sort index should always start from row zero
willdealtry Jul 15, 2024
20b3b06
Swap out random_shuffle
willdealtry Jul 16, 2024
c5b02c0
Add object information to the exception messages (#1700)
G-D-Petrov Jul 22, 2024
5b64ad4
-Make modification of snapshot atomic
phoebusm Jul 19, 2024
9421fb8
Added symbol to ReadRequest docstring (#1702)
DrNickClarke Jul 22, 2024
d4f18ed
Port sparse data test from old repo
phoebusm Jul 18, 2024
be8b32f
Remove todo
phoebusm Jul 18, 2024
c7046b1
Add support for cgroup v2 cpu limits (#1696)
G-D-Petrov Jul 23, 2024
b11de42
Fix setup.py to cope with newer pip versions (#1706)
G-D-Petrov Jul 23, 2024
e5dde93
Bugfix 1683: Add validate_index arg to staged data finalization in bo…
alexowens90 Jul 23, 2024
34e2ca3
Fix resampling using invalid iterator (#1661)
vasil-pashov Jul 23, 2024
b0ffcd2
Adds a failing test to demonstrate #1384
IvoDD Mar 19, 2024
3fbd7ce
Move updating the entry inside write_tombstone function
IvoDD Apr 18, 2024
cf5a048
Simplify version map has_cached_entry check
IvoDD Mar 21, 2024
66fbc98
Invalidate undeleted cache entry when deleting version
IvoDD Apr 18, 2024
5a13693
Extract include_deleted out of LoadType and allow full cartesian product
IvoDD Mar 20, 2024
d80a695
Rename following version chain end conditions
IvoDD Mar 20, 2024
0757f9e
Update version map unsync tolerance to 200ms
IvoDD Mar 26, 2024
a149404
Rename `ToLoad` to `LoadObjective`
IvoDD May 1, 2024
da6b9a8
Removes `iterate_on_failure` argument and related functionality
IvoDD May 1, 2024
9d22629
Re-adds iterate_on_failure and deprecates it
IvoDD May 21, 2024
119d4ad
Better naming of version map LoadType and LoadObjective enum values
IvoDD Jul 11, 2024
7d42122
Add option to omit all runtime checks for MSVC. (#1705)
vasil-pashov Jul 24, 2024
ebe9877
Bugfix 1692: fix batch read with row range no query builder (#1710)
alexowens90 Jul 26, 2024
af0cc68
Change the default `dynamic_strings` to True
IvoDD Jul 26, 2024
e408beb
Add a better error message for bad variant access when taking buffer …
poodlewars Jul 29, 2024
1c472da
Add test for version map cache consistency when compacting
IvoDD Jul 26, 2024
784565c
Fix version map cache update when compacting
IvoDD Jul 26, 2024
07e317b
Add tests for new cgroup handling (#1718)
G-D-Petrov Jul 30, 2024
83bb372
Add support for testing NFS based layout with S3 backend (#1719)
poodlewars Jul 30, 2024
c88a3b8
conda-build: Unpin libcxx <18 (#1726)
jjerphan Jul 31, 2024
e3b2b73
Adds a test to expose #1716
IvoDD Jul 31, 2024
b5d0ed0
Fixes #1716 by using an std::vector copy
IvoDD Jul 31, 2024
da7e34b
Bugfix 1720: Speedup processing pipeline with lots of columns (#1723)
alexowens90 Jul 31, 2024
5fcc1b4
Read index API (#1568)
vasil-pashov Aug 1, 2024
c6b999a
Fix use after move in `op_log.cpp` (#1732)
alexowens90 Aug 1, 2024
a4f978c
Release GIL on recursive normalizers (master) (#1684)
willdealtry Aug 1, 2024
6e12f30
Add maybe_unused to fix MSVC compilation error (#1739)
vasil-pashov Aug 2, 2024
7a78945
Properly hash descriptor in v1
willdealtry Jul 11, 2024
5df9259
Add comment
willdealtry Aug 5, 2024
2466903
Test single-letter column names
willdealtry Aug 5, 2024
6ee6c5a
Check identical segements v2 encoding
willdealtry Aug 5, 2024
abaaabd
Extend tests for merge sort (#1708)
vasil-pashov Aug 6, 2024
93a241a
Check if object is an instance of QueryBuilder when comparing for equ…
vasil-pashov Aug 7, 2024
74e8e2d
Add resampling offset (#1743)
vasil-pashov Aug 7, 2024
4d5b8d8
Refactor 1722: Remove Composite from processing pipeline
alexowens90 Jul 31, 2024
3384d76
Fix Segment use-after-move when replicating to NFS (#1756)
poodlewars Aug 9, 2024
0ea2508
Throw exception if appending using sort_and_finalize will create unor…
vasil-pashov Aug 12, 2024
7342f58
Improve logging for stress tests (#1759)
G-D-Petrov Aug 12, 2024
af3f7c4
Remove rocksdb
poodlewars Aug 10, 2024
9b52fd8
Update BSL table for v4.5.0 (#1768)
G-D-Petrov Aug 16, 2024
ad4c4cf
Handle KeyNotFoundException in recurse_index_key (#1766)
G-D-Petrov Aug 16, 2024
132eb1b
Add Storage API to check for existence of a key matching a predicate …
poodlewars Aug 16, 2024
576a056
WIP plumbing in ArcticDB
poodlewars Aug 19, 2024
b8b6e7a
Revert "Handle KeyNotFoundException in recurse_index_key" (#1776)
G-D-Petrov Aug 20, 2024
ad15d76
Add 1 Billion Row Challenge Notebook (#1774)
DrNickClarke Aug 20, 2024
11a028c
Remove KeyNotFoundException catch in CopyCompressedInterStoreTask (#1…
G-D-Petrov Aug 21, 2024
914e4f2
Limit 3 item ref key bypass to only when loading undeleted versions (…
G-D-Petrov Aug 21, 2024
e266eee
Fixes for sort and finalize (#1763)
vasil-pashov Aug 21, 2024
4986432
Bugfix 1552: Set row_count/rows to None in get_description/get_info a…
alexowens90 Aug 21, 2024
bbeec13
Bugfix 1641: Remove unused staged arg from write_pickle_batch (#1642)
alexowens90 Aug 22, 2024
1e0e6a9
Refactor 1749: make processing tests great again (#1758)
alexowens90 Aug 22, 2024
2da2476
append docstring improvement: added that append on a new symbol will …
DrNickClarke Aug 22, 2024
22f315e
Release GIL on read
willdealtry Jul 2, 2024
5987203
Bugfix 1460: preserve nanoseconds and timezones in date range from ge…
alexowens90 Aug 23, 2024
4f7c408
Enhancement 1666: Do not fallback to iterating snapshots if if versio…
alexowens90 Aug 23, 2024
64bf94c
Add interface to pass a cached entry to tombstone all (#1793)
G-D-Petrov Aug 28, 2024
b8e7365
Adds a test to showcase the sort_index wrong version issue
IvoDD Aug 27, 2024
3b56967
Fix the sort_index and delete_range wrong version id bug
IvoDD Aug 27, 2024
386ebc9
Fix c++ tests which wrote wrong version index keys
IvoDD Aug 28, 2024
289ea78
Fix windpws CI by adding the string header in decimal.cpp
Aug 30, 2024
599a5f3
Issue #1802 Remove unused RuntimeConfig feature
poodlewars Aug 30, 2024
8c86fc1
Skip a failing Windows test
poodlewars Aug 30, 2024
204f639
Add unimplemented increment counter
poodlewars Aug 28, 2024
4f6c7a3
Enhancement 1792: Use float64 as the result type for all division ope…
alexowens90 Sep 2, 2024
6c71527
Always use strict weak ordering in std::sort comparators (#1747)
ianthomas23 Sep 2, 2024
6430623
Use EncodedFieldImpl.blocks() rather than casting member variable _bl…
ianthomas23 Sep 2, 2024
ce06484
Release GIL on read
willdealtry Jul 2, 2024
fee8156
Refactor to different allocation types
willdealtry Aug 21, 2024
91ce370
More WIP
willdealtry Aug 21, 2024
4a8f670
Refactor segment construction
willdealtry Sep 2, 2024
3eec5dd
Address code review comments
willdealtry Sep 2, 2024
7378e9f
Fix regression in bool pickling (#1810)
vasil-pashov Sep 4, 2024
1368b2c
Improve full build time by 30% by adding PCH (#1764)
vasil-pashov Sep 4, 2024
62a6830
Add API to remove Prometheus metrics (#1806)
poodlewars Sep 4, 2024
4eafe7b
Enhancement 1750: lazy dataframe implementation (#1703)
alexowens90 Sep 5, 2024
96bc84c
Fix symbol list bug on compaction (#1324)
IvoDD Aug 26, 2024
fe0516c
Allow metrics to be registered twice
poodlewars Sep 5, 2024
ea291c2
Docs 1809: lazy dataframe documentation and example notebook (#1815)
alexowens90 Sep 9, 2024
b432508
Bugfix 1818: Fix QueryBuilder equality checks (#1819)
alexowens90 Sep 9, 2024
8a8d293
Support reading from prefixes that include a dot
Sep 9, 2024
3eb9303
Fix fixture scope to avoid interfering with other tests
Sep 9, 2024
861de3a
Remove s3_api_ reset call
Sep 10, 2024
0d73959
Add failing test for batch_restore_version non-increasing version
IvoDD Sep 10, 2024
e9bc9aa
Fixes batch_restore_version not increasing version_id
IvoDD Sep 10, 2024
8d19d94
Fix compilation errors for clang 18 (#1813)
vasil-pashov Sep 13, 2024
8a8326d
Add StagedDataFinalizeMethod to docs (#1744)
jamesmunro Sep 16, 2024
2c1dc6a
Add demo video link to README.md (#1837)
jamesmunro Sep 23, 2024
2c84298
More sort and finalize fixes (#1799)
vasil-pashov Sep 24, 2024
7ce7e5a
Fix conda build
IvoDD Sep 26, 2024
2c82db6
Fix row slicing with sort_and_finalize throw on column slicing when t…
vasil-pashov Sep 27, 2024
a2aaee2
Don't override the https value for backwards compatibility (#1840)
G-D-Petrov Sep 30, 2024
238cf81
Support the ignore missing key read option in recurse_index_keys
poodlewars Sep 20, 2024
8a694d1
Add flag to clear staged data on failure to both finalize functions (…
vasil-pashov Oct 1, 2024
42a448f
Change test for library emptyness to allow specifying key types to ex…
poodlewars Sep 30, 2024
1c72b73
Put version_store.empty() back - for backwards compat
poodlewars Sep 30, 2024
33e908b
Ignore symbol list in version_store.empty() - for backwards compat
poodlewars Oct 1, 2024
d3c6a62
Fixing clean builds (#1868)
vasil-pashov Oct 4, 2024
1af59d9
Extend LMDB instance lifetime (#1879)
vasil-pashov Oct 10, 2024
f6fd0d6
Allow customizing the most useful AWS SDK settings.
Oct 6, 2024
677467e
Make it possible to override the conf with the env var
Oct 7, 2024
00c413f
Refactor 1833:use entt as ecs (#1834)
alexowens90 Oct 11, 2024
aa621ab
Bugfix arcticdb-man 96: Improve error message with pd.Timedelta colum…
alexowens90 Oct 11, 2024
fe99a12
Bugfix 1865: Make SymbolDescription return type match type hints (#1877)
alexowens90 Oct 11, 2024
9945220
Bugfix 1652: Improve error message when object dtype columns contains…
alexowens90 Oct 11, 2024
2c8d74b
Bugfix 1841: Correctly roundtrip None and empty string pd.Series name…
alexowens90 Oct 11, 2024
3369cac
Enhancement 1721: arbitrary clause ordering (#1860)
alexowens90 Oct 11, 2024
d11f27d
Introducing compatibility tests
IvoDD Sep 30, 2024
de75434
Make `modify_library_option` backwards compatible
IvoDD Oct 4, 2024
387d27c
Inspect unaltered library config
IvoDD Oct 10, 2024
6fa8abf
Mongo destruction segfault fix
IvoDD Oct 11, 2024
e5ae90f
Use ubuntu-22.04
IvoDD Oct 14, 2024
8fc3938
Fix lib_tool.read_to_keys
IvoDD Sep 12, 2024
77370cd
Skip lmdb compat tests for versions <= 4.5.0
IvoDD Oct 15, 2024
d93ef27
Use relative paths for compat tests
IvoDD Oct 16, 2024
7332fc9
Bugfix 1830: fix resampling with multiindex (#1873)
alexowens90 Oct 16, 2024
5834992
ci: Use mamba 2.0 via `mamba-org/[email protected]` (#1855)
jjerphan Oct 16, 2024
5d98af6
Remove unneeded step (#1929)
G-D-Petrov Oct 17, 2024
db92f2c
Adds a type change utility for append data keys in lib_tool
IvoDD Sep 12, 2024
b6bc28a
fix introduction help (#1933)
grusev Oct 23, 2024
181a61d
Rewrite the mkdocs template so the links point to the correct version…
G-D-Petrov Oct 24, 2024
36ee362
Reduce logo size
jamesmunro Oct 24, 2024
843d4bf
add pyquantnews video thumbnail
jamesmunro Oct 24, 2024
c16a0fd
Replace thumbnail with repo image
jamesmunro Oct 24, 2024
45975e7
Change the pull_request trigger to pull_request_target (#1947)
maxim-morozov Oct 25, 2024
bbb827f
Revert pull_request_target back to pull_request (#1949)
maxim-morozov Oct 25, 2024
35a1e07
Revert "Bugfix 1841: Correctly roundtrip None and empty string pd.Ser…
alexowens90 Oct 28, 2024
b027d52
Fix typo in docs flow (#1952)
G-D-Petrov Oct 28, 2024
b71fc34
Fix entt dep management for conda builds (#1955)
G-D-Petrov Oct 29, 2024
0fb973d
Sort segment by multiple columns
willdealtry Sep 30, 2024
a303dc8
Fix tests
willdealtry Oct 11, 2024
f620053
Log warning on failed normalization
willdealtry Oct 17, 2024
17aa011
Run the pull request on the PR sha with the pull_request_target event…
maxim-morozov Oct 29, 2024
0862b19
Make library manager an LRU cache
willdealtry Oct 14, 2024
d068c21
Correct header for shared_mutex
willdealtry Oct 17, 2024
5239567
Address code review comments
willdealtry Oct 28, 2024
385d78c
Fix the timers macro, configure it with a cmake option
Oct 14, 2024
7db2720
Small fixes for mac arm compilation and fix some tests (#1905)
maxim-morozov Oct 29, 2024
271aa95
Make snapshot names in run_scenario more unique (#1961)
G-D-Petrov Oct 30, 2024
25a99e4
Fix Mac ARM Build (#1960)
G-D-Petrov Oct 30, 2024
2a37ccf
Commit version placeholder change (#1966)
G-D-Petrov Oct 31, 2024
9bf31bf
Make sure that the git user is configured
G-D-Petrov Oct 31, 2024
d6e6eb2
Update BSL table for 5.0.0 release (#1967)
G-D-Petrov Oct 31, 2024
3c5da61
Update docs for the update function (#1977)
vasil-pashov Nov 5, 2024
b06d618
Fix delete snapshot so that it doesn't orphan data keys or delete the…
vasil-pashov Nov 5, 2024
35e5edf
Tests delete snapshot (#1951)
grusev Nov 5, 2024
be878cc
Bugfix 1970: Give helpful error message if and/or/not operators are p…
alexowens90 Nov 5, 2024
18d7b21
Bugfix 127: Improve error message when recursively normalized metastr…
alexowens90 Nov 5, 2024
908a753
fix publish result curl fail
phoebusm Nov 1, 2024
72c5cc4
Bugfix/1841/maintain empty series names (#1983)
alexowens90 Nov 5, 2024
a9d35a5
Add custom overlay fixing clang builds for bitmagic (#1975)
vasil-pashov Nov 5, 2024
018959c
Skip azure compat tests because of #1979
IvoDD Nov 5, 2024
f10479e
Use `shell=True` for linux compat tests to fix feedstock runs
IvoDD Nov 5, 2024
998d30d
Skip mongo tests entirely
IvoDD Nov 5, 2024
5605163
Re comments on #1979 we skip the lmdb compat tests as well
IvoDD Nov 5, 2024
009a7d6
Bugfix/1937/do not write append ref keys when staging incompletes (#1…
alexowens90 Nov 6, 2024
e74a03c
Bugfix 1306: Add nonreg test for update failing with 1-nanosecond dif…
alexowens90 Nov 6, 2024
a6c0002
Enhancement 1895: Fully parallelise processing in read_batch (#1950)
alexowens90 Nov 7, 2024
dd84ba9
Test meaningful error message when reading incompletes from non-exist…
alexowens90 Nov 7, 2024
67a58c3
Bugfix 1655: Disallow / character in mongo lib names (#1992)
alexowens90 Nov 8, 2024
a4f120b
Add coverity CI step (#1919)
vasil-pashov Nov 9, 2024
ad97c06
Get more disk space
phoebusm Nov 11, 2024
7c0c02b
Make static analysis a cron job (#2003)
vasil-pashov Nov 15, 2024
3023fd5
new bi sample benchmark tests
Nov 12, 2024
600bd5a
update comment
Nov 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
26 changes: 26 additions & 0 deletions .git-blame-ignore-revs
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# .git-blame-ignore-revs

# Build time improvements
9d2c8ef41589a4e8635b972911d25c0125dc3728

# Moving implementation from column.hpp to column.cpp
# Reordering/removing some inclusions
801cf4b6f0f9ec0a997311fbdc14639537d3bbb6

# Moving implementation from data_error.hpp to data_error.cpp
9e10e4b129eadba325db50ec1c683f3aeca1d6ea

# maint: Remove headers in types.hpp
f32e342aeaf598e36a3ef20c5e0364a9b8956975

# maint: Move implementations in memory_segment_impl.hpp to `memory_segment_impl.cpp`
8cf2a80d7c2f8539718cad038128f663a8e96810

# maint: Remove unneeded includes
fbc2a76bdbf56c0181b38caa76863b1e4c5e5bec

# build: Remove some adherence to folly
44aaab5493b8132453591b653ab07f432f88b557

# Clean and reorganize `OffsetString` and `StringPool`
06a252f99f9e0c1cc19dd95c61dd239ddf68845c
2 changes: 2 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/cpp/ @alexowens90 @willdealtry @poodlewars
/python/ @alexowens90 @willdealtry @poodlewars
88 changes: 88 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
name: Bug Report
description: Create a report to help us reproduce and correct the bug
labels: ['bug']

body:
- type: markdown
attributes:
value: >
#### Before submitting a bug, please make sure the issue hasn't been already
addressed by searching through [the past issues](https://github.com/man-group/ArcticDB/issues).

- type: textarea
attributes:
label: Describe the bug
description: >
A clear and concise description of what the bug is.

- type: textarea
attributes:
label: Steps/Code to Reproduce
description: |
Please add a [minimal code example](https://matthewrocklin.com/minimal-bug-reports) that can reproduce the error
and the output/exception/stack trace if possible.

Be as succinct as possible and avoid external data files. Instead you can generate synthetic data using `numpy.random`.
If your bug report appears to be dependent on external data files and cannot be reproduced without them, please describe your data
files in as much detail as you can.

If the code is too long, feel free to put it in a [public gist](https://gist.github.com) and link it in the issue.

We acknowledge that this requires some effort but it really helps us locate the problem and we thank you for helping us out ❤️
placeholder: |
```python
from arcticdb import Arctic

Arctic()
```

Output:
```python-traceback
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[4], line 1
----> 1 Arctic()
TypeError: Arctic.__init__() missing 1 required positional argument: 'uri'
```
validations:
required: true

- type: textarea
attributes:
label: Expected Results
placeholder: >
Example: No error is thrown.
validations:
required: true

- type: textarea
attributes:
label: OS, Python Version and ArcticDB Version
description: |
Run this command and paste the result bellow:
```bash
python3 -c "import sys; print('Python:', sys.version); import platform; print('OS:', platform.platform()); import arcticdb; print('ArcticDB:', arcticdb.__version__);"
```
placeholder: >
Python: 3.11.3 | packaged by conda-forge | (main, Apr 6 2023, 08:57:19) [GCC 11.3.0]
OS: Linux-6.2.15-300.fc38.x86_64-x86_64-with-glibc2.37
ArcticDB: 1.1.0
validations:
required: true

- type: input
attributes:
label: Backend storage used
placeholder: >
AWS S3, GCP S3, LMDB, etc.

- type: textarea
attributes:
label: Additional Context
description: |
Add any other context about the problem here.

- type: markdown
attributes:
value: >
Thank you for contributing to ArcticDB!
20 changes: 20 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: enhancement
assignees: ''

---

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context or screenshots about the feature request here.
26 changes: 26 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#### Reference Issues/PRs
<!--Example: Fixes #1234. See also #3456.-->

#### What does this implement or fix?

#### Any other comments?

#### Checklist

<details>
<summary>
Checklist for code changes...
</summary>

- [ ] Have you updated the relevant docstrings, documentation and copyright notice?
- [ ] Is this contribution tested against [all ArcticDB's features](../docs/mkdocs/docs/technical/contributing.md)?
- [ ] Do all exceptions introduced raise appropriate [error messages](https://docs.arcticdb.io/error_messages/)?
- [ ] Are API changes highlighted in the PR description?
- [ ] Is the PR labelled as enhancement or bug so it appears in autogenerated release notes?
</details>

<!--
Thanks for contributing a Pull Request to ArcticDB! Please ensure you have taken a look at:
- ArcticDB's Code of Conduct: https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md
- ArcticDB's Contribution Licensing: https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing
-->
26 changes: 26 additions & 0 deletions .github/actions/run_local_pytest/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: 'Run Local Pytest'
description: 'Installs Python dependencies and runs Pytest against locally built ArcticDB binaries'
inputs:
build_type: {default: 'release', type: string, description: Specifies whether the build that we are testing is release or debug}
threads: {default: '1', type: string, description: Specifies the number of threads that will be used for tests }
fast_tests_only: {default: '0', type: string, description: Specifies whether we want to run only the fast tests or not }
other_params: {default: '', type: string, description: Other params that should be passed to the python command }
runs:
using: "composite"
steps:
- name: Run local pytests
shell: bash
run: |
yum install nodejs npm -y
npm config set strict-ssl false
npm install -g azurite

cd python
ln -s ../cpp/out/linux-${{ inputs.build_type }}-build/arcticdb/arcticdb_ext.cpython-36m-x86_64-linux-gnu.so
export ARCTICDB_RAND_SEED=$RANDOM
python ${{inputs.other_params}} -m pytest --timeout=3600 -n ${{ inputs.threads }} tests
env:
TEST_OUTPUT_DIR: ${{ runner.temp }}
ARCTICDB_FAST_TESTS_ONLY: ${{ inputs.fast_tests_only }}
# Use the Mongo created in the service container above to test against
CI_MONGO_HOST: mongodb
34 changes: 34 additions & 0 deletions .github/actions/set_persistent_storage_env_vars/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: 'Set Persistent storages env variables'
description: 'Set the necessary variables for Persistent storage tests'
inputs:
bucket: {default: 'arcticdb-ci-test-bucket-02', type: string, description: The name of the S3 bucket that we will test against}
endpoint: {default: 'https://s3.eu-west-1.amazonaws.com', type: string, description: The address of the S3 endpoint}
region: {default: 'eu-west-1', type: string, description: The S3 region of the bucket}
aws_access_key: {required: true, type: string, description: The value for the AWS Access key}
aws_secret_key: {required: true, type: string, description: The value for the AWS Secret key}
strategy_branch: {default: 'ignore', type: string, description: a unique combination of the parameters for the given job strategy branch, e.g. linux_cp36}
runs:
using: "composite"
steps:
- name: Set real storage variables
shell: bash

run: |
# Common
echo "ARCTICDB_PERSISTENT_STORAGE_TESTS=1" >> $GITHUB_ENV
echo "ARCTICDB_PERSISTENT_STORAGE_UNIQUE_ID=${{ github.ref_name }}_${{ github.run_id }}" >> $GITHUB_ENV
echo "ARCTICDB_PERSISTENT_STORAGE_STRATEGY_BRANCH=${{ inputs.strategy_branch }}" >> $GITHUB_ENV
# This is the top level path for all test, this is where to write data that should be shared between jobs (e.g. seed job)
echo "ARCTICDB_PERSISTENT_STORAGE_SHARED_PATH_PREFIX=ci_tests/${{ github.ref_name }}_${{ github.run_id }}" >> $GITHUB_ENV
# This is a path that should be used for specific job and its tests to avoid cross contamination and race conditions
echo "ARCTICDB_PERSISTENT_STORAGE_UNIQUE_PATH_PREFIX=ci_tests/${{ github.ref_name }}_${{ github.run_id }}_${{ inputs.strategy_branch }}" >> $GITHUB_ENV
# S3 Specific
echo "ARCTICDB_REAL_S3_BUCKET=${{ inputs.bucket }}" >> $GITHUB_ENV
echo "ARCTICDB_REAL_S3_ENDPOINT=${{ inputs.endpoint }}" >> $GITHUB_ENV
echo "ARCTICDB_REAL_S3_REGION=${{ inputs.region }}" >> $GITHUB_ENV
echo "ARCTICDB_REAL_S3_CLEAR=1" >> $GITHUB_ENV
echo "ARCTICDB_REAL_S3_ACCESS_KEY=${{ inputs.aws_access_key }}" >> $GITHUB_ENV
echo "ARCTICDB_REAL_S3_SECRET_KEY=${{ inputs.aws_secret_key }}" >> $GITHUB_ENV
# Enable all debug logs
# echo "ARCTICDB_all_loglevel=debug" >> $GITHUB_ENV
# echo "ARCTICDB_AWS_LogLevel_int=6" >> $GITHUB_ENV
Loading
Loading