1.38.0

CODE_COLOR: CODE_YELLOW_MAINNET
RELEASE_VERSION: 1.38.0
PROTOCOL_UPGRADE: TRUE
DATABASE_UPGRADE: FALSE
SECURITY_UPGRADE: FALSE

Protocol Upgrade Voting

This release contains a protocol upgrade. Voting for upgrading to protocol version 65 will start on Wednesday 2024-03-20 14:00:00 UTC. Depending on epoch durations, resharding is expected to happen around Wednesday 2024-03-20 18:00 UTC +/-4h, and protocol adoption is expected to happen around Thursday 2024-03-21 10:00 UTC +/-4h. Resharding and adoption times are only estimates, since many factors influence them.

Managing Resharding

This protocol upgrade includes resharding of shard 2. Which means that after protocol upgrade we will have 6 shards. New shard split is defined by these border accounts vec!["aurora", "aurora-0", "game.hot.tg", "kkuuue2akv_1630967379.near", "tge-lockup.sweat"]

Important points:

• Resharding will happen in the epoch RIGHT AFTER protocol voting epoch (and RIGHT BEFORE actual protocol upgrade).
• If your node fails to reshard before protocol upgrade, it will not be able to continue being a part of the chain. If that happens, you should restart from a Pagoda provided backup.
• Resharding creates a database snapshot that is deleted right after resharding is finished. This snapshot will not take extra space at first, but you may notice a progressive increase of data folder size up to around 50GB higher than before. There should no longer be an issue with the data directory increasing to an unmanageable size.
• To monitor resharding you can use metrics near_resharding_status, near_resharding_batch_size, and near_resharding_batch_prepare_time_bucket. You can read more here https://github.com/near/nearcore/blob/master/docs/architecture/how/resharding.md#monitoring.
• If you observe problems with block production or resharding performance, you can adjust resharding throttling configuration. Read more here https://github.com/near/nearcore/blob/master/docs/architecture/how/resharding.md#monitoring.
• RAM usage during resharding is around 8-10GB.

Full resharding doc can be found here https://github.com/near/nearcore/blob/master/docs/architecture/how/resharding.md.

Protocol Changes

• Resharding v3 - resharding of shard 2 implemented by #10725

Known issues

• This release removes state snapshot compaction configuration options, as they are no longer needed. Your node may emit a warning that those configuration options are not recognized, but this is not an issue.

1.37.3

CODE_COLOR: CODE_YELLOW_MAINNET
RELEASE_VERSION: 1.37.3
PROTOCOL_UPGRADE: FALSE
DATABASE_UPGRADE: FALSE
SECURITY_UPGRADE: FALSE

This release includes important performance optimisations.
This release does not contain a protocol upgrade.

Fixes

• Return in-memory cache of loaded artifacts #10788
• Support Included transaction status #10796

1.38.0-rc.2

CODE_COLOR: CODE_YELLOW_TESTNET
RELEASE_VERSION: 1.38.0-rc.2
PROTOCOL_UPGRADE: TRUE
DATABASE_UPGRADE: FALSE
SECURITY_UPGRADE: FALSE

Protocol Upgrade Voting

This release contains a protocol upgrade. Voting for upgrading to protocol version 65 will start on Tuesday 2024-03-12 18:00:00 UTC.

Fixes

This release applies the fixes included in 1.37.2 on top of 1.38.0-rc.1, as well as a minor metrics fix

1.37.2

Fixes

Fix an issue where flat storage is not properly updated during resharding.

1.38.0-rc.1

CODE_COLOR: CODE_YELLOW_TESTNET
RELEASE_VERSION: 1.38.0-rc.1
PROTOCOL_UPGRADE: TRUE
DATABASE_UPGRADE: FALSE
SECURITY_UPGRADE: FALSE

Protocol Upgrade Voting

This release contains a protocol upgrade. Voting for upgrading to protocol version 65 will start on Tuesday 2024-03-12 18:00:00 UTC.

Resharding

This release schedules another resharding operation that will split shard 2 to address congestion issues that have been occurring on mainnet. This contains no functional code changes on top of 1.37.1 aside from scheduling this new resharding operation. After the new protocol version is voted on, nodes will begin another reshard of shard 2, which is expected to take a couple hours. After the new protocol version is adopted, there will be 6 shards.

1.37.1

CODE_COLOR: CODE_GREEN_MAINNET
RELEASE_VERSION: 1.37.1
PROTOCOL_UPGRADE: FALSE
DATABASE_UPGRADE: FALSE
SECURITY_UPGRADE: FALSE

This release does not contain protocol upgrade, and it is completely optional.

Fixes

This release includes two fixes for common problems:

• 2567e70 addresses OOM crashes
• 915aea7 addresses stack overflow crashes

1.37.0

CODE_COLOR: CODE_YELLOW_MAINNET
RELEASE_VERSION: 1.37.0
PROTOCOL_UPGRADE: TRUE
DATABASE_UPGRADE: FALSE
SECURITY_UPGRADE: FALSE

Protocol Upgrade Voting

This release contains a protocol upgrade. Voting for upgrading to protocol version 64 will start on Monday 2024-03-11 18:00:00 UTC.

Managing Resharding

This protocol upgrade includes resharding of shard 3. Which means that after protocol upgrade we will have 5 shards. New shard split is defined by these border accounts vec!["aurora", "aurora-0", "kkuuue2akv_1630967379.near", "tge-lockup.sweat"]

Important points:

• Resharding will happen in the epoch RIGHT AFTER protocol voting epoch (and RIGHT BEFORE actual protocol upgrade).
• If your node fails to reshard before protocol upgrade, it will not be able to continue being a part of the chain. If that happens, you should restart from a Pagoda provided backup.
• Resharding creates a database snapshot that is deleted right after resharding is finished.During resharding, though, you may notice progressive increase of data folder size (around 150GB).
• To monitor resharding you can use metrics near_resharding_status, near_resharding_batch_size, and near_resharding_batch_prepare_time_bucket. You can read more here https://github.com/near/nearcore/blob/master/docs/architecture/how/resharding.md#monitoring.
• If you observe problems with block production or resharding performance, you can adjust resharding throttling configuration. Read more here https://github.com/near/nearcore/blob/master/docs/architecture/how/resharding.md#monitoring.
• RAM usage during resharding is around 8-10GB.

Full resharding doc can be found here https://github.com/near/nearcore/blob/master/docs/architecture/how/resharding.md.

Protocol Changes

• Resharding v2 - new implementation for resharding and a new shard layout for production networks. #10303, NEP-0508
• Restrict the creation of non-implicit top-level accounts that are longer than 32 bytes. Only the registrar account can create them. #9589
• Adjust the number of block producers and chunk producers on testnet to facilitate testing of chunk-only producers #9563

Non-protocol Changes

• Add prometheus metrics for the internal state of the doomslug. #9458
• Fix EXPERIMENTAL_protocol_config to apply overrides from EpochConfig. #9692
• Add config option tx_routing_height_horizon to configure how many chunk producers are notified about the tx. #10251

Known issues

• [This item includes the mitigations] State snapshot compaction may cause stack overflow. To avoid this issue, make sure that fields store.state_snapshot_config.compaction_enabled and store.state_snapshot_compaction_enabled are set to false in config.
• [This one does not] A bug in network tracing spans management may lead to stack overflow. This issue will be fixed in 1.37.1 release that we plan to make after the protocol upgrade.

1.37.0-rc.3

Notice

This release contains important changes for resharding. It fixes a bug around shard management that can lead to a node crash during the transition into the first epoch with a new shard layout.
Voting for the protocol version 64 will happen on Tuesday 2024-02-06 12:00:00 UTC and will trigger resharding in the next epoch. By our estimations, resharding will start on Tuesday 2024-02-06 20:00:00 UTC, and first epoch with 5 shards will start on Wednesday 2024-02-07 12:00:00 UTC.
Please, update as soon as possible. If your node fails to perform resharding before the chain switches to 5 shards, you will have to wait for Pagoda to upload a testnet DB snapshot and start from it.

CODE_COLOR: CODE_RED_TESTNET
RELEASE_VERSION: 1.37.0-rc.3
PROTOCOL_UPGRADE: FALSE
DATABASE_UPGRADE: FALSE
SECURITY_UPGRADE: TRUE

1.36.5

Notice

This release of nearcore is a security release (CODE_RED_MAINNET). It fixes a bug around shard management that, if exploited, can lead to a node crash.

Apart from the bugfix mentioned above, the release also includes a few other minor functional fixes.

All node operators must upgrade to 1.36.5 immediately.

CODE_COLOR: CODE_RED_MAINNET
RELEASE_VERSION: 1.36.5
PROTOCOL_UPGRADE: FALSE
DATABASE_UPGRADE: FALSE
SECURITY_UPGRADE: TRUE

1.37.0-rc.2

CODE_COLOR: CODE_GREEN_TESTNET
RELEASE_VERSION: 1.37.0-rc.2
PROTOCOL_UPGRADE: FALSE
DATABASE_UPGRADE: FALSE
SECURITY_UPGRADE: FALSE

Note (Upgraded Protocol Upgrade Voting Date)

Testnet is still running protocol version 63. Voting for upgrading to protocol version 64 will now start on Monday 2024-02-05 15:00:00 UTC (one week later than in 1.37.0-rc.1).
Pagoda will update their testnet validators ASAP. As they hold more than 20% of the stake, this is enough to postpone the voting.
If you run your own testnet validator, you are not required to update. The voting will be finished later either way.

Non-protocol Changes

• Split storage optimisation (#10437)