RFC: Exclude signature from objectID computation

[damip]

RFC: Exclude signature from objectID computation

Current state

Signatures are taken into account when hashing blocks, ops, etc... to generate object IDs

• we need to compute the hash of the signed contents to sign them, then another hash of the signed contents + signature in order to get the hash of the object

• we absolutely need to make sure that signatures are not malleable, otherwise attackers could "clone" existing transactions by changing their signature in order to execute them multiple times under a new hash. I think we are currently INSECURE against this.

  • https://en.bitcoin.it/wiki/BIP_0340#Motivation
  • https://en.bitcoin.it/wiki/BIP_0062
  • https://en.bitcoin.it/wiki/BIP_0146

• if the signatures are non-deterministic, how do we define genesis block hashes ?

  • we could make a special case for genesis blocks to force constant block IDs that are not recomputed

Proposal

This RFC proposes removing signatures from the object ID computation process.

• we only compute the hash of the signed contents for signing, and this hash is also the hash of the object (object ID) => more efficient (1 less hash computation, and less data hashed)
• problem: the same object (by ID) could be signed by multiple different signatures/private keys.
  • always including the signer's pubkey (or account number in RFC: Native DNS #2646) in the object's signed contents solves the problem. Note that we are already doing that for all current objects.
• signature malleability is not a problem anymore, so we can open up to more signature algorithms
• warning: signatures stop being consensual
  • Apart from verification, other read operations on signatures must not be used directly in any deterministic process
  • Smart contracts should not be given read access to block/op signatures
• performance impact: the signatures of ops that are in blocks need to be re verified because they might differ from the ones seen on naked op propagation
  • note that if we switch to including only the list of op IDs in blocks, and asking for missing ops through the normal naked op ask process, this perf problem is solved as verif is done only once on naked propagation. This is the topic of the RFC RFC: Do not propagate operations inside blocks #2706 .
• if the signatures are non-deterministic: no problem, no ambiguity with genesis block hashes

The main long term advantage of this proposal is that it decouples the architecture from the choice of signature scheme (no assumptions on malleability, determinism etc...), thus allowing us to update the signature algo more freely in the future. Less coupling = more maintainable code.

Short term proposal (to be discussed):

• remove signatures from the object ID computation process
• ensure that the signer is always identified in all signed data (pubkey or account number) => it is the case currently
• ensure that the signatures do not participate in any consensus process (except through pure signature verification), especially do not allow SCs to read block/op signatures
• (re-)verify signatures of ops that are in blocks, accepting a small performance hit until we remove redundant op propagation by not including ops in blocks but only their hashes

[Eitu33]

I agree with the given proposition but have a few questions:

performance impact: the signatures of ops that are in blocks need to be re verified because they might differ from the ones seen on naked op propagation

If we decide to do this we would want to release the update about blocks containing only hashes in the same testnet right? And that might take a little while because we still need to specify how we'd want to do it. Or is it okay to launch a testnet with just that part?

ensure that the signer is always identified in all signed data (pubkey or account number) => it is the case currently

When you say that it is currently the case you're talking about deducing it from the signature right? We don't actually send a pubkey for every signed object do we?

(re-)verify signatures of ops that are in blocks until we unify op propagation

What do you mean by until we unify op propagation? Isn't it until block content update?

Also isn't having deterministic signatures for the genesis blocks only not an option? Do you not like it because you feel like it is more of an hotfix than an actual solution?

[damip]

I agree with the given proposition but have a few questions:

performance impact: the signatures of ops that are in blocks need to be re verified because they might differ from the ones seen on naked op propagation

If we decide to do this we would want to release the update about blocks containing only hashes in the same testnet right? And that might take a little while because we still need to specify how we'd want to do it. Or is it okay to launch a testnet with just that part?

It's ok to release with just the first part and a temporary performance regression.

ensure that the signer is always identified in all signed data (pubkey or account number) => it is the case currently

When you say that it is currently the case you're talking about deducing it from the signature right? We don't actually send a pubkey for every signed object do we?

No, I mean that the public key of the sender must always be included in the signed data, otherwise someone else can reclaim the object by signing it without changing its object hash. Currently, all signed objects already contain the sender pubkey so there is nothing to change there.

(re-)verify signatures of ops that are in blocks until we unify op propagation

What do you mean by until we unify op propagation? Isn't it until block content update?

I'm talking about the paragraph

note that if we switch to including only the list of op IDs in blocks, and asking for missing ops through the normal naked op ask process, this perf problem is solved as verif is done only once on naked propagation. This will be the topic of another RFC.

[Eitu33]

Ok cool, and what about:

Also isn't having deterministic signatures for the genesis blocks only not an option? Do you not like it because you feel like it is more of an hotfix than an actual solution?

[damip]

That's what I meant with "we could make a special case for genesis blocks to force constant block IDs that are not recomputed": technically we could hardcode the blockID of genesis blocks and make a special case so that no signature is needed at all for them (or filled with zeros).

The proposal solves that problem more elegantly but also (and more importantly) it improves security against malleability etc... and decouples the choice of signature algorithm from the protocol, allowing us to more freely choose/update the signature scheme as the architecture stops making extra assumptions on the signature scheme

[Eitu33]

Not sure about what approach would be the best here, I don't feel like the proposition is that much longer to implement than the hotfix so I guess that's what I would go for.

[AurelienFT]

I agree with this proposal I'm currently working on changing the operation structure to always have those fields so I think we should synchronize on that

[sebastien-forestier]

note that if we switch to including only the list of op IDs in blocks, and asking for missing ops through the normal naked op ask process, this perf problem is solved as verif is done only once on naked propagation.

• If we switch to only opids in the block: then an operation in a block can be valid or invalid depending on the signature we received
• Otherwise, it's a x2 increase in cpu usage

[damip]

If we check a naked op, see it has a valid sig, save it in cache, then assume the sig is valid when we see the op again in a block, then we have a problem: maybe the sig of the op in the block is invalid.

2 solutions:
1 - check the naked op sig AND the in-block op sig. This could be optimized by caching the sig as well and just compare (and re-verify only on sig mismatch), in order to avoid doubling the verif load in the nominal case.
2 - implement the linked RFC to separate block and op propagation, that way we remove the problem completely AND avoid any performance hit. If we receive a naked op with an invalid sig, we will just ignore + drop + ban, nothing is cached, so the block won't be invalid, it will just be missing a dependency (just like if we announced an invalidly signed header, and then used it as a parent in a followup block).

[sebastien-forestier]

I think you suppose that a block with an op that has an invalid signature is an invalid block, is it the case ?

We discussed it would be better if block-header valid implies block valid:

• we have the possibility to check op sigs lazily when blocks become final (saving cpu when there are attacks and stale blocks) while still participating in consensus
• when there are multistaking attacks, it avoids some cases where an invalid block becomes valid

I think we should study this more

[damip]

I think you suppose that a block with an op that has an invalid signature is an invalid block, is it the case ?

We might be mixing things up here. The problem in this RFC is independent from that question.
In the current implem, invalid op implies invalid block until we figure out ledger fusion.

So currently, having a cache marking an op as "valid" because we saw it naked with a valid sig would cause some invalid blocks to be perceived as valid.
After ledger fusion, the problem will move elsewhere and become worse: some nodes will consider some ops as executed, and some won't, durably and silently.

The solutions I propose above solve the problem independently of that question.

[sebastien-forestier]

I'm talking about the combination of this RFC and #2706

In this context, I think you have to suppose that a sig-invalid op in a block makes that block invalid

[damip]

Here is the workflow:

• a node N announces a block header to us
• ask that node for the block
• the node sends us the list of op hashes
• we ask the sender for the ops we don't already have
• if N doesn't respond quickly or responds with an invalid signature somewhere, ban N, ask for missing ops to someone else

That way the problem won't appear