EXPERIMENTAL: map/list interfaces

[smrz2001]

Very experimental changes for (hopefully) more usable map and list interfaces for IPLD nodes, based on amend.

I wanted to get some early feedback so only the map interface is implemented. The tests in map_test.go demonstrate some ways in which the interface can be used.

cc @RangerMauve @rvagg @aschmahmann @BigLep @warpfork

type Map interface {
	Put(key string, value interface{}) bool
	Get(key string) (value Node, found bool)
	Remove(key string) bool
	Keys() []string

	Container
}

type List interface {
	Get(idx int64) (Node, bool)
	Remove(idx int64)
	Append(values ...interface{})
	Insert(idx int64, values ...interface{})
	Set(idx int64, value interface{})

	Container
}

type Container interface {
	Empty() bool
	Length() int64
	Clear()
	Values() []Node
}

[RangerMauve]

Could you elaborate a bit more on the motivation for introducing these types? Particularly what in the existing codebase is lacking that this addresses.

Also, regarding methods that return arrays, would it be possible to return iterators instead? There are ADLs where a Map's keys might potentially be too large to store in memory, similarly with Lists. I think we should have streaming in consideration from the get go so that we don't run into issues later.

[smrz2001]

Could you elaborate a bit more on the motivation for introducing these types? Particularly what in the existing codebase is lacking that this addresses.

Yes, sorry, I should have added more context. This is an attempt to:

• Establish a more user-friendly interface for recursive types. A big pain point for me (and others) has been the current, inconvenient way to use maps or lists (builders, assemblers, etc.)
• Allow modification transparently and with all the optimizations amend brings.

Once a reasonable interface is established, it can be extended to other map types/ADLs like HAMT, that are also quite inconvenient to use like normal maps.

Also, regarding methods that return arrays, would it be possible to return iterators instead? There are ADLs where a Map's keys might potentially be too large to store in memory, similarly with Lists. I think we should have streaming in consideration from the get go so that we don't run into issues later.

Yes, makes sense! I wasn't too thrilled about them and was considering just removing them but returning iterators is much better.

Thanks, @RangerMauve!

[RangerMauve]

Does golang have a concept of async iterators? I think Adin brought it up on the IPFS discord. 😁

[smrz2001]

Does golang have a concept of async iterators? I think Adin brought it up on the IPFS discord. 😁

Good question 😊 I've also been thinking about it since I read Adin's comments. I think it's possible to provide thread-safe, concurrent iteration in the current model, with some tweaks.

Can chat a bit about this on the call too, if we have time.

[smrz2001]

I also wanted to pass another thought by you, @RangerMauve.

I can't get the idea out of my head that what amend really is, is an updatable ADL. It takes a source Node, allows the accumulation of updates, then "presents" a new Node to consumers.

I could easily turn Amender.Build into Reify and add Substrate to expose the source Node.

[smrz2001]

Closing in favor of PR against main repo.