Cosmos DB Expansion

[richard-einfinity]

Picking up from a discussion we were having over on the Mobile Apps repository. The OOB EntityTableRepository will work with the basic cosmos DB configuration but falls down a little bit when you start introducing things like partition keys. The default behaviour of EntityFrameworkCore with cosmosdb is that it creates a partition key of __partitionKey if you don't specify one. This limits the amount of Data Stored to 20GB.

When the partition key is specified we need a mechanism for passing it to the repository to the Find and Single LINQ methods in the repository.

We previously discussed mechanisms like packing the keys and passing additional parameters through the query string or headers. The current AccessControlProvider will now work will with filtering the partitioned data, with the release of EF9 the GetDataView expression should be lifted up so that parameters that are partition keys are recognised as such. So no work needs to be done there.

The find method in the OOB EntityTableRepository will fail as it requires that the id and the partition keys are passed through, with the partition key values ordered in the same order as the partition keys are defined on the model.

I think there are 2 potential ways of supporting partition keys.

We could expand the AccessControlProvider to include methods to return the keys to pass to the Repository and perhaps and expression for the lookups. My concern with this approach is polluting the ecosystem with requirements of the cosmosdb implementation. The only question being is if there is composite primary key lookups requirement with the other providers which this will also support.

Or we could look at a Cosmos specific repository which passes an Options object akin to the cosmosdb work in progress on the 8.0 branch of the mobile apps project. My concern with this approach is are we shifting some of the responsibility to the repository which really belongs further up the chain, especially if we're dealing with query strings or headers.

Happy to take a look at this if we can get some consensus on the path forward.

[adrianhall]

I don't like overloading the access control provider, except to limit the data view of things. I'm also noting your note about EF9 and GetDataView(). I need to take a look at this in the new year since I'm going to be tackling .NET 9 at that point.

Question: Is this something that can be generalized within the codebase, or is this so unique to cosmos DB that it always will require custom code? I'm not familiar enough with the concepts involved here to render an opinion.

[richard-einfinity]

Hi @adrianhall

I think the difference between the cosmosdb requirement and the current implementation is the need / ability to pass multiple keys / ids down to the repository.

I guess the only question would be is there a general requirement to support multiple keys for other data stores? For example with a composite key lookup. If not it would be a specific cosmos concern.

EF9 isn't dependent on .net 9 so I'm going to have a go at implementing a cosmosdb friendly EF repository with the current framework. I believe that the current package restriction won't prevent me from benefiting from the enhanced EF9 cosmosdb query translation.

[adrianhall]

My recollection was correct then.

The problem is that so much of the client are server code is centered around a single ID property that it would be hard to split. The basic problem is that /tables/tableName/{id?} doesn’t work when there is more than one thing for the ID - and that’s basic HTTP.

REST and HTTP is not going to change anytime soon.

That leaves us with your previous method of packing the partition key into the ID. It can be automated using two properties, but I can’t see a way of generically handling such a thing.