Set fields from value templates when using an "include" index

[AlexStansfield]

I'm trying to design an index for one of my tables and I want it include only a small section of attributes.

The PK and SK are both value templates that include some of the attributes from the model.

I was wondering if when using this if onetable will automatically extract those attributes and set the fields on the model, or if I still need to add those attributes to my includes list in order to be able to access them.

A "keys only" index and follow is unfortunately not an option as the use case for this is to fetch a few thousand records at a time. And I've taken a look at the new encode option which is nice, but I was hoping to avoid that as I've not really had a chance to try it out and wouldn't want this table to be my first time using it.

[mobsense]

If you want a GSI to have a subset of attributes from the table, you need to provide a list in the schema via the project property. You need to specify all attributes including those referenced by value templates.

[AlexStansfield]

Just to confirm, say I had this model:

    User: {
      pk: {
        type: String,
        value: 'Account#${accountId}',
      },
      sk: {
        type: String,
        value: 'User#${id}',
      },
      accountId: {
        type: String,
        required: true,
      },
      id: {
        type: String,
        generate: 'uuid',
      },
      name: {
        type: String,
        required: true,
      },
      email: {
        type: String,
        required: true,
      },
   }

If I wanted the id and accountId from a GSI I would have to configure the index in dynamodb to include both those attributes as OneTable wont extract them from the pk and sk attributes? Unless I use the encode feature.

[mobsense]

I'm not sure exactly what you mean.

Typically you define a separate PK and SK for the GSI. Say: gs1pk and gs2sk. If these include the accountId and id, then you would use:

indexes: {
      gs1: {hash: 'gs1pk', sort: 'gs1sk', project: ['gs1pk', 'gs1sk', 'accountId', 'id']},
}

[AlexStansfield]

@mobsense

thanks, the example I gave was because an index will always return the table's partition and sort keys, but yes it could be that the GSI hash and sort keys could be value templates too.

I was just trying to work out if OneTable had a way of automatically getting attribute values from available attributes that use value templates when the exact one is not projected. It seems the answer is no and the only way to do this would be with the encode option so looking into that now.

[mobsense]

Can you not just project those attributes? Encode is really just an optimization to not duplicate attributes.

[AlexStansfield]

Yes we can, but the table in question will have over 100m records just once we migrate the existing data to the new structure, so I was looking for ways to minimise the sizes of indexes.