Updating a nested field #146

fifi98 · 2021-09-30T19:09:01Z

fifi98
Sep 30, 2021

Hello,

I appreciate all the work you put into onetable development! I've been having some trouble executing update command on my model. My data in JSON format looks like this:

 "pk": "story:iVsIUKLk69s3dkwHs-S8z",
 "sk": "story:",
 "location": {},
 "slides": {
  "4SWys5q0BhHOiTxlRbBsK": {
   "id": "4SWys5q0BhHOiTxlRbBsK",
   "mediaType": "image",
   "mediaUri": "qmLisXFMD5z7g-e0fsdzb",
  },
  "FSzd9sjpenCAbZ_klOszZ": {
   "id": "FSzd9sjpenCAbZ_klOszZ",
   "mediaType": "image",
   "mediaUri": "jG45dB31jcDXXTYjRVMGX",
   "processed": true
  }
 }

What I want is to set processed to true on slides[4SWys5q0BhHOiTxlRbBsK].processed. Basically I have to execute the following update statement from aws-sdk using Onetable:

const params = {
  TableName: "table-name-dev",
  Key: {
    pk: "story:iVsIUKLk69s3dkwHs-S8z",
    sk: "story:",
  },
  UpdateExpression: "set slides.#slideId.#key = :r",
  ExpressionAttributeNames: {
    "#slideId": "hvhf1VCCBqsVI3KngYk3N",
    "#key": "processed",
  },
  ExpressionAttributeValues: {
    ":r": true,
  },
  ReturnValues: "ALL_NEW",
};

docClient.update(params, function (error, data) {
  if (err) {
    console.error("Unable to update item. Error:", error);
  } else {
    console.log(data.Attributes);
  }
});

Can you please point me into the right direction how can I execute that update command with Onetable? I know there is this example of updating nested fields:

user = await User.update({email: '[email protected]'}, {set: {'address.zip': '{"98034"}'}})

but I didn't have any luck with doing it like set: { 'slides.4SWys5q0BhHOiTxlRbBsK.processed': true }

(the reason why slides isn't an array but an object is because I wouldn't be able to update a specific field of a specific object inside the array as far as I know, please let me know if I'm wrong)

Thank you!

Answered by mobsense

Sep 30, 2021

Wait on.

I've got your create and schema code wrong. Should be slides just as an object with no nested schema.

export default {
    version: '0.0.1',
    indexes: {
        primary: { hash: 'pk', sort: 'sk' },
    },
    models: {
        User: {
            pk:          { type: String, value: '${_type}#' },
            sk:          { type: String, value: '${_type}#${email}' },
            id:          { type: String, uuid: true },
            email:       { type: String },
            slides:      { type: Object }
            },
        }
    }
}

Then the code should be

    user = await User.create({
        email: '[email protected]',
        slides: {
            '4SWys5q0BhHOiTxlRbBsK': {
…

View full answer

mobsense · 2021-09-30T19:39:30Z

mobsense
Sep 30, 2021
Maintainer

This works for me:

Model:

export default {
    version: '0.0.1',
    indexes: {
        primary: { hash: 'pk', sort: 'sk' },
    },
    models: {
        User: {
            pk:          { type: String, value: '${_type}#' },
            sk:          { type: String, value: '${_type}#${email}' },
            id:          { type: String, uuid: true },
            email:       { type: String },
            slides:      { type: Object, schema: {
                            id:         { type: String },
                            mediaType:  { type: String },
                            mediaUrl:   { type: String },
                            processed:  { type: Boolean },
                        }
            },
        }
    }
}

Code:

    user = await User.create({
        email: '[email protected]',
        slides: {
            id:         '4SWys5q0BhHOiTxlRbBsK',
            mediaType:  'video',
            mediaUrl:   'https://acme.com/',
            processed:  false,
        }
    })

    user = await User.update({email: '[email protected]'}, {set: {'slides.4SWys5q0BhHOiTxlRbBsK.processed': true}})

3 replies

fifi98 Sep 30, 2021
Author

Thanks for the fast answer! Unfortunately I am getting this error with that code when trying to update the user:
TypeError: Cannot read property 'processed' of null

And besides that error, the code you provided would not allow me to store multiple slides under "slides" property.

mobsense Sep 30, 2021
Maintainer

Try with the 1.9 just released. I tested with that.

fifi98 Sep 30, 2021
Author

Unfortunately no luck. I have tested with the newly released 1.9, but processed isn't set to true after .update. Also, is it possible to have the slides property to contain multiple slide objects like this, but not only one:

 "pk": "story:iVsIUKLk69s3dkwHs-S8z",
 "sk": "story:",
 "location": {},
 "slides": {
  "4SWys5q0BhHOiTxlRbBsK": {
   "id": "4SWys5q0BhHOiTxlRbBsK",
   "mediaType": "image",
   "mediaUri": "qmLisXFMD5z7g-e0fsdzb",
  },
  "FSzd9sjpenCAbZ_klOszZ": {
   "id": "FSzd9sjpenCAbZ_klOszZ",
   "mediaType": "image",
   "mediaUri": "jG45dB31jcDXXTYjRVMGX",
   "processed": true
  }
 }

mobsense · 2021-09-30T20:23:22Z

mobsense
Sep 30, 2021
Maintainer

Can you look at test/debug.ts to see if you can see a difference. I tested with jest debug.

Did you get the same error message? What happened?
Can you use {debug: true} in the params and see the actual command generated which will be traced to stdout.

6 replies

mobsense Sep 30, 2021
Maintainer

I need to see the actual command output. Add {debug: true} to the 2nd parameter to update.

mobsense Sep 30, 2021
Maintainer

I mean the actual command issued to dynamodb

fifi98 Sep 30, 2021
Author

await table
    .getModel("Testing")
    .update({ email: "[email protected]" }, { set: { "slides.4SWys5q0BhHOiTxlRbBsK.processed": true }, debug: true });

Did you mean like this? If yes, there wasn't any output.

mobsense Sep 30, 2021
Maintainer

See the README section on the logger.

OneTable can log complete request parameters and responses to assist you in debugging and understanding how your API requests are being translated to DynamoDB.

You can set logger parameter to true for simple logging to the console. Alternatively, the logger may be set to logging callback that will be invoked as required to log data. The logger function has the signature:

const table = new Table({
    ...
    logger: (level, message, context) => {
        if (level == 'trace' || level == 'data') return
        console.log(`${new Date().toLocaleString()}: ${level}: ${message}`)
        console.log(JSON.stringify(context, null, 4) + '\n')
    }
})
Where level is set to info, error, warn, exception, trace or data. The trace level is for verbose debugging messages. The data level logs user data retrieved find and get API calls.

The message is a simple String containing a descriptive message. The context is a hash of contextual properties regarding the request, response or error.

If you use {log: true} in the various OneTable Model API options, the more verbose trace and data levels will be changed to info for that call before passing to the logging callback. In this way you can emit trace and data output on a per API basis.

fifi98 Sep 30, 2021
Author

Got it, sorry. Here is the output of the update function:

30/09/2021, 22:55:23: info: OneTable "update" "Testing"
{
    "trace": {
        "model": "Testing",
        "cmd": {
            "ConditionExpression": "(attribute_exists(pk)) and (attribute_exists(sk))",
            "ExpressionAttributeNames": {
                "#_0": "slides",
                "#_1": "4SWys5q0BhHOiTxlRbBsK",
                "#_2": "email",
                "#_3": "updatedAt"
            },
            "ExpressionAttributeValues": {
                ":_0": {
                    "BOOL": true
                },
                ":_1": {
                    "S": "[email protected]"
                },
                ":_2": {
                    "N": "1633035323030"
                }
            },
            "TableName": "restory-dev",
            "ReturnValues": "ALL_NEW",
            "UpdateExpression": "set #_0.#_1 = :_0, #_2 = :_1, #_3 = :_2",
            "Key": {
                "pk": {
                    "S": "Testing#"
                },
                "sk": {
                    "S": "Testing#[email protected]"
                }
            }
        },
        "op": "update",
        "properties": {
            "pk": "Testing#",
            "sk": "Testing#[email protected]",
            "email": "[email protected]",
            "_type": "Testing",
            "updatedAt": 1633035323030
        }
    }
}

30/09/2021, 22:55:23: info: OneTable result for "update" "Testing"
{
    "cmd": {
        "ConditionExpression": "(attribute_exists(pk)) and (attribute_exists(sk))",
        "ExpressionAttributeNames": {
            "#_0": "slides",
            "#_1": "4SWys5q0BhHOiTxlRbBsK",
            "#_2": "email",
            "#_3": "updatedAt"
        },
        "ExpressionAttributeValues": {
            ":_0": {
                "BOOL": true
            },
            ":_1": {
                "S": "[email protected]"
            },
            ":_2": {
                "N": "1633035323030"
            }
        },
        "TableName": "restory-dev",
        "ReturnValues": "ALL_NEW",
        "UpdateExpression": "set #_0.#_1 = :_0, #_2 = :_1, #_3 = :_2",
        "Key": {
            "pk": {
                "S": "Testing#"
            },
            "sk": {
                "S": "Testing#[email protected]"
            }
        }
    },
    "items": [
        {
            "id": "493d93218fe2df508da923d883832739",
            "email": "[email protected]",
            "slides": {
                "4SWys5q0BhHOiTxlRbBsK": true,
                "id": "4SWys5q0BhHOiTxlRbBsK",
                "mediaType": "video",
                "mediaUrl": "https://acme.com/",
                "processed": false
            },
            "createdAt": "2021-09-30T20:55:22.439Z",
            "updatedAt": "2021-09-30T20:55:23.030Z"
        }
    ],
    "op": "update",
    "properties": {
        "pk": "Testing#",
        "sk": "Testing#[email protected]",
        "email": "[email protected]",
        "_type": "Testing",
        "updatedAt": 1633035323030
    },
    "params": {
        "exists": true,
        "parse": true,
        "high": true,
        "set": {
            "slides.4SWys5q0BhHOiTxlRbBsK.processed": true
        },
        "log": true,
        "checked": true
    }
}

mobsense · 2021-09-30T21:30:05Z

mobsense
Sep 30, 2021
Maintainer

Thanks. Got it.

The reason is that your '4SWys5q0BhHOiTxlRbBsK' is not a property defined in the schema. i.e. not named. The current code supports named properties and one last trailing unnamed property.

I'll see if this can be fixed.

0 replies

mobsense · 2021-09-30T21:45:24Z

mobsense
Sep 30, 2021
Maintainer

Wait on.

I've got your create and schema code wrong. Should be slides just as an object with no nested schema.

export default {
    version: '0.0.1',
    indexes: {
        primary: { hash: 'pk', sort: 'sk' },
    },
    models: {
        User: {
            pk:          { type: String, value: '${_type}#' },
            sk:          { type: String, value: '${_type}#${email}' },
            id:          { type: String, uuid: true },
            email:       { type: String },
            slides:      { type: Object }
            },
        }
    }
}

Then the code should be

    user = await User.create({
        email: '[email protected]',
        slides: {
            '4SWys5q0BhHOiTxlRbBsK': {
                mediaType:  'video',
                mediaUrl:   'https://acme.com/',
                processed:  false,
            }
        }
    })

    user = await User.update({email: '[email protected]'}, {log: true, set: {'slides.4SWys5q0BhHOiTxlRbBsK.processed': true}})

Then there is a fix in Expression.js to make it work for mult-level set expressions like yours.

We'll apply this fix:

diff --git a/src/Expression.js b/src/Expression.js
index 0991938..a0b469e 100644
--- a/src/Expression.js
+++ b/src/Expression.js
@@ -384,7 +384,7 @@ export class Expression {
             } else {
                 subscript = ''
             }
-            let field = fields[prop]
+            let field = fields ? fields[prop] : null
             if (field) {
                 target.push(`#_${this.addName(field.attribute[0])}${subscript}`)
                 //  If nested schema, advance to the next level
@@ -394,9 +394,6 @@ export class Expression {
                 target.push(`#_${this.addName(prop)}${subscript}`)
                 fields = null
             }
-            if (fields == null) {
-                break
-            }
         }
         return target.join('.')
     }

1 reply

fifi98 Oct 1, 2021
Author

Awesome, thank you very much! I will try the fix and report back.

onhate · 2022-01-17T19:27:35Z

onhate
Jan 17, 2022

I have the following schema

const PermissionsSchema = {
  fields: {
    pk: { type: String, value: '${typename}#${resource}' },
    sk: { type: String, value: '${typename}#${subject}' },
    typename: { type: String },
    resource: { type: String },
    subject: { type: String },
    policies: { type: Object }
  }
};

and running the following update command, what I really want is that each time I call the set(res, sub, { docs: '1111' }) and later set(res, sub, { forums: '1111' }) I don't want the latest call to overwrite the already existing policies.

async set(resource: string, subject: string, policies: Policies) {
  const set = Object.fromEntries(
    Object.entries(policies).map(([key, value]) => ['policies.' + key, value])
  );
  return await this.model.update({ resource, subject }, { set, exists: null });
}

but this is not working, the response of this is an object without policies field, looks like it is completly ignoring the { set } param

this is the log for the update operation

console.log
   {
     "trace": {
       "model": "Permissions",
       "cmd": {
         "ExpressionAttributeNames": {
           "#_0": "createdAt",
           "#_1": "typename",
           "#_2": "resource",
           "#_3": "subject",
           "#_4": "updatedAt"
         },
         "ExpressionAttributeValues": {
           ":_0": "2022-01-17T19:27:03.412Z",
           ":_1": "Permissions",
           ":_2": "resource",
           ":_3": "subject"
         },
         "TableName": "BigTable-test",
         "ReturnValues": "ALL_NEW",
         "UpdateExpression": "set #_0 = if_not_exists(#_0, :_0), #_1 = :_1, #_2 = :_2, #_3 = :_3, #_4 = :_0",
         "Key": {
           "pk": "Permissions#resource",
           "sk": "Permissions#subject"
         }
       },
       "op": "update",
       "properties": {
         "pk": "Permissions#resource",
         "sk": "Permissions#subject",
         "typename": "Permissions",
         "resource": "resource",
         "subject": "subject",
         "updatedAt": "2022-01-17T19:27:03.412Z"
       }
     }
   } BigtableService

     at Logger.log (../../node_modules/@nestjs/common/services/logger.service.js:41:75)

 console.log
   [AWS dynamodb 200 0.016s 0 retries] updateItem({
     ExpressionAttributeNames: {
       '#_0': 'createdAt',
       '#_1': 'typename',
       '#_2': 'resource',
       '#_3': 'subject',
       '#_4': 'updatedAt'
     },
     ExpressionAttributeValues: {
       ':_0': { S: '2022-01-17T19:27:03.412Z' },
       ':_1': { S: 'Permissions' },
       ':_2': { S: 'resource' },
       ':_3': { S: 'subject' }
     },
     TableName: 'BigTable-test',
     ReturnValues: 'ALL_NEW',
     UpdateExpression: 'set #_0 = if_not_exists(#_0, :_0), #_1 = :_1, #_2 = :_2, #_3 = :_3, #_4 = :_0',
     Key: {
       pk: { S: 'Permissions#resource' },
       sk: { S: 'Permissions#subject' }
     }
   })

0 replies

onhate · 2022-01-17T19:43:40Z

onhate
Jan 17, 2022

@mobsense got it, it has to do with timastamps and the exists: null property, when using exists null it overwrites the set param, I'll raise a PR fixing this

0 replies

onhate · 2022-01-17T20:01:56Z

onhate
Jan 17, 2022

#251 here it is @mobsense

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updating a nested field #146

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 7 comments 10 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

Updating a nested field #146

fifi98 Sep 30, 2021

Replies: 7 comments · 10 replies

mobsense Sep 30, 2021 Maintainer

fifi98 Sep 30, 2021 Author

mobsense Sep 30, 2021 Maintainer

fifi98 Sep 30, 2021 Author

mobsense Sep 30, 2021 Maintainer

mobsense Sep 30, 2021 Maintainer

mobsense Sep 30, 2021 Maintainer

fifi98 Sep 30, 2021 Author

mobsense Sep 30, 2021 Maintainer

fifi98 Sep 30, 2021 Author

mobsense Sep 30, 2021 Maintainer

mobsense Sep 30, 2021 Maintainer

fifi98 Oct 1, 2021 Author

onhate Jan 17, 2022

onhate Jan 17, 2022

onhate Jan 17, 2022

fifi98
Sep 30, 2021

Replies: 7 comments 10 replies

mobsense
Sep 30, 2021
Maintainer

fifi98 Sep 30, 2021
Author

mobsense Sep 30, 2021
Maintainer

fifi98 Sep 30, 2021
Author

mobsense
Sep 30, 2021
Maintainer

mobsense Sep 30, 2021
Maintainer

mobsense Sep 30, 2021
Maintainer

fifi98 Sep 30, 2021
Author

mobsense Sep 30, 2021
Maintainer

fifi98 Sep 30, 2021
Author

mobsense
Sep 30, 2021
Maintainer

mobsense
Sep 30, 2021
Maintainer

fifi98 Oct 1, 2021
Author

onhate
Jan 17, 2022

onhate
Jan 17, 2022

onhate
Jan 17, 2022