Syntax for returning a set of indices/keys of an array/object

[gregsdennis]

This question popped up on StackOverflow in which the user wants to select the keys present in the resulting object. To do this, they're expecting to end the query with ~.

So $.*~ would select all of the items and then return the keys for those items.

In a more complex query, you could have $[[email protected] == 5]~ to find the property names (or array indices) of every child that meets the condition.

To date, I've asked what implementation they're using that supports this syntax, but I've yet to receive a result.

We also have #324 that proposed a function extension to do this rather than syntax, but there wasn't much discussion there other than the data is available through the location paths in the query result. The problem with this is that you can't check that until the query is complete.

I think a benefit of this is that you could continue querying on the property name set.

Building on the more complex example, with an array of objects, if I wanted the index of the third item where @.foo == 5, I could do $[[email protected] == 5]~[2].

Logging for visibility and discussion. It seems simple enough. (This would also be the first functionality that actually uses the location data as part of the query.)

[glyn]

Returning indices or keys from a JSONPath query could require extending the spec's current notion of a nodelist, since the spec doesn't currently regard an index or a key as a node with a location. I suppose such a location could be defined, for example, by extending Normalized Paths so that they can address an index or a key, e.g. by allowing ~ at the end of a Normalized Path.

If the spec was to admit indices and keys as nodes, the definition of the descendant segment would need adjusting.

Building on the more complex example, with an array of objects, if I wanted the index of the third item where @.foo == 5, I could do $[[email protected] == 5]~[2].

Not sure what example this refers to, but I have a couple of observations. If that index exists, its value is 2. Also, the query $[[email protected] == 5]~[2] ends with [2] although the intermediate result of $[[email protected] == 5]~ would presumably be a nodelist of indices. So [2] would be indexing a nodelist, which is another extension to the spec.

[gregsdennis]

the spec doesn't currently regard an index or a key as a node with a location

Technically, it's just the last segment of the path to the node, which the implementation should be tracking anyway.

If the spec was to admit indices and keys as node

I don't think it needs to in order to support this, for the reason above.

So [2] would be indexing a nodelist, which is another extension to the spec.

Yes. That's true. I'll see if I can devise another example. Extending the segment that way wasn't my intent.

[cabo]

I think we already had discussed extensively whether to extend JSONPath from selection to projection, and how member names (or array indices) are not nodes in the JSONPath model.

Your example may be a bit contrived, but is an indication that we would want to extend the model to cover this kind of use cases:

$[[email protected] == 5]~[2]

If I translate the intent I gather from this into what we have today, ~ would start a new query by converting the nodelist coming from the previous segment to a new JSON value (containing an array of indices) that serves as the query argument to the [2] selection.

[jg-rp]

To date, I've asked what implementation they're using that supports this syntax, but I've yet to receive a result.

Jayway JsonPath, JSONPath Plus and Python JSONPath all support using ~ as a keys selector, each with subtly different behaviour.

[glyn]

@cabo That's an interesting approach. For the output of ~ to be input to a segment such as [2], wouldn't we have to extend the meaning of such segments to apply to arrays as well as nodelists (with values from the array and locations referring to those values)? That's quite a departure from the current model!

Also, if I've understood correctly the array produced by ~ could contain indices and member names, as in the example below.

Value:

[{"x": "a", "y": "b"},[1,2]]

Query:

..[*]~[0]

could produce the value 0, "x", or "y".

[danielaparker]

Jayway JsonPath, JSONPath Plus and Python JSONPath all support using ~ as a keys selector, each with subtly different behaviour.

Jayway supports a keys() function as an alternative to the tilde operator, I don't think it supports the tilde operator itself.

Note that Jayway (and some other implementations) support functions applied to paths, the IETF document restricts them to filter expressions.

[He-Pin]

That's ture, for my case I want to update the json object string to a json object, do some update on it and then translate it back json object string
with json

{
  "data": {
    "header_1": {
      "a": "1",
      "b": "2",
      "body": "{\"c\":\"3\"}"
    },
    "header_2": {
      "a": "1",
      "b": "2",
      "body": "{\"c\":\"3\"}"
    }
  }
}

the progress

//to json
{"data":{"header_1":{"a":"1","b":"2","body":{"c":"3"}},"header_2":{"a":"1","b":"2","body":{"c":"3"}}}}
// update `c`
{"data":{"header_1":{"a":"1","b":"2","body":{"c":"6666"}},"header_2":{"a":"1","b":"2","body":{"c":"6666"}}}}
//conver back
{"data":{"header_1":{"a":"1","b":"2","body":"{\"c\":\"6666\"}"},"header_2":{"a":"1","b":"2","body":"{\"c\":\"6666\"}"}}}

[gregsdennis]

@He-Pin JSON Path isn't going to do any kind of conversions. It's a query language, nothing more. It can get you the values that you want to modify, but you'll need to something custom for your use case. I don't think JSON Path is the tool you need.

[glyn]

@He-Pin I agree with @gregsdennis in that JSONPath per se doesn't solve your problem. However, it would be possible to implement JSONPath to support the use case you describe. Since RFC 9535 doesn't stipulate how a nodelist is implemented, it could be a list of references into a tree representing the argument. That's the approach I took with yaml-jsonpath (NB. this pre-dates and does not implement RFC 9535). The result can then be used to manipulate the input tree, etc.

[He-Pin]

Thanks for your replay, actually, the well defined RFC helps me a lot to implement my own parser, I will start implement the interpreter after holiday

The current RFC is in good shape, but I think there are some specific user cases as I statemented, If that jsonpath its self is more extensible or expressive, that would be better.

A good news I want to share is our implementation is based on the current RFC, and thank you very much, I would like to buy you coffee if I can, thanks.

[timbray]

Interesting discussion. @He-Pin - if your implementation becomes public, do please let us know, that would be very interesting.

[He-Pin]

Interesting discussion. @He-Pin - if your implementation becomes public, do please let us know, that would be very interesting.

I will, it's currently written in Java21 and Scala, as the.RFC is well defined, implement it ourself is very straightforward.

[gregsdennis]

@He-Pin have you run your implementation against the test suite?

(BTW - To the (former) WG who decided that Slack shouldn't be a part of the JSON Path effort, this conversation about a new implementation would be perfect for Slack instead of off-topic in a GH issue.)

[He-Pin]

@He-Pin have you run your implementation against the test suite?

(BTW - To the (former) WG who decided that Slack shouldn't be a part of the JSON Path effort, this conversation about a new implementation would be perfect for Slack instead of off-topic in a GH issue.)

Thanks for the pointer, yes, I will make our implementation full pass the tests, it's holiday in China, will keep you updated once done.

[danielaparker]

(BTW - To the (former) WG who decided that Slack shouldn't be a part of the JSON Path effort, this conversation about a new implementation would be perfect for Slack instead of off-topic in a GH issue.)

Why not simply enable github discussions? That's what they're for. I never fully understood why the WG opposed that when it was suggested earlier.

[cabo]

The jsonpath WG has completed its chartered work.
I think that maintaining a space for discussions is a laudable goal, but we'd need to find people that might want to curate that, as the WG chairs are now out of that job.

[gregsdennis]

I'm going to open a new issue for the community building discussion. See #521.

[He-Pin]

@gregsdennis I have all the test suites passed, and my parser is written with fast parse.

[gregsdennis]

I recently tried to implement a key() function in my library and ran into a problem that's likely due to how .Net designed their JSON DOM rather than any shortcoming inherent to this proposal.

In the .Net JSON DOM, they've decided to merge the ideas of .Net's null and JSON's null literal. That is, given a variable of type JsonNode, a null value (what would be 0x0 in something like C) is used for the JSON value null.

This design decision means that I can't identify where a null value may be in a JSON structure if I'm only given the value itself. Even though the JsonNode API has methods for finding its parent or the key/index that it's under, this API is inaccessible to me because I need an actual value to use it.

The impact for me is that, in order to support this, I need to (internally) represent @ not as the value but as the entire node (which is defined as the value and its location).

I expect if .Net has this issue, it's at least likely that some other languages could have this problem as well. It's not a show-stopper, but it's definitely something that needs to be considered when implementing this.