The thing that I find myself wanting, which is lacking in both jq and zq afaik, is interactive exploration. I want to move around in a large JSON file, narrow my context to the portion I'm interested in, and do specialized queries and transformations on just the data I care about.
I wrote a tool to do this -- https://github.com/hotsphink/sfink-tools/blob/master/bin/jso... -- but I do not recommend it to anyone other than as perhaps a source of inspiration. It's slow and buggy, the syntax is cryptic and just matches whatever I came up with when I had a new need, etc. It probably wouldn't exist if I had heard of jq sooner.
But for what it does, it's awesome. I can do things like:
% json somefile.json
> ls
0/
1/
2/
> cd 0
> ls
info/
files/
timings/
version
> cat version
1.2b
> cat timings/*/mean
timings/firstPaint/mean = 51
timings/loadEventEnd/mean = 103
timings/timeToContentfulPaint/mean = 68
timings/timeToDomContentFlushed/mean = 67
timings/timeToFirstInteractive/mean = 658
timings/ttfb/mean = 6
There are commands for searching, modifying data, aggregating, etc., but those would be better done in a more principled, full-featured syntax like jq's.
I see ijq, and it looks really nice. But it doesn't have the context and restriction of focus that I'm looking for.
Do you know a way to copy the current selection in jless to the paste buffer? I find myself narrowing down a query for jq using jless but then having to manually remember the query to then jump to my terminal.
I really like fx (https://github.com/antonmedv/fx) for interactive stuff. It does exactly what I think you want. You can expand individual fields and explore the schema.
However, I really do like jq for queries and scripting, so I keep both around.
This is almost exactly how I think about the problem of deciding how to deep-key to a specific field of a nested json structure.
If you can emit the syntactic form as a Python or perl ref, or a jq array ref, then I could use your tool to find the structure and the other ones to stream.
That's one of the main steps forward for Brackit, a retargetable JSONiq query engine/compiler (http://brackit.io) and the append-only data store SirixDB (https://sirix.io) and a new web frontend. My vision is not only to explore the most recent revision but also any other older revisions, to display the diffs, to display the results of time travel queries... help is highly welcome as I'm myself a backend engineer and working on the query engine and the data store itself :-)
Detect changes of a specific node and the whole subtree/subtree:
let $node := jn:doc('mycol.jn','mydoc.jn')=>fieldName[[1]]
let $result := for $node-in-rev in jn:all-times($node)
return
if ((not(exists(jn:previous($node-in-rev))))
or (sdb:hash($node-in-rev) ne sdb:hash(jn:previous($node-in-rev)))) then
$node-in-rev
else
()
return [
for $jsonItem in $result
return { "node": $jsonItem, "revision": sdb:revision($jsonItem) }
]
Get all diffs between all revisions and serialize the output in an array:
let $maxRevision := sdb:revision(jn:doc('mycol.jn','mydoc.jn'))
let $result := for $i in (1 to $maxRevision)
return
if ($i > 1) then
jn:diff('mycol.jn','mydoc.jn',$i - 1, $i)
else
()
return [
for $diff at $pos in $result
return {"diffRev" || $pos || "toRev" || $pos + 1: jn:parse($diff)=>diffs}
]
One solution I’ve seen is basically to hijack fzf to interactively input a jq query, add closing brackets in a naive way, run jq -C … | head on an input file, and display the result as a fzf ‘preview’. fzf ends up handling things like the preview command and display and line-editing logic but it may be slow if you don’t get early results.
I wrote a tool to do this -- https://github.com/hotsphink/sfink-tools/blob/master/bin/jso... -- but I do not recommend it to anyone other than as perhaps a source of inspiration. It's slow and buggy, the syntax is cryptic and just matches whatever I came up with when I had a new need, etc. It probably wouldn't exist if I had heard of jq sooner.
But for what it does, it's awesome. I can do things like:
There are commands for searching, modifying data, aggregating, etc., but those would be better done in a more principled, full-featured syntax like jq's.I see ijq, and it looks really nice. But it doesn't have the context and restriction of focus that I'm looking for.