JSON with curl and jq
This blog post captures some quick commands to deal with JSON from servers (Http APIs) using curl and jq command line tools
About Jq:
https://stedolan.github.io/jq/
Install jq:
Mac:
brew install jq
Ubuntu or Debian:
sudo apt-get install jq
Sample JSON content:
Assume the following JSON content will be returned from
URL http://api.example.com/some-json-content
{
"user_name": "User 123",
"some_other_field": {
"sub_field": "sub_value"
},
"array_field": [
{
"field1": "value1",
"field2": "value2"
},
{
"field1": "value3",
"field2": "value4"
}
]
}
URL http://api.example.com/some-json-array-content
[
{
"field1": "array value 1",
"field2": "array value 2",
"field3": ["one","two"]
},
{
"field1": "array value 3",
"field2": "array value 4",
"field3": ["three","four"]
}
]
URL http://api.example.com/some-primitive-json-array
[
1,
2,
3,
4
2,
3,
4
]
Pretty print json from curl
curl -s "http://api.example.com/some-json-content" | jq '.'Extract and print a field from json object
curl -s "http://api.example.com/some-json-content" | jq -r '.array_field[].field1'value1
curl -s "http://api.example.com/some-json-content" | jq -r '.some_other_field.sub_field'
sub_value
Extract and print a field from all elements of json array
curl -s "http://api.example.com/some-json-array-content" | jq -r '.[].field1'array value 1
array value 3
Extract and print a field from first element of json array
curl -s "http://api.example.com/some-json-array-content" | jq -r '.[0].field1'array value 1
Convert json object array to json lines
curl -s "http://api.example.com/some-json-array-content" | jq -c '.[]'
{"field1":"array value 1","field2":"array value 2"}
{"field1":"array value 3","field2":"array value 4"}
Convert json lines to json object array
printf '{"A":"B"}\n{"C":"D"}' | jq -cs '.'
[{"A":"B"},{"C":"D"}]
Map and select elements from primitive json array
curl -s "http://api.example.com/some-primitive-json-array" | jq -r 'map(select(. >= 3)) | .[]'3
4
Select an element from object array and print a field from the object
curl -s "http://api.example.com/some-json-array-content" | jq -r '.[] | select(.field1 == "array value 1") | .field2'array value 4
curl -s "http://api.example.com/some-json-array-content" | jq -r '.[] | select(.field3 | contains(["two"])) | .field2'
array value 2
Extract a field from json and store it in bash variable and use it in subsequent curls
username=$(curl -s "http://api.example.com/some-json-content" | jq -r '.user_name')echo ${username}
curl -X PUT "http://api.example.com/some-operation" -H "Content-Type: application/json;charset=UTF-8" --data-binary "{\"user_name\": \"$username\", \"first_name\": \"test_user\"}"
Get length of an array and iterate through array using index
JSON_CONTENT=$( curl -s "http://api.example.com/some-json-content" )
ARRAY_LENGTH=$( echo "$JSON_CONTENT" | jq -r '.array_field | length' )
for (( c=0; c<$ARRAY_LENGTH; c=$c+1 )); do echo "$JSON_CONTENT" | jq -r ".array_field[$c].field2"; done;
Create a json with jq and send it as request payload in curl
STRING_VALUE="sample string"
NUMBER_VALUE=21
DECIMAL_VALUE=20.02
BOOLEAN_VALUE=true
JSON_OBJECT={}
JSON_OBJECT=$( echo "$JSON_OBJECT" | jq -c ". += {\"string_field\":\"$STRING_VALUE\"}" )
JSON_OBJECT=$( echo "$JSON_OBJECT" | jq -c ". += {\"number_field\":$NUMBER_VALUE}" )
JSON_OBJECT=$( echo "$JSON_OBJECT" | jq -c ". += {\"decimal_field\":$DECIMAL_VALUE}" )
JSON_OBJECT=$( echo "$JSON_OBJECT" | jq -c ". += {\"boolean_field\":$BOOLEAN_VALUE}" )
echo "JSON with primitive data types"
echo "$JSON_OBJECT"
ELEMENT1="sample value 1"
ELEMENT2="sample value 2"
JSON_PRIMITIVE_ARRAY=[]
JSON_PRIMITIVE_ARRAY=$( echo "$JSON_PRIMITIVE_ARRAY" | jq -c ". += [\"$ELEMENT1\"]" )
JSON_PRIMITIVE_ARRAY=$( echo "$JSON_PRIMITIVE_ARRAY" | jq -c ". += [\"$ELEMENT2\"]" )
echo "JSON array with primitive data types"
echo "$JSON_PRIMITIVE_ARRAY"
JSON_OBJECT_ARRAY=[]
JSON_OBJECT_ARRAY=$( echo "$JSON_OBJECT_ARRAY" | jq -c ". += [{\"value\":\"$ELEMENT1\"}]" )
JSON_OBJECT_ARRAY=$( echo "$JSON_OBJECT_ARRAY" | jq -c ". += [{\"value\":\"$ELEMENT2\"}]" )
echo "JSON array with JSON objects"
echo "$JSON_OBJECT_ARRAY"
JSON_OBJECT=$( echo "$JSON_OBJECT" | jq -c ". += {\"primitive_array\":$JSON_PRIMITIVE_ARRAY}" )
JSON_OBJECT=$( echo "$JSON_OBJECT" | jq -c ". += {\"object_array\":$JSON_OBJECT_ARRAY}" )
echo "JSON with primitives and arrays"
echo "$JSON_OBJECT"
echo "pretty printing the json"
echo "$JSON_OBJECT" | jq -r '.'
curl -v --header "Content-Type: application/json; charset=utf-8" -X POST "https://www.example.org/" --data-binary "$JSON_OBJECT"
Comments
Post a Comment