jq Basics in Practical DevSecOps Labs: Filtering and Processing JSON Data
Many security tools produce output in JSON format. jq is the standard command-line tool for filtering, transforming, and reading JSON data. This article covers the jq skills you need for the Practical DevSecOps courses.
Table of Contents
- Getting Started with jq
- Accessing Properties
- Working with Arrays
- Built-in Functions
- Common Questions
Getting Started with jq
jq is pre-installed on the DevSecOps Box. Explore available options:
jq --help
Prettifying JSON
The simplest filter is . (dot), which copies input to output with nice formatting:
echo '{"cloudprovider":{"name":"AWS","url":"www.aws.com"}}' | jq '.'
{
"cloudprovider": {
"name": "AWS",
"url": "www.aws.com"
}
}
Filtering API responses
curl https://randomuser.me/api/ | jq '.'
This makes an otherwise unreadable JSON API response easy to read.
Filtering JSON files
jq '.' learnjq.json
Accessing Properties
Use .field to access a property value:
jq '.[] | .details' learnjq.json
.[] iterates over each element in the array. .details selects the details property.
Chaining properties
jq '.[] | {name: .details.name, url: .details.url}' learnjq.json
{
"name": "AWS",
"url": "www.amazon.com"
}
{
"name": "Azure",
"url": "www.azure.com"
}
Wrapping output in an array
jq '[.[] | {name: .details.name, url: .details.url}]' learnjq.json
Collects all results into a single array.
Working with Arrays
Accessing by index
jq '.[1] | {name: .details.name, url: .details.url}' learnjq.json
Returns only the second element (index 1 = Azure).
Nested arrays
When a property is itself an array:
jq '.[] | {name: .details.name, services: [.services[]]}' learnjq.json
[.services[]] wraps the nested array items into a single array per parent object.
Built-in Functions
length — Count items
jq '.[] | {name: .details.name, servicecount: .services | length}' learnjq.json
{
"name": "AWS",
"servicecount": 2
}
select — Filter by condition
jq '.[] | select(.details.name=="AWS")' learnjq.json
Returns only entries where the cloud provider name is "AWS".
Challenge: Filter by nested property
jq '.[] | select(.services[].type=="CI-CD") | .details.name' learnjq.json
Finds all cloud providers that offer a CI-CD service.
Common Questions
| Question | Answer |
|---|---|
| How do I get just a raw string value? | Use -r: jq -r '.name' file.json strips quotes from string output. |
| Can I combine multiple filters? | Yes, use ` |
| How do I handle missing fields? | Use ? to suppress errors: jq '.possibly_missing_field?'. |
| Why does `jq '.' prettify the output? | By default, jq formats JSON with indentation. Use -c for compact output. |
| Is jq pre-installed in the labs? | Yes, jq is available on the DevSecOps Box. For local installation, see jq Installation. |
Read the full manual at jq Manual.
Wrap-Up
jq is essential for parsing security scan results, API responses, and configuration files in JSON format. Master .[], .field, select, and length to handle the vast majority of JSON processing tasks in the Practical DevSecOps courses.