YQ(1)

NAME

yq is a portable command-line YAML processor

SYNOPSIS

yq [eval/eval-all] [expression] files..

eval/e - (default) Apply the expression to each document in each yaml file in sequence

eval-all/ea - Loads all yaml documents of all yaml files and runs expression once

DESCRIPTION

a lightweight and portable command-line YAML processor. yq uses jq (https://github.com/stedolan/jq) like syntax but works with yaml files as well as json. It doesn’t yet support everything jq does - but it does support the most common operations and functions, and more is being added continuously.

This documentation is also available at https://mikefarah.gitbook.io/yq/ # QUICK GUIDE

  Read a value:

    yq '.a.b[0].c' file.yaml

  Pipe from STDIN:

    cat file.yaml | yq '.a.b[0].c'

  Update a yaml file, inplace

    yq -i '.a.b[0].c = "cool"' file.yaml

  Update using environment variables

    NAME=mike yq -i '.a.b[0].c = strenv(NAME)' file.yaml

  Merge multiple files

    yq ea '. as $item ireduce ({}; . * $item )' path/to/*.yml

Note the use of ea to evaluate all files at once (instead of in sequence.)

  Multiple updates to a yaml file

    yq -i '
      .a.b[0].c = "cool" |
      .x.y.z = "foobar" |
      .person.name = strenv(NAME)
    ' file.yaml

See the documentation (https://mikefarah.gitbook.io/yq/) for more.

KNOWN ISSUES / MISSING FEATURES

•   yq attempts to preserve comment positions and whitespace as much as possible, but it does not handle all scenarios (see https://github.com/go-yaml/yaml/tree/v3 for details)

•   Powershell has its own...opinions: https://mikefarah.gitbook.io/yq/usage/tips-and-tricks#quotes-in-windows-powershell

BUGS / ISSUES / FEATURE REQUESTS

Please visit the GitHub page https://github.com/mikefarah/yq/.

HOW IT WORKS

In yq expressions are made up of operators and pipes. A context of nodes is passed through the expression and each operation takes the context as input and returns a new context as output. That output is piped in as input for the next operation in the expression. To begin with, the context is set to the first yaml document of the first yaml file (if processing in sequence using eval).

Lets look at a couple of examples.

  Simple assignment example

Given a document like:

    a: cat
    b: dog

with an expression:

    .a = .b

Like math expressions - operator precedence is important.

The = operator takes two arguments, a lhs expression, which in this case is .a and rhs expression which is .b.

It pipes the current, lets call it `root' context through the lhs expression of .a to return the node

    cat

Sidenote: this node holds not only its value `cat', but comments and metadata too, including path and parent information.

The = operator then pipes the `root' context through the rhs expression of .b to return the node

    dog

Both sides have now been evaluated, so now the operator copies across the value from the RHS (.b) to the LHS (.a), and it returns the now updated context:

    a: dog
    b: dog

  Complex assignment, operator precedence rules

Just like math expressions - yq expressions have an order of precedence. The pipe | operator has a low order of precedence, so operators with higher precedence will get evaluated first.

Most of the time, this is intuitively what you’d want, for instance .a = "cat" | .b = "dog" is effectively: (.a = "cat") | (.b = "dog").

However, this is not always the case, particularly if you have a complex LHS or RHS expression, for instance if you want to select particular nodes to update.

Lets say you had:

    - name: bob
      fruit: apple
    - name: sally
      fruit: orange

Lets say you wanted to update the sally entry to have fruit: `mango'. The incorrect way to do that is: .[] | select(.name == "sally") | .fruit = "mango".

Because | has a low operator precedence, this will be evaluated (incorrectly) as : (.[]) | (select(.name == "sally")) | (.fruit = "mango"). What you’ll see is only the updated segment returned:

    name: sally
    fruit: mango

To properly update this yaml, you will need to use brackets (think BODMAS from maths) and wrap the entire LHS: (.[] | select(.name == "sally") | .fruit) = "mango"

Now that entire LHS expression is passed to the `assign' (=) operator, and the yaml is correctly updated and returned:

    - name: bob
      fruit: apple
    - name: sally
      fruit: mango

  Relative update (e.g. |=)

There is another form of the = operator which we call the relative form. It’s very similar to = but with one key difference when evaluating the RHS expression.

In the plain form, we pass in the `root' level context to the RHS expression. In relative form, we pass in each result of the LHS to the RHS expression. Let’s go through an example.

Given a document like:

    a: 1
    b: thing

with an expression:

    .a |= . + 1

Similar to the = operator, |= takes two operands, the LHS and RHS.

It pipes the current context (the whole document) through the LHS expression of .a to get the node value:

    1

Now it pipes that LHS context into the RHS expression . + 1 (whereas in the = plain form it piped the original document context into the RHS) to yield:

    2

The assignment operator then copies across the value from the RHS to the value on the LHS, and it returns the now updated `root' context:

    a: 2
    b: thing

Add

Add behaves differently according to the type of the LHS: * arrays: concatenate * number scalars: arithmetic addition * string scalars: concatenate * maps: shallow merge (use the multiply operator (*) to deeply merge)

Use += as a relative append assign for things like increment. Note that .a += .x is equivalent to running .a = .a + .x.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Concatenate arrays

Given a sample.yml file of:

    a:
      - 1
      - 2
    b:
      - 3
      - 4

then

    yq '.a + .b' sample.yml

will output

    - 1
    - 2
    - 3
    - 4

  Concatenate to existing array

Note that the styling of a is kept.

Given a sample.yml file of:

    a: [1,2]
    b:
      - 3
      - 4

then

    yq '.a += .b' sample.yml

will output

    a: [1, 2, 3, 4]
    b:
      - 3
      - 4

  Concatenate null to array

Given a sample.yml file of:

    a:
      - 1
      - 2

then

    yq '.a + null' sample.yml

will output

    - 1
    - 2

  Append to existing array

Note that the styling is copied from existing array elements

Given a sample.yml file of:

    a: ['dog']

then

    yq '.a += "cat"' sample.yml

will output

    a: ['dog', 'cat']

  Add new object to array

Given a sample.yml file of:

    a:
      - dog: woof

then

    yq '.a + {"cat": "meow"}' sample.yml

will output

    - dog: woof
    - cat: meow

  Relative append

Given a sample.yml file of:

    a:
      a1:
        b:
          - cat
      a2:
        b:
          - dog
      a3: {}

then

    yq '.a[].b += ["mouse"]' sample.yml

will output

    a:
      a1:
        b:
          - cat
          - mouse
      a2:
        b:
          - dog
          - mouse
      a3:
        b:
          - mouse

  String concatenation

Given a sample.yml file of:

    a: cat
    b: meow

then

    yq '.a += .b' sample.yml

will output

    a: catmeow
    b: meow

  Number addition - float

If the lhs or rhs are floats then the expression will be calculated with floats.

Given a sample.yml file of:

    a: 3
    b: 4.9

then

    yq '.a = .a + .b' sample.yml

will output

    a: 7.9
    b: 4.9

  Number addition - int

If both the lhs and rhs are ints then the expression will be calculated with ints.

Given a sample.yml file of:

    a: 3
    b: 4

then

    yq '.a = .a + .b' sample.yml

will output

    a: 7
    b: 4

  Increment numbers

Given a sample.yml file of:

    a: 3
    b: 5

then

    yq '.[] += 1' sample.yml

will output

    a: 4
    b: 6

  Date addition

You can add durations to dates. Assumes RFC3339 date time format, see date-time operators (https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information.

Given a sample.yml file of:

    a: 2021-01-01T00:00:00Z

then

    yq '.a += "3h10m"' sample.yml

will output

    a: 2021-01-01T03:10:00Z

  Date addition - custom format

You can add durations to dates. See date-time operators (https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information.

Given a sample.yml file of:

    a: Saturday, 15-Dec-01 at 2:59AM GMT

then

    yq 'with_dtf("Monday, 02-Jan-06 at 3:04PM MST", .a += "3h1m")' sample.yml

will output

    a: Saturday, 15-Dec-01 at 6:00AM GMT

  Add to null

Adding to null simply returns the rhs

Running

    yq --null-input 'null + "cat"'

will output

    cat

  Add maps to shallow merge

Adding objects together shallow merges them. Use * to deeply merge.

Given a sample.yml file of:

    a:
      thing:
        name: Astuff
        value: x
      a1: cool
    b:
      thing:
        name: Bstuff
        legs: 3
      b1: neat

then

    yq '.a += .b' sample.yml

will output

    a:
      thing:
        name: Bstuff
        legs: 3
      a1: cool
      b1: neat
    b:
      thing:
        name: Bstuff
        legs: 3
      b1: neat

  Custom types: that are really strings

When custom tags are encountered, yq will try to decode the underlying type.

Given a sample.yml file of:

    a: !horse cat
    b: !goat _meow

then

    yq '.a += .b' sample.yml

will output

    a: !horse cat_meow
    b: !goat _meow

  Custom types: that are really numbers

When custom tags are encountered, yq will try to decode the underlying type.

Given a sample.yml file of:

    a: !horse 1.2
    b: !goat 2.3

then

    yq '.a += .b' sample.yml

will output

    a: !horse 3.5
    b: !goat 2.3

Alternative (Default value)

This operator is used to provide alternative (or default) values when a particular expression is either null or false.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  LHS is defined

Given a sample.yml file of:

    a: bridge

then

    yq '.a // "hello"' sample.yml

will output

    bridge

  LHS is not defined

Given a sample.yml file of:

    {}

then

    yq '.a // "hello"' sample.yml

will output

    hello

  LHS is null

Given a sample.yml file of:

    a: ~

then

    yq '.a // "hello"' sample.yml

will output

    hello

  LHS is false

Given a sample.yml file of:

    a: false

then

    yq '.a // "hello"' sample.yml

will output

    hello

  RHS is an expression

Given a sample.yml file of:

    a: false
    b: cat

then

    yq '.a // .b' sample.yml

will output

    cat

  Update or create - entity exists

This initialises a if it’s not present

Given a sample.yml file of:

    a: 1

then

    yq '(.a // (.a = 0)) += 1' sample.yml

will output

    a: 2

  Update or create - entity does not exist

This initialises a if it’s not present

Given a sample.yml file of:

    b: camel

then

    yq '(.a // (.a = 0)) += 1' sample.yml

will output

    b: camel
    a: 1

Anchor and Alias Operators

Use the alias and anchor operators to read and write yaml aliases and anchors. The explode operator normalises a yaml file (dereference (or expands) aliases and remove anchor names).

yq supports merge aliases (like <<: *blah) however this is no longer in the standard yaml spec (1.2) and so yq will automatically add the !!merge tag to these nodes as it is effectively a custom tag.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Merge one map

see https://yaml.org/type/merge.html

Given a sample.yml file of:

    - &CENTER
      x: 1
      y: 2
    - &LEFT
      x: 0
      y: 2
    - &BIG
      r: 10
    - &SMALL
      r: 1
    - !!merge <<: *CENTER
      r: 10

then

    yq '.[4] | explode(.)' sample.yml

will output

    x: 1
    y: 2
    r: 10

  Merge multiple maps

see https://yaml.org/type/merge.html

Given a sample.yml file of:

    - &CENTER
      x: 1
      y: 2
    - &LEFT
      x: 0
      y: 2
    - &BIG
      r: 10
    - &SMALL
      r: 1
    - !!merge <<:
        - *CENTER
        - *BIG

then

    yq '.[4] | explode(.)' sample.yml

will output

    r: 10
    x: 1
    y: 2

  Override

see https://yaml.org/type/merge.html

Given a sample.yml file of:

    - &CENTER
      x: 1
      y: 2
    - &LEFT
      x: 0
      y: 2
    - &BIG
      r: 10
    - &SMALL
      r: 1
    - !!merge <<:
        - *BIG
        - *LEFT
        - *SMALL
      x: 1

then

    yq '.[4] | explode(.)' sample.yml

will output

    r: 10
    x: 1
    y: 2

  Get anchor

Given a sample.yml file of:

    a: &billyBob cat

then

    yq '.a | anchor' sample.yml

will output

    billyBob

  Set anchor

Given a sample.yml file of:

    a: cat

then

    yq '.a anchor = "foobar"' sample.yml

will output

    a: &foobar cat

  Set anchor relatively using assign-update

Given a sample.yml file of:

    a:
      b: cat

then

    yq '.a anchor |= .b' sample.yml

will output

    a: &cat
      b: cat

  Get alias

Given a sample.yml file of:

    b: &billyBob meow
    a: *billyBob

then

    yq '.a | alias' sample.yml

will output

    billyBob

  Set alias

Given a sample.yml file of:

    b: &meow purr
    a: cat

then

    yq '.a alias = "meow"' sample.yml

will output

    b: &meow purr
    a: *meow

  Set alias to blank does nothing

Given a sample.yml file of:

    b: &meow purr
    a: cat

then

    yq '.a alias = ""' sample.yml

will output

    b: &meow purr
    a: cat

  Set alias relatively using assign-update

Given a sample.yml file of:

    b: &meow purr
    a:
      f: meow

then

    yq '.a alias |= .f' sample.yml

will output

    b: &meow purr
    a: *meow

  Explode alias and anchor

Given a sample.yml file of:

    f:
      a: &a cat
      b: *a

then

    yq 'explode(.f)' sample.yml

will output

    f:
      a: cat
      b: cat

  Explode with no aliases or anchors

Given a sample.yml file of:

    a: mike

then

    yq 'explode(.a)' sample.yml

will output

    a: mike

  Explode with alias keys

Given a sample.yml file of:

    f:
      a: &a cat
      *a: b

then

    yq 'explode(.f)' sample.yml

will output

    f:
      a: cat
      cat: b

  Explode with merge anchors

Given a sample.yml file of:

    foo: &foo
      a: foo_a
      thing: foo_thing
      c: foo_c
    bar: &bar
      b: bar_b
      thing: bar_thing
      c: bar_c
    foobarList:
      b: foobarList_b
      !!merge <<:
        - *foo
        - *bar
      c: foobarList_c
    foobar:
      c: foobar_c
      !!merge <<: *foo
      thing: foobar_thing

then

    yq 'explode(.)' sample.yml

will output

    foo:
      a: foo_a
      thing: foo_thing
      c: foo_c
    bar:
      b: bar_b
      thing: bar_thing
      c: bar_c
    foobarList:
      b: bar_b
      thing: foo_thing
      c: foobarList_c
      a: foo_a
    foobar:
      c: foo_c
      a: foo_a
      thing: foobar_thing

  Dereference and update a field

`Use explode with multiply to dereference an object

Given a sample.yml file of:

    item_value: &item_value
      value: true
    thingOne:
      name: item_1
      !!merge <<: *item_value
    thingTwo:
      name: item_2
      !!merge <<: *item_value

then

    yq '.thingOne |= explode(.) * {"value": false}' sample.yml

will output

    item_value: &item_value
      value: true
    thingOne:
      name: item_1
      value: false
    thingTwo:
      name: item_2
      !!merge <<: *item_value

Assign (Update)

This operator is used to update node values. It can be used in either the:

  plain form: =

Which will assign the LHS node values to the RHS node values. The RHS expression is run against the matching nodes in the pipeline.

  relative form: |=

This will do a similar thing to the plain form, however, the RHS expression is run against the LHS nodes. This is useful for updating values based on old values, e.g. increment.

  Flags

•   c clobber custom tags

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Create yaml file

Running

    yq --null-input '.a.b = "cat" | .x = "frog"'

will output

    a:
      b: cat
    x: frog

  Update node to be the child value

Given a sample.yml file of:

    a:
      b:
        g: foof

then

    yq '.a |= .b' sample.yml

will output

    a:
      g: foof

  Double elements in an array

Given a sample.yml file of:

    - 1
    - 2
    - 3

then

    yq '.[] |= . * 2' sample.yml

will output

    - 2
    - 4
    - 6

  Update node from another file

Note this will also work when the second file is a scalar (string/number)

Given a sample.yml file of:

    a: apples

And another sample another.yml file of:

    b: bob

then

    yq eval-all 'select(fileIndex==0).a = select(fileIndex==1) | select(fileIndex==0)' sample.yml another.yml

will output

    a:
      b: bob

  Update node to be the sibling value

Given a sample.yml file of:

    a:
      b: child
    b: sibling

then

    yq '.a = .b' sample.yml

will output

    a: sibling
    b: sibling

  Updated multiple paths

Given a sample.yml file of:

    a: fieldA
    b: fieldB
    c: fieldC

then

    yq '(.a, .c) = "potato"' sample.yml

will output

    a: potato
    b: fieldB
    c: potato

  Update string value

Given a sample.yml file of:

    a:
      b: apple

then

    yq '.a.b = "frog"' sample.yml

will output

    a:
      b: frog

  Update string value via |=

Note there is no difference between = and |= when the RHS is a scalar

Given a sample.yml file of:

    a:
      b: apple

then

    yq '.a.b |= "frog"' sample.yml

will output

    a:
      b: frog

  Update deeply selected results

Note that the LHS is wrapped in brackets! This is to ensure we don’t first filter out the yaml and then update the snippet.

Given a sample.yml file of:

    a:
      b: apple
      c: cactus

then

    yq '(.a[] | select(. == "apple")) = "frog"' sample.yml

will output

    a:
      b: frog
      c: cactus

  Update array values

Given a sample.yml file of:

    - candy
    - apple
    - sandy

then

    yq '(.[] | select(. == "*andy")) = "bogs"' sample.yml

will output

    - bogs
    - apple
    - bogs

  Update empty object

Given a sample.yml file of:

    {}

then

    yq '.a.b |= "bogs"' sample.yml

will output

    a:
      b: bogs

  Update node value that has an anchor

Anchor will remaple

Given a sample.yml file of:

    a: &cool cat

then

    yq '.a = "dog"' sample.yml

will output

    a: &cool dog

  Update empty object and array

Given a sample.yml file of:

    {}

then

    yq '.a.b.[0] |= "bogs"' sample.yml

will output

    a:
      b:
        - bogs

  Custom types are maintained by default

Given a sample.yml file of:

    a: !cat meow
    b: !dog woof

then

    yq '.a = .b' sample.yml

will output

    a: !cat woof
    b: !dog woof

  Custom types: clovver

Use the c option to clobber custom tags

Given a sample.yml file of:

    a: !cat meow
    b: !dog woof

then

    yq '.a =c .b' sample.yml

will output

    a: !dog woof
    b: !dog woof

Boolean Operators

The or and and operators take two parameters and return a boolean result.

not flips a boolean from true to false, or vice versa.

any will return true if there are any true values in a array sequence, and all will return true if all elements in an array are true.

any_c(condition) and all_c(condition) are like any and all but they take a condition expression that is used against each element to determine if it’s true. Note: in jq you can simply pass a condition to any or all and it simply works - yq isn’t that clever..yet

These are most commonly used with the select operator to filter particular nodes.

  Related Operators

•   equals / not equals (==, !=) operators here (https://mikefarah.gitbook.io/yq/operators/equals)

•   comparison (>=, < etc) operators here (https://mikefarah.gitbook.io/yq/operators/compare)

•   select operator here (https://mikefarah.gitbook.io/yq/operators/select)

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  or example

Running

    yq --null-input 'true or false'

will output

    true

  and example

Running

    yq --null-input 'true and false'

will output

    false

  Matching nodes with select, equals and or

Given a sample.yml file of:

    - a: bird
      b: dog
    - a: frog
      b: bird
    - a: cat
      b: fly

then

    yq '[.[] | select(.a == "cat" or .b == "dog")]' sample.yml

will output

    - a: bird
      b: dog
    - a: cat
      b: fly

  any returns true if any boolean in a given array is true

Given a sample.yml file of:

    - false
    - true

then

    yq 'any' sample.yml

will output

    true

  any returns false for an empty array

Given a sample.yml file of:

    []

then

    yq 'any' sample.yml

will output

    false

  any_c returns true if any element in the array is true for the given condition.

Given a sample.yml file of:

    a:
      - rad
      - awesome
    b:
      - meh
      - whatever

then

    yq '.[] |= any_c(. == "awesome")' sample.yml

will output

    a: true
    b: false

  all returns true if all booleans in a given array are true

Given a sample.yml file of:

    - true
    - true

then

    yq 'all' sample.yml

will output

    true

  all returns true for an empty array

Given a sample.yml file of:

    []

then

    yq 'all' sample.yml

will output

    true

  all_c returns true if all elements in the array are true for the given condition.

Given a sample.yml file of:

    a:
      - rad
      - awesome
    b:
      - meh
      - 12

then

    yq '.[] |= all_c(tag == "!!str")' sample.yml

will output

    a: true
    b: false

  Not true is false

Running

    yq --null-input 'true | not'

will output

    false

  Not false is true

Running

    yq --null-input 'false | not'

will output

    true

  String values considered to be true

Running

    yq --null-input '"cat" | not'

will output

    false

  Empty string value considered to be true

Running

    yq --null-input '"" | not'

will output

    false

  Numbers are considered to be true

Running

    yq --null-input '1 | not'

will output

    false

  Zero is considered to be true

Running

    yq --null-input '0 | not'

will output

    false

  Null is considered to be false

Running

    yq --null-input '~ | not'

will output

    true

Collect into Array

This creates an array using the expression between the square brackets.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Collect empty

Running

    yq --null-input '[]'

will output

    []

  Collect single

Running

    yq --null-input '["cat"]'

will output

    - cat

  Collect many

Given a sample.yml file of:

    a: cat
    b: dog

then

    yq '[.a, .b]' sample.yml

will output

    - cat
    - dog

Column

Returns the column of the matching node. Starts from 1, 0 indicates there was no column data.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Returns column of value node

Given a sample.yml file of:

    a: cat
    b: bob

then

    yq '.b | column' sample.yml

will output

    4

  Returns column of key node

Pipe through the key operator to get the column of the key

Given a sample.yml file of:

    a: cat
    b: bob

then

    yq '.b | key | column' sample.yml

will output

    1

  First column is 1

Given a sample.yml file of:

    a: cat

then

    yq '.a | key | column' sample.yml

will output

    1

  No column data is 0

Running

    yq --null-input '{"a": "new entry"} | column'

will output

    0

Comment Operators

Use these comment operators to set or retrieve comments. Note that line comments on maps/arrays are actually set on the key node as opposed to the value (map/array). See below for examples.

Like the = and |= assign operators, the same syntax applies when updating comments:

  plain form: =

This will assign the LHS nodes comments to the expression on the RHS. The RHS is run against the matching nodes in the pipeline

  relative form: |=

Similar to the plain form, however the RHS evaluates against each matching LHS node! This is useful if you want to set the comments as a relative expression of the node, for instance its value or path.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Set line comment

Set the comment on the key node for more reliability (see below).

Given a sample.yml file of:

    a: cat

then

    yq '.a line_comment="single"' sample.yml

will output

    a: cat # single

  Set line comment of a maps/arrays

For maps and arrays, you need to set the line comment on the key node. This will also work for scalars.

Given a sample.yml file of:

    a:
      b: things

then

    yq '(.a | key) line_comment="single"' sample.yml

will output

    a: # single
      b: things

  Use update assign to perform relative updates

Given a sample.yml file of:

    a: cat
    b: dog

then

    yq '.. line_comment |= .' sample.yml

will output

    a: cat # cat
    b: dog # dog

  Where is the comment - map key example

The underlying yaml parser can assign comments in a document to surprising nodes. Use an expression like this to find where you comment is. `p' indicates the path, `isKey' is if the node is a map key (as opposed to a map value). From this, you can see the `hello-world-comment' is actually on the `hello' key

Given a sample.yml file of:

    hello: # hello-world-comment
      message: world

then

    yq '[... | {"p": path | join("."), "isKey": is_key, "hc": headComment, "lc": lineComment, "fc": footComment}]' sample.yml

will output

    - p: ""
      isKey: false
      hc: ""
      lc: ""
      fc: ""
    - p: hello
      isKey: true
      hc: ""
      lc: hello-world-comment
      fc: ""
    - p: hello
      isKey: false
      hc: ""
      lc: ""
      fc: ""
    - p: hello.message
      isKey: true
      hc: ""
      lc: ""
      fc: ""
    - p: hello.message
      isKey: false
      hc: ""
      lc: ""
      fc: ""

  Retrieve comment - map key example

From the previous example, we know that the comment is on the `hello' key as a lineComment

Given a sample.yml file of:

    hello: # hello-world-comment
      message: world

then

    yq '.hello | key | line_comment' sample.yml

will output

    hello-world-comment

  Where is the comment - array example

The underlying yaml parser can assign comments in a document to surprising nodes. Use an expression like this to find where you comment is. `p' indicates the path, `isKey' is if the node is a map key (as opposed to a map value). From this, you can see the `under-name-comment' is actually on the first child

Given a sample.yml file of:

    name:
      # under-name-comment
      - first-array-child

then

    yq '[... | {"p": path | join("."), "isKey": is_key, "hc": headComment, "lc": lineComment, "fc": footComment}]' sample.yml

will output

    - p: ""
      isKey: false
      hc: ""
      lc: ""
      fc: ""
    - p: name
      isKey: true
      hc: ""
      lc: ""
      fc: ""
    - p: name
      isKey: false
      hc: ""
      lc: ""
      fc: ""
    - p: name.0
      isKey: false
      hc: under-name-comment
      lc: ""
      fc: ""

  Retrieve comment - array example

From the previous example, we know that the comment is on the first child as a headComment

Given a sample.yml file of:

    name:
      # under-name-comment
      - first-array-child

then

    yq '.name[0] | headComment' sample.yml

will output

    under-name-comment

  Set head comment

Given a sample.yml file of:

    a: cat

then

    yq '. head_comment="single"' sample.yml

will output

    # single
    a: cat

  Set head comment of a map entry

Given a sample.yml file of:

    f: foo
    a:
      b: cat

then

    yq '(.a | key) head_comment="single"' sample.yml

will output

    f: foo
    # single
    a:
      b: cat

  Set foot comment, using an expression

Given a sample.yml file of:

    a: cat

then

    yq '. foot_comment=.a' sample.yml

will output

    a: cat
    # cat

  Remove comment

Given a sample.yml file of:

    a: cat # comment
    b: dog # leave this

then

    yq '.a line_comment=""' sample.yml

will output

    a: cat
    b: dog # leave this

  Remove (strip) all comments

Note the use of ... to ensure key nodes are included.

Given a sample.yml file of:

    a: cat # comment
    # great
    b: # key comment

then

    yq '... comments=""' sample.yml

will output

    a: cat
    b:

  Get line comment

Given a sample.yml file of:

    a: cat # meow
    # have a great day

then

    yq '.a | line_comment' sample.yml

will output

    meow

  Get head comment

Given a sample.yml file of:

    # welcome!
    a: cat # meow
    # have a great day

then

    yq '. | head_comment' sample.yml

will output

    welcome!

  Head comment with document split

Given a sample.yml file of:

    # welcome!
    ---
    # bob
    a: cat # meow
    # have a great day

then

    yq 'head_comment' sample.yml

will output

    welcome!
    bob

  Get foot comment

Given a sample.yml file of:

    # welcome!
    a: cat # meow
    # have a great day
    # no really

then

    yq '. | foot_comment' sample.yml

will output

    have a great day
    no really

Compare Operators

Comparison operators (>, >=, <, <=) can be used for comparing scalar values of the same time.

The following types are currently supported:

•   numbers

•   strings

•   datetimes

  Related Operators

•   equals / not equals (==, !=) operators here (https://mikefarah.gitbook.io/yq/operators/equals)

•   boolean operators (and, or, any etc) here (https://mikefarah.gitbook.io/yq/operators/boolean-operators)

•   select operator here (https://mikefarah.gitbook.io/yq/operators/select)

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Compare numbers (>)

Given a sample.yml file of:

    a: 5
    b: 4

then

    yq '.a > .b' sample.yml

will output

    true

  Compare equal numbers (>=)

Given a sample.yml file of:

    a: 5
    b: 5

then

    yq '.a >= .b' sample.yml

will output

    true

  Compare strings

Compares strings by their bytecode.

Given a sample.yml file of:

    a: zoo
    b: apple

then

    yq '.a > .b' sample.yml

will output

    true

  Compare date times

You can compare date times. Assumes RFC3339 date time format, see date-time operators (https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information.

Given a sample.yml file of:

    a: 2021-01-01T03:10:00Z
    b: 2020-01-01T03:10:00Z

then

    yq '.a > .b' sample.yml

will output

    true

  Both sides are null: > is false

Running

    yq --null-input '.a > .b'

will output

    false

  Both sides are null: >= is true

Running

    yq --null-input '.a >= .b'

will output

    true

Contains

This returns true if the context contains the passed in parameter, and false otherwise.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Array contains array

Array is equal or subset of

Given a sample.yml file of:

    - foobar
    - foobaz
    - blarp

then

    yq 'contains(["baz", "bar"])' sample.yml

will output

    true

  Object included in array

Given a sample.yml file of:

    "foo": 12
    "bar":
      - 1
      - 2
      - "barp": 12
        "blip": 13

then

    yq 'contains({"bar": [{"barp": 12}]})' sample.yml

will output

    true

  Object not included in array

Given a sample.yml file of:

    "foo": 12
    "bar":
      - 1
      - 2
      - "barp": 12
        "blip": 13

then

    yq 'contains({"foo": 12, "bar": [{"barp": 15}]})' sample.yml

will output

    false

  String contains substring

Given a sample.yml file of:

    foobar

then

    yq 'contains("bar")' sample.yml

will output

    true

  String equals string

Given a sample.yml file of:

    meow

then

    yq 'contains("meow")' sample.yml

will output

    true

Create, Collect into Object

This is used to construct objects (or maps). This can be used against existing yaml, or to create fresh yaml documents.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Collect empty object

Running

    yq --null-input '{}'

will output

    {}

  Wrap (prefix) existing object

Given a sample.yml file of:

    name: Mike

then

    yq '{"wrap": .}' sample.yml

will output

    wrap:
      name: Mike

  Using splat to create multiple objects

Given a sample.yml file of:

    name: Mike
    pets:
      - cat
      - dog

then

    yq '{.name: .pets.[]}' sample.yml

will output

    Mike: cat
    Mike: dog

  Working with multiple documents

Given a sample.yml file of:

    name: Mike
    pets:
      - cat
      - dog
    ---
    name: Rosey
    pets:
      - monkey
      - sheep

then

    yq '{.name: .pets.[]}' sample.yml

will output

    Mike: cat
    Mike: dog
    Rosey: monkey
    Rosey: sheep

  Creating yaml from scratch

Running

    yq --null-input '{"wrap": "frog"}'

will output

    wrap: frog

  Creating yaml from scratch with multiple objects

Running

    yq --null-input '(.a.b = "foo") | (.d.e = "bar")'

will output

    a:
      b: foo
    d:
      e: bar

Date Time

Various operators for parsing and manipulating dates.

  Date time formattings

This uses the golangs built in time library for parsing and formatting date times.

When not specified, the RFC3339 standard is assumed 2006-01-02T15:04:05Z07:00 for parsing.

To specify a custom parsing format, use the with_dtf operator. The first parameter sets the datetime parsing format for the expression in the second parameter. The expression can be any valid yq expression tree.

    yq 'with_dtf("myformat"; .a + "3h" | tz("Australia/Melbourne"))'

See https://pkg.go.dev/time#pkg-constants for examples of formatting options.

  Timezones

This uses golangs built in LoadLocation function to parse timezones strings. See https://pkg.go.dev/time#LoadLocation for more details.

  Durations

Durations are parsed using golangs built in ParseDuration (https://pkg.go.dev/time#ParseDuration) function.

You can durations to time using the + operator.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Format: from standard RFC3339 format

Providing a single parameter assumes a standard RFC3339 datetime format. If the target format is not a valid yaml datetime format, the result will be a string tagged node.

Given a sample.yml file of:

    a: 2001-12-15T02:59:43.1Z

then

    yq '.a |= format_datetime("Monday, 02-Jan-06 at 3:04PM")' sample.yml

will output

    a: Saturday, 15-Dec-01 at 2:59AM

  Format: from custom date time

Use with_dtf to set a custom datetime format for parsing.

Given a sample.yml file of:

    a: Saturday, 15-Dec-01 at 2:59AM

then

    yq '.a |= with_dtf("Monday, 02-Jan-06 at 3:04PM"; format_datetime("2006-01-02"))' sample.yml

will output

    a: 2001-12-15

  Format: get the day of the week

Given a sample.yml file of:

    a: 2001-12-15T02:59:43.1Z

then

    yq '.a | format_datetime("Monday")' sample.yml

will output

    Saturday

  Now

Given a sample.yml file of:

    a: cool

then

    yq '.updated = now' sample.yml

will output

    a: cool
    updated: 2021-05-19T01:02:03Z

  Timezone: from standard RFC3339 format

Returns a new datetime in the specified timezone. Specify standard IANA Time Zone format or `utc', `local'. When given a single parameter, this assumes the datetime is in RFC3339 format.

Given a sample.yml file of:

    a: cool

then

    yq '.updated = (now | tz("Australia/Sydney"))' sample.yml

will output

    a: cool
    updated: 2021-05-19T11:02:03+10:00

  Timezone: with custom format

Specify standard IANA Time Zone format or `utc', `local'

Given a sample.yml file of:

    a: Saturday, 15-Dec-01 at 2:59AM GMT

then

    yq '.a |= with_dtf("Monday, 02-Jan-06 at 3:04PM MST"; tz("Australia/Sydney"))' sample.yml

will output

    a: Saturday, 15-Dec-01 at 1:59PM AEDT

  Add and tz custom format

Specify standard IANA Time Zone format or `utc', `local'

Given a sample.yml file of:

    a: Saturday, 15-Dec-01 at 2:59AM GMT

then

    yq '.a |= with_dtf("Monday, 02-Jan-06 at 3:04PM MST"; tz("Australia/Sydney"))' sample.yml

will output

    a: Saturday, 15-Dec-01 at 1:59PM AEDT

  Date addition

Given a sample.yml file of:

    a: 2021-01-01T00:00:00Z

then

    yq '.a += "3h10m"' sample.yml

will output

    a: 2021-01-01T03:10:00Z

  Date subtraction

You can subtract durations from dates. Assumes RFC3339 date time format, see date-time operators (https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information.

Given a sample.yml file of:

    a: 2021-01-01T03:10:00Z

then

    yq '.a -= "3h10m"' sample.yml

will output

    a: 2021-01-01T00:00:00Z

  Date addition - custom format

Given a sample.yml file of:

    a: Saturday, 15-Dec-01 at 2:59AM GMT

then

    yq 'with_dtf("Monday, 02-Jan-06 at 3:04PM MST"; .a += "3h1m")' sample.yml

will output

    a: Saturday, 15-Dec-01 at 6:00AM GMT

  Date script with custom format

You can embed full expressions in with_dtf if needed.

Given a sample.yml file of:

    a: Saturday, 15-Dec-01 at 2:59AM GMT

then

    yq 'with_dtf("Monday, 02-Jan-06 at 3:04PM MST"; .a = (.a + "3h1m" | tz("Australia/Perth")))' sample.yml

will output

    a: Saturday, 15-Dec-01 at 2:00PM AWST

Delete

Deletes matching entries in maps or arrays.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Delete entry in map

Given a sample.yml file of:

    a: cat
    b: dog

then

    yq 'del(.b)' sample.yml

will output

    a: cat

  Delete nested entry in map

Given a sample.yml file of:

    a:
      a1: fred
      a2: frood

then

    yq 'del(.a.a1)' sample.yml

will output

    a:
      a2: frood

  Delete entry in array

Given a sample.yml file of:

    - 1
    - 2
    - 3

then

    yq 'del(.[1])' sample.yml

will output

    - 1
    - 3

  Delete nested entry in array

Given a sample.yml file of:

    - a: cat
      b: dog

then

    yq 'del(.[0].a)' sample.yml

will output

    - b: dog

  Delete no matches

Given a sample.yml file of:

    a: cat
    b: dog

then

    yq 'del(.c)' sample.yml

will output

    a: cat
    b: dog

  Delete matching entries

Given a sample.yml file of:

    a: cat
    b: dog
    c: bat

then

    yq 'del( .[] | select(. == "*at") )' sample.yml

will output

    b: dog

  Recursively delete matching keys

Given a sample.yml file of:

    a:
      name: frog
      b:
        name: blog
        age: 12

then

    yq 'del(.. | select(has("name")).name)' sample.yml

will output

    a:
      b:
        age: 12

Document Index

Use the documentIndex operator (or the di shorthand) to select nodes of a particular document.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Retrieve a document index

Given a sample.yml file of:

    a: cat
    ---
    a: frog

then

    yq '.a | document_index' sample.yml

will output

    0
    ---
    1

  Retrieve a document index, shorthand

Given a sample.yml file of:

    a: cat
    ---
    a: frog

then

    yq '.a | di' sample.yml

will output

    0
    ---
    1

  Filter by document index

Given a sample.yml file of:

    a: cat
    ---
    a: frog

then

    yq 'select(document_index == 1)' sample.yml

will output

    a: frog

  Filter by document index shorthand

Given a sample.yml file of:

    a: cat
    ---
    a: frog

then

    yq 'select(di == 1)' sample.yml

will output

    a: frog

  Print Document Index with matches

Given a sample.yml file of:

    a: cat
    ---
    a: frog

then

    yq '.a | ({"match": ., "doc": document_index})' sample.yml

will output

    match: cat
    doc: 0
    match: frog
    doc: 1

Encoder / Decoder

Encode operators will take the piped in object structure and encode it as a string in the desired format. The decode operators do the opposite, they take a formatted string and decode it into the relevant object structure.

Note that you can optionally pass an indent value to the encode functions (see below).

These operators are useful to process yaml documents that have stringified embedded yaml/json/props in them.

Format | Decode (from string) | Encode (to string)
Yaml | from_yaml/@yamld | to_yaml(i)/@yaml
JSON | from_json/@jsond | to_json(i)/@json
Properties | from_props/@propsd | to_props/@props
CSV | from_csv/@csvd | to_csv/@csv
TSV | from_tsv/@tsvd | to_tsv/@tsv
XML | from_xml/@xmld | to_xml(i)/@xml
Base64 | @base64d | @base64

See CSV and TSV documentation (https://mikefarah.gitbook.io/yq/usage/csv-tsv) for accepted formats.

XML uses the --xml-attribute-prefix and xml-content-name flags to identify attributes and content fields.

Base64 assumes rfc4648 (https://rfc-editor.org/rfc/rfc4648.html) encoding. Encoding and decoding both assume that the content is a string.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Encode value as json string

Given a sample.yml file of:

    a:
      cool: thing

then

    yq '.b = (.a | to_json)' sample.yml

will output

    a:
      cool: thing
    b: |
      {
        "cool": "thing"
      }

  Encode value as json string, on one line

Pass in a 0 indent to print json on a single line.

Given a sample.yml file of:

    a:
      cool: thing

then

    yq '.b = (.a | to_json(0))' sample.yml

will output

    a:
      cool: thing
    b: '{"cool":"thing"}'

  Encode value as json string, on one line shorthand

Pass in a 0 indent to print json on a single line.

Given a sample.yml file of:

    a:
      cool: thing

then

    yq '.b = (.a | @json)' sample.yml

will output

    a:
      cool: thing
    b: '{"cool":"thing"}'

  Decode a json encoded string

Keep in mind JSON is a subset of YAML. If you want idiomatic yaml, pipe through the style operator to clear out the JSON styling.

Given a sample.yml file of:

    a: '{"cool":"thing"}'

then

    yq '.a | from_json | ... style=""' sample.yml

will output

    cool: thing

  Encode value as props string

Given a sample.yml file of:

    a:
      cool: thing

then

    yq '.b = (.a | @props)' sample.yml

will output

    a:
      cool: thing
    b: |
      cool = thing

  Decode props encoded string

Given a sample.yml file of:

    a: |-
      cats=great
      dogs=cool as well

then

    yq '.a |= @propsd' sample.yml

will output

    a:
      cats: great
      dogs: cool as well

  Decode csv encoded string

Given a sample.yml file of:

    a: |-
      cats,dogs
      great,cool as well

then

    yq '.a |= @csvd' sample.yml

will output

    a:
      - cats: great
        dogs: cool as well

  Decode tsv encoded string

Given a sample.yml file of:

    a: |-
      cats  dogs
      great cool as well

then

    yq '.a |= @tsvd' sample.yml

will output

    a:
      - cats: great
        dogs: cool as well

  Encode value as yaml string

Indent defaults to 2

Given a sample.yml file of:

    a:
      cool:
        bob: dylan

then

    yq '.b = (.a | to_yaml)' sample.yml

will output

    a:
      cool:
        bob: dylan
    b: |
      cool:
        bob: dylan

  Encode value as yaml string, with custom indentation

You can specify the indentation level as the first parameter.

Given a sample.yml file of:

    a:
      cool:
        bob: dylan

then

    yq '.b = (.a | to_yaml(8))' sample.yml

will output

    a:
      cool:
        bob: dylan
    b: |
      cool:
              bob: dylan

  Decode a yaml encoded string

Given a sample.yml file of:

    a: 'foo: bar'

then

    yq '.b = (.a | from_yaml)' sample.yml

will output

    a: 'foo: bar'
    b:
      foo: bar

  Update a multiline encoded yaml string

Given a sample.yml file of:

    a: |
      foo: bar
      baz: dog

then

    yq '.a |= (from_yaml | .foo = "cat" | to_yaml)' sample.yml

will output

    a: |
      foo: cat
      baz: dog

  Update a single line encoded yaml string

Given a sample.yml file of:

    a: 'foo: bar'

then

    yq '.a |= (from_yaml | .foo = "cat" | to_yaml)' sample.yml

will output

    a: 'foo: cat'

  Encode array of scalars as csv string

Scalars are strings, numbers and booleans.

Given a sample.yml file of:

    - cat
    - thing1,thing2
    - true
    - 3.40

then

    yq '@csv' sample.yml

will output

    cat,"thing1,thing2",true,3.40

  Encode array of arrays as csv string

Given a sample.yml file of:

    - - cat
      - thing1,thing2
      - true
      - 3.40
    - - dog
      - thing3
      - false
      - 12

then

    yq '@csv' sample.yml

will output

    cat,"thing1,thing2",true,3.40
    dog,thing3,false,12

  Encode array of array scalars as tsv string

Scalars are strings, numbers and booleans.

Given a sample.yml file of:

    - - cat
      - thing1,thing2
      - true
      - 3.40
    - - dog
      - thing3
      - false
      - 12

then

    yq '@tsv' sample.yml

will output

    cat thing1,thing2   true    3.40
    dog thing3  false   12

  Encode value as xml string

Given a sample.yml file of:

    a:
      cool:
        foo: bar
        +id: hi

then

    yq '.a | to_xml' sample.yml

will output

    <cool id="hi">
      <foo>bar</foo>
    </cool>

  Encode value as xml string on a single line

Given a sample.yml file of:

    a:
      cool:
        foo: bar
        +id: hi

then

    yq '.a | @xml' sample.yml

will output

    <cool id="hi"><foo>bar</foo></cool>

  Encode value as xml string with custom indentation

Given a sample.yml file of:

    a:
      cool:
        foo: bar
        +id: hi

then

    yq '{"cat": .a | to_xml(1)}' sample.yml

will output

    cat: |
      <cool id="hi">
       <foo>bar</foo>
      </cool>

  Decode a xml encoded string

Given a sample.yml file of:

    a: <foo>bar</foo>

then

    yq '.b = (.a | from_xml)' sample.yml

will output

    a: <foo>bar</foo>
    b:
      foo: bar

  Encode a string to base64

Given a sample.yml file of:

    coolData: a special string

then

    yq '.coolData | @base64' sample.yml

will output

    YSBzcGVjaWFsIHN0cmluZw==

  Encode a yaml document to base64

Pipe through @yaml first to convert to a string, then use @base64 to encode it.

Given a sample.yml file of:

    a: apple

then

    yq '@yaml | @base64' sample.yml

will output

    YTogYXBwbGUK

  Decode a base64 encoded string

Decoded data is assumed to be a string.

Given a sample.yml file of:

    coolData: V29ya3Mgd2l0aCBVVEYtMTYg8J+Yig==

then

    yq '.coolData | @base64d' sample.yml

will output

    Works with UTF-16 😊

  Decode a base64 encoded yaml document

Pipe through from_yaml to parse the decoded base64 string as a yaml document.

Given a sample.yml file of:

    coolData: YTogYXBwbGUK

then

    yq '.coolData |= (@base64d | from_yaml)' sample.yml

will output

    coolData:
      a: apple

Entries

Similar to the same named functions in jq these functions convert to/from an object and an array of key-value pairs. This is most useful for performing operations on keys of maps.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  to_entries Map

Given a sample.yml file of:

    a: 1
    b: 2

then

    yq 'to_entries' sample.yml

will output

    - key: a
      value: 1
    - key: b
      value: 2

  to_entries Array

Given a sample.yml file of:

    - a
    - b

then

    yq 'to_entries' sample.yml

will output

    - key: 0
      value: a
    - key: 1
      value: b

  to_entries null

Given a sample.yml file of:

    null

then

    yq 'to_entries' sample.yml

will output

  from_entries map

Given a sample.yml file of:

    a: 1
    b: 2

then

    yq 'to_entries | from_entries' sample.yml

will output

    a: 1
    b: 2

  from_entries with numeric key indexes

from_entries always creates a map, even for numeric keys

Given a sample.yml file of:

    - a
    - b

then

    yq 'to_entries | from_entries' sample.yml

will output

    0: a
    1: b

  Use with_entries to update keys

Given a sample.yml file of:

    a: 1
    b: 2

then

    yq 'with_entries(.key |= "KEY_" + .)' sample.yml

will output

    KEY_a: 1
    KEY_b: 2

  Use with_entries to filter the map

Given a sample.yml file of:

    a:
      b: bird
    c:
      d: dog

then

    yq 'with_entries(select(.value | has("b")))' sample.yml

will output

    a:
      b: bird

Env Variable Operators

These operators are used to handle environment variables usage in expressions and documents. While environment variables can, of course, be passed in via your CLI with string interpolation, this often comes with complex quote escaping and can be tricky to write and read.

There are three operators:

•   env which takes a single environment variable name and parse the variable as a yaml node (be it a map, array, string, number of boolean)

•   strenv which also takes a single environment variable name, and always parses the variable as a string.

•   envsubst which you pipe strings into and it interpolates environment variables in strings using envsubst (https://github.com/a8m/envsubst).

  EnvSubst Options

You can optionally pass envsubst any of the following options:

•   nu: NoUnset, this will fail if there are any referenced variables that are not set

•   ne: NoEmpty, this will fail if there are any referenced variables that are empty

•   ff: FailFast, this will abort on the first failure (rather than collect all the errors)

E.g: envsubst(ne, ff) will fail on the first empty variable.

See Imposing Restrictions (https://github.com/a8m/envsubst#imposing-restrictions) in the envsubst documentation for more information, and below for examples.

  Tip

To replace environment variables across all values in a document, envsubst can be used with the recursive descent operator as follows:

    yq '(.. | select(tag == "!!str")) |= envsubst' file.yaml

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Read string environment variable

Running

    myenv="cat meow" yq --null-input '.a = env(myenv)'

will output

    a: cat meow

  Read boolean environment variable

Running

    myenv="true" yq --null-input '.a = env(myenv)'

will output

    a: true

  Read numeric environment variable

Running

    myenv="12" yq --null-input '.a = env(myenv)'

will output

    a: 12

  Read yaml environment variable

Running

    myenv="{b: fish}" yq --null-input '.a = env(myenv)'

will output

    a: {b: fish}

  Read boolean environment variable as a string

Running

    myenv="true" yq --null-input '.a = strenv(myenv)'

will output

    a: "true"

  Read numeric environment variable as a string

Running

    myenv="12" yq --null-input '.a = strenv(myenv)'

will output

    a: "12"

  Dynamically update a path from an environment variable

The env variable can be any valid yq expression.

Given a sample.yml file of:

    a:
      b:
        - name: dog
        - name: cat

then

    pathEnv=".a.b[0].name"  valueEnv="moo" yq 'eval(strenv(pathEnv)) = strenv(valueEnv)' sample.yml

will output

    a:
      b:
        - name: moo
        - name: cat

  Dynamic key lookup with environment variable

Given a sample.yml file of:

    cat: meow
    dog: woof

then

    myenv="cat" yq '.[env(myenv)]' sample.yml

will output

    meow

  Replace strings with envsubst

Running

    myenv="cat" yq --null-input '"the ${myenv} meows" | envsubst'

will output

    the cat meows

  Replace strings with envsubst, missing variables

Running

    yq --null-input '"the ${myenvnonexisting} meows" | envsubst'

will output

    the  meows

  Replace strings with envsubst(nu), missing variables

(nu) not unset, will fail if there are unset (missing) variables

Running

    yq --null-input '"the ${myenvnonexisting} meows" | envsubst(nu)'

will output

    Error: variable ${myenvnonexisting} not set

  Replace strings with envsubst(ne), missing variables

(ne) not empty, only validates set variables

Running

    yq --null-input '"the ${myenvnonexisting} meows" | envsubst(ne)'

will output

    the  meows

  Replace strings with envsubst(ne), empty variable

(ne) not empty, will fail if a references variable is empty

Running

    myenv="" yq --null-input '"the ${myenv} meows" | envsubst(ne)'

will output

    Error: variable ${myenv} set but empty

  Replace strings with envsubst, missing variables with defaults

Running

    yq --null-input '"the ${myenvnonexisting-dog} meows" | envsubst'

will output

    the dog meows

  Replace strings with envsubst(nu), missing variables with defaults

Having a default specified skips over the missing variable.

Running

    yq --null-input '"the ${myenvnonexisting-dog} meows" | envsubst(nu)'

will output

    the dog meows

  Replace strings with envsubst(ne), missing variables with defaults

Fails, because the variable is explicitly set to blank.

Running

    myEmptyEnv="" yq --null-input '"the ${myEmptyEnv-dog} meows" | envsubst(ne)'

will output

    Error: variable ${myEmptyEnv} set but empty

  Replace string environment variable in document

Given a sample.yml file of:

    v: ${myenv}

then

    myenv="cat meow" yq '.v |= envsubst' sample.yml

will output

    v: cat meow

  (Default) Return all envsubst errors

By default, all errors are returned at once.

Running

    yq --null-input '"the ${notThere} ${alsoNotThere}" | envsubst(nu)'

will output

    Error: variable ${notThere} not set
    variable ${alsoNotThere} not set

  Fail fast, return the first envsubst error (and abort)

Running

    yq --null-input '"the ${notThere} ${alsoNotThere}" | envsubst(nu,ff)'

will output

    Error: variable ${notThere} not set

Equals / Not Equals

This is a boolean operator that will return true if the LHS is equal to the RHS and false otherwise.

    .a == .b

It is most often used with the select operator to find particular nodes:

    select(.a == .b)

The not equals != operator returns false if the LHS is equal to the RHS.

  Related Operators

•   comparison (>=, < etc) operators here (https://mikefarah.gitbook.io/yq/operators/compare)

•   boolean operators (and, or, any etc) here (https://mikefarah.gitbook.io/yq/operators/boolean-operators)

•   select operator here (https://mikefarah.gitbook.io/yq/operators/select)

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Match string

Given a sample.yml file of:

    - cat
    - goat
    - dog

then

    yq '.[] | (. == "*at")' sample.yml

will output

    true
    true
    false

  Don’t match string

Given a sample.yml file of:

    - cat
    - goat
    - dog

then

    yq '.[] | (. != "*at")' sample.yml

will output

    false
    false
    true

  Match number

Given a sample.yml file of:

    - 3
    - 4
    - 5

then

    yq '.[] | (. == 4)' sample.yml

will output

    false
    true
    false

  Don’t match number

Given a sample.yml file of:

    - 3
    - 4
    - 5

then

    yq '.[] | (. != 4)' sample.yml

will output

    true
    false
    true

  Match nulls

Running

    yq --null-input 'null == ~'

will output

    true

  Non existent key doesn’t equal a value

Given a sample.yml file of:

    a: frog

then

    yq 'select(.b != "thing")' sample.yml

will output

    a: frog

  Two non existent keys are equal

Given a sample.yml file of:

    a: frog

then

    yq 'select(.b == .c)' sample.yml

will output

    a: frog

Error

Use this operation to short-circuit expressions. Useful for validation.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Validate a particular value

Given a sample.yml file of:

    a: hello

then

    yq 'select(.a == "howdy") or error(".a [" + .a + "] is not howdy!")' sample.yml

will output

    Error: .a [hello] is not howdy!

  Validate the environment variable is a number - invalid

Running

    numberOfCats="please" yq --null-input 'env(numberOfCats) | select(tag == "!!int") or error("numberOfCats is not a number :(")'

will output

    Error: numberOfCats is not a number :(

  Validate the environment variable is a number - valid

with can be a convenient way of encapsulating validation.

Given a sample.yml file of:

    name: Bob
    favouriteAnimal: cat

then

    numberOfCats="3" yq '
        with(env(numberOfCats); select(tag == "!!int") or error("numberOfCats is not a number :(")) |
        .numPets = env(numberOfCats)
    ' sample.yml

will output

    name: Bob
    favouriteAnimal: cat
    numPets: 3

Eval

Use eval to dynamically process an expression - for instance from an environment variable.

eval takes a single argument, and evaluates that as a yq expression. Any valid expression can be used, beit a path .a.b.c | select(. == "cat"), or an update .a.b.c = "gogo".

Tip: This can be useful way parameterise complex scripts.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Dynamically evaluate a path

Given a sample.yml file of:

    pathExp: .a.b[] | select(.name == "cat")
    a:
      b:
        - name: dog
        - name: cat

then

    yq 'eval(.pathExp)' sample.yml

will output

    name: cat

  Dynamically update a path from an environment variable

The env variable can be any valid yq expression.

Given a sample.yml file of:

    a:
      b:
        - name: dog
        - name: cat

then

    pathEnv=".a.b[0].name"  valueEnv="moo" yq 'eval(strenv(pathEnv)) = strenv(valueEnv)' sample.yml

will output

    a:
      b:
        - name: moo
        - name: cat

File Operators

File operators are most often used with merge when needing to merge specific files together. Note that when doing this, you will need to use eval-all to ensure all yaml documents are loaded into memory before performing the merge (as opposed to eval which runs the expression once per document).

Note that the fileIndex operator has a short alias of fi.

  Merging files

Note the use of eval-all to ensure all documents are loaded into memory.

    yq eval-all 'select(fi == 0) * select(filename == "file2.yaml")' file1.yaml file2.yaml

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Get filename

Given a sample.yml file of:

    a: cat

then

    yq 'filename' sample.yml

will output

    sample.yml

  Get file index

Given a sample.yml file of:

    a: cat

then

    yq 'file_index' sample.yml

will output

    0

  Get file indices of multiple documents

Given a sample.yml file of:

    a: cat

And another sample another.yml file of:

    a: cat

then

    yq eval-all 'file_index' sample.yml another.yml

will output

    0
    ---
    1

  Get file index alias

Given a sample.yml file of:

    a: cat

then

    yq 'fi' sample.yml

will output

    0

Flatten

This recursively flattens arrays.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Flatten

Recursively flattens all arrays

Given a sample.yml file of:

    - 1
    - - 2
    - - - 3

then

    yq 'flatten' sample.yml

will output

    - 1
    - 2
    - 3

  Flatten with depth of one

Given a sample.yml file of:

    - 1
    - - 2
    - - - 3

then

    yq 'flatten(1)' sample.yml

will output

    - 1
    - 2
    - - 3

  Flatten empty array

Given a sample.yml file of:

    - []

then

    yq 'flatten' sample.yml

will output

    []

  Flatten array of objects

Given a sample.yml file of:

    - foo: bar
    - - foo: baz

then

    yq 'flatten' sample.yml

will output

    - foo: bar
    - foo: baz

Group By

This is used to group items in an array by an expression.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Group by field

Given a sample.yml file of:

    - foo: 1
      bar: 10
    - foo: 3
      bar: 100
    - foo: 1
      bar: 1

then

    yq 'group_by(.foo)' sample.yml

will output

    - - foo: 1
        bar: 10
      - foo: 1
        bar: 1
    - - foo: 3
        bar: 100

  Group by field, with nuls

Given a sample.yml file of:

    - cat: dog
    - foo: 1
      bar: 10
    - foo: 3
      bar: 100
    - no: foo for you
    - foo: 1
      bar: 1

then

    yq 'group_by(.foo)' sample.yml

will output

    - - cat: dog
      - no: foo for you
    - - foo: 1
        bar: 10
      - foo: 1
        bar: 1
    - - foo: 3
        bar: 100

Has

This is operation that returns true if the key exists in a map (or index in an array), false otherwise.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Has map key

Given a sample.yml file of:

    - a: yes
    - a: ~
    - a:
    - b: nope

then

    yq '.[] | has("a")' sample.yml

will output

    true
    true
    true
    false

  Select, checking for existence of deep paths

Simply pipe in parent expressions into has

Given a sample.yml file of:

    - a:
        b:
          c: cat
    - a:
        b:
          d: dog

then

    yq '.[] | select(.a.b | has("c"))' sample.yml

will output

    a:
      b:
        c: cat

  Has array index

Given a sample.yml file of:

    - []
    - [1]
    - [1, 2]
    - [1, null]
    - [1, 2, 3]

then

    yq '.[] | has(1)' sample.yml

will output

    false
    false
    true
    true
    true

Keys

Use the keys operator to return map keys or array indices.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Map keys

Given a sample.yml file of:

    dog: woof
    cat: meow

then

    yq 'keys' sample.yml

will output

    - dog
    - cat

  Array keys

Given a sample.yml file of:

    - apple
    - banana

then

    yq 'keys' sample.yml

will output

    - 0
    - 1

  Retrieve array key

Given a sample.yml file of:

    - 1
    - 2
    - 3

then

    yq '.[1] | key' sample.yml

will output

    1

  Retrieve map key

Given a sample.yml file of:

    a: thing

then

    yq '.a | key' sample.yml

will output

    a

  No key

Given a sample.yml file of:

    {}

then

    yq 'key' sample.yml

will output

  Update map key

Given a sample.yml file of:

    a:
      x: 3
      y: 4

then

    yq '(.a.x | key) = "meow"' sample.yml

will output

    a:
      meow: 3
      y: 4

  Get comment from map key

Given a sample.yml file of:

    a:
      # comment on key
      x: 3
      y: 4

then

    yq '.a.x | key | headComment' sample.yml

will output

    comment on key

  Check node is a key

Given a sample.yml file of:

    a:
      b:
        - cat
      c: frog

then

    yq '[... | { "p": path | join("."), "isKey": is_key, "tag": tag }]' sample.yml

will output

    - p: ""
      isKey: false
      tag: '!!map'
    - p: a
      isKey: true
      tag: '!!str'
    - p: a
      isKey: false
      tag: '!!map'
    - p: a.b
      isKey: true
      tag: '!!str'
    - p: a.b
      isKey: false
      tag: '!!seq'
    - p: a.b.0
      isKey: false
      tag: '!!str'
    - p: a.c
      isKey: true
      tag: '!!str'
    - p: a.c
      isKey: false
      tag: '!!str'

Length

Returns the lengths of the nodes. Length is defined according to the type of the node.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  String length

returns length of string

Given a sample.yml file of:

    a: cat

then

    yq '.a | length' sample.yml

will output

    3

  null length

Given a sample.yml file of:

    a: null

then

    yq '.a | length' sample.yml

will output

    0

  Map length

returns number of entries

Given a sample.yml file of:

    a: cat
    c: dog

then

    yq 'length' sample.yml

will output

    2

  Array length

returns number of elements

Given a sample.yml file of:

    - 2
    - 4
    - 6
    - 8

then

    yq 'length' sample.yml

will output

    4

Line

Returns the line of the matching node. Starts from 1, 0 indicates there was no line data.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Returns line of value node

Given a sample.yml file of:

    a: cat
    b:
      c: cat

then

    yq '.b | line' sample.yml

will output

    3

  Returns line of key node

Pipe through the key operator to get the line of the key

Given a sample.yml file of:

    a: cat
    b:
      c: cat

then

    yq '.b | key| line' sample.yml

will output

    2

  First line is 1

Given a sample.yml file of:

    a: cat

then

    yq '.a | line' sample.yml

will output

    1

  No line data is 0

Running

    yq --null-input '{"a": "new entry"} | line'

will output

    0

Load

The load operators allows you to load in content from another file.

Note that you can use string operators like + and sub to modify the value in the yaml file to a path that exists in your system.

You can load files of the following supported types:

Format | Load Operator
Yaml | load
XML | load_xml
Properties | load_props
Plain String | load_str
Base64 | load_base64

  Samples files for tests:

  yaml

../../examples/thing.yml:

    a: apple is included
    b: cool

  xml

small.xml:

    <this>is some xml</this>

  properties

small.properties:

    this.is = a properties file

  base64

base64.txt:

    bXkgc2VjcmV0IGNoaWxsaSByZWNpcGUgaXMuLi4u

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Simple example

Given a sample.yml file of:

    myFile: ../../examples/thing.yml

then

    yq 'load(.myFile)' sample.yml

will output

    a: apple is included
    b: cool.

  Replace node with referenced file

Note that you can modify the filename in the load operator if needed.

Given a sample.yml file of:

    something:
      file: thing.yml

then

    yq '.something |= load("../../examples/" + .file)' sample.yml

will output

    something:
      a: apple is included
      b: cool.

  Replace all nodes with referenced file

Recursively match all the nodes (..) and then filter the ones that have a `file' attribute.

Given a sample.yml file of:

    something:
      file: thing.yml
    over:
      here:
        - file: thing.yml

then

    yq '(.. | select(has("file"))) |= load("../../examples/" + .file)' sample.yml

will output

    something:
      a: apple is included
      b: cool.
    over:
      here:
        - a: apple is included
          b: cool.

  Replace node with referenced file as string

This will work for any text based file

Given a sample.yml file of:

    something:
      file: thing.yml

then

    yq '.something |= load_str("../../examples/" + .file)' sample.yml

will output

    something: |-
      a: apple is included
      b: cool.

  Load from XML

Given a sample.yml file of:

    cool: things

then

    yq '.more_stuff = load_xml("../../examples/small.xml")' sample.yml

will output

    cool: things
    more_stuff:
      this: is some xml

  Load from Properties

Given a sample.yml file of:

    cool: things

then

    yq '.more_stuff = load_props("../../examples/small.properties")' sample.yml

will output

    cool: things
    more_stuff:
      this:
        is: a properties file

  Merge from properties

This can be used as a convenient way to update a yaml document

Given a sample.yml file of:

    this:
      is: from yaml
      cool: ay

then

    yq '. *= load_props("../../examples/small.properties")' sample.yml

will output

    this:
      is: a properties file
      cool: ay

  Load from base64 encoded file

Given a sample.yml file of:

    cool: things

then

    yq '.more_stuff = load_base64("../../examples/base64.txt")' sample.yml

will output

    cool: things
    more_stuff: my secret chilli recipe is....

Map

Maps values of an array. Use map_values to map values of an object.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Map array

Given a sample.yml file of:

    - 1
    - 2
    - 3

then

    yq 'map(. + 1)' sample.yml

will output

    - 2
    - 3
    - 4

  Map object values

Given a sample.yml file of:

    a: 1
    b: 2
    c: 3

then

    yq 'map_values(. + 1)' sample.yml

will output

    a: 2
    b: 3
    c: 4

Multiply (Merge)

Like the multiple operator in jq, depending on the operands, this multiply operator will do different things. Currently numbers, arrays and objects are supported.

  Objects and arrays - merging

Objects are merged deeply matching on matching keys. By default, array values override and are not deeply merged.

Note that when merging objects, this operator returns the merged object (not the parent). This will be clearer in the examples below.

  Merge Flags

You can control how objects are merged by using one or more of the following flags. Multiple flags can be used together, e.g. .a *+? .b. See examples below

•   + append arrays

•   d deeply merge arrays

•   ? only merge existing fields

•   n only merge new fields

•   c clobber custom tags

  Merge two files together

This uses the load operator to merge file2 into file1.

    yq '. *= load("file2.yml")' file1.yml

  Merging all files

Note the use of eval-all to ensure all documents are loaded into memory.

    yq eval-all '. as $item ireduce ({}; . * $item )' *.yml

Merging complex arrays together by a key field

By default - yq merge is naive. It merges maps when they match the key name, and arrays are merged either by appending them together, or merging the entries by their position in the array.

For more complex array merging (e.g. merging items that match on a certain key) please see the example here (https://mikefarah.gitbook.io/yq/operators/multiply-merge#merge-arrays-of-objects-together-matching-on-a-key)

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Multiply integers

Given a sample.yml file of:

    a: 3
    b: 4

then

    yq '.a *= .b' sample.yml

will output

    a: 12
    b: 4

  Merge objects together, returning merged result only

Given a sample.yml file of:

    a:
      field: me
      fieldA: cat
    b:
      field:
        g: wizz
      fieldB: dog

then

    yq '.a * .b' sample.yml

will output

    field:
      g: wizz
    fieldA: cat
    fieldB: dog

  Merge objects together, returning parent object

Given a sample.yml file of:

    a:
      field: me
      fieldA: cat
    b:
      field:
        g: wizz
      fieldB: dog

then

    yq '. * {"a":.b}' sample.yml

will output

    a:
      field:
        g: wizz
      fieldA: cat
      fieldB: dog
    b:
      field:
        g: wizz
      fieldB: dog

  Merge keeps style of LHS

Given a sample.yml file of:

    a: {things: great}
    b:
      also: "me"

then

    yq '. * {"a":.b}' sample.yml

will output

    a: {things: great, also: "me"}
    b:
      also: "me"

  Merge arrays

Given a sample.yml file of:

    a:
      - 1
      - 2
      - 3
    b:
      - 3
      - 4
      - 5

then

    yq '. * {"a":.b}' sample.yml

will output

    a:
      - 3
      - 4
      - 5
    b:
      - 3
      - 4
      - 5

  Merge, only existing fields

Given a sample.yml file of:

    a:
      thing: one
      cat: frog
    b:
      missing: two
      thing: two

then

    yq '.a *? .b' sample.yml

will output

    thing: two
    cat: frog

  Merge, only new fields

Given a sample.yml file of:

    a:
      thing: one
      cat: frog
    b:
      missing: two
      thing: two

then

    yq '.a *n .b' sample.yml

will output

    thing: one
    cat: frog
    missing: two

  Merge, appending arrays

Given a sample.yml file of:

    a:
      array:
        - 1
        - 2
        - animal: dog
      value: coconut
    b:
      array:
        - 3
        - 4
        - animal: cat
      value: banana

then

    yq '.a *+ .b' sample.yml

will output

    array:
      - 1
      - 2
      - animal: dog
      - 3
      - 4
      - animal: cat
    value: banana

  Merge, only existing fields, appending arrays

Given a sample.yml file of:

    a:
      thing:
        - 1
        - 2
    b:
      thing:
        - 3
        - 4
      another:
        - 1

then

    yq '.a *?+ .b' sample.yml

will output

    thing:
      - 1
      - 2
      - 3
      - 4

  Merge, deeply merging arrays

Merging arrays deeply means arrays are merge like objects, with indexes as their key. In this case, we merge the first item in the array, and do nothing with the second.

Given a sample.yml file of:

    a:
      - name: fred
        age: 12
      - name: bob
        age: 32
    b:
      - name: fred
        age: 34

then

    yq '.a *d .b' sample.yml

will output

    - name: fred
      age: 34
    - name: bob
      age: 32

  Merge arrays of objects together, matching on a key

This is a fairly complex expression - you can use it as is by providing the environment variables as seen in the example below.

It merges in the array provided in the second file into the first - matching on equal keys.

Explanation:

The approach, at a high level, is to reduce into a merged map (keyed by the unique key) and then convert that back into an array.

First the expression will create a map from the arrays keyed by the idPath, the unique field we want to merge by. The reduce operator is merging `({}; . * $item )', so array elements with the matching key will be merged together.

Next, we convert the map back to an array, using reduce again, concatenating all the map values together.

Finally, we set the result of the merged array back into the first doc.

Thanks Kev from stackoverflow (https://stackoverflow.com/a/70109529/1168223)

Given a sample.yml file of:

    myArray:
      - a: apple
        b: appleB
      - a: kiwi
        b: kiwiB
      - a: banana
        b: bananaB
    something: else

And another sample another.yml file of:

    newArray:
      - a: banana
        c: bananaC
      - a: apple
        b: appleB2
      - a: dingo
        c: dingoC

then

    idPath=".a"  originalPath=".myArray"  otherPath=".newArray" yq eval-all '
    (
      (( (eval(strenv(originalPath)) + eval(strenv(otherPath)))  | .[] | {(eval(strenv(idPath))):  .}) as $item ireduce ({}; . * $item )) as $uniqueMap
      | ( $uniqueMap  | to_entries | .[]) as $item ireduce([]; . + $item.value)
    ) as $mergedArray
    | select(fi == 0) | (eval(strenv(originalPath))) = $mergedArray
    ' sample.yml another.yml

will output

    myArray:
      - a: apple
        b: appleB2
      - a: kiwi
        b: kiwiB
      - a: banana
        b: bananaB
        c: bananaC
      - a: dingo
        c: dingoC
    something: else

  Merge to prefix an element

Given a sample.yml file of:

    a: cat
    b: dog

then

    yq '. * {"a": {"c": .a}}' sample.yml

will output

    a:
      c: cat
    b: dog

  Merge with simple aliases

Given a sample.yml file of:

    a: &cat
      c: frog
    b:
      f: *cat
    c:
      g: thongs

then

    yq '.c * .b' sample.yml

will output

    g: thongs
    f: *cat

  Merge copies anchor names

Given a sample.yml file of:

    a:
      c: &cat frog
    b:
      f: *cat
    c:
      g: thongs

then

    yq '.c * .a' sample.yml

will output

    g: thongs
    c: &cat frog

  Merge with merge anchors

Given a sample.yml file of:

    foo: &foo
      a: foo_a
      thing: foo_thing
      c: foo_c
    bar: &bar
      b: bar_b
      thing: bar_thing
      c: bar_c
    foobarList:
      b: foobarList_b
      !!merge <<:
        - *foo
        - *bar
      c: foobarList_c
    foobar:
      c: foobar_c
      !!merge <<: *foo
      thing: foobar_thing

then

    yq '.foobar * .foobarList' sample.yml

will output

    c: foobarList_c
    !!merge <<:
      - *foo
      - *bar
    thing: foobar_thing
    b: foobarList_b

  Custom types: that are really numbers

When custom tags are encountered, yq will try to decode the underlying type.

Given a sample.yml file of:

    a: !horse 2
    b: !goat 3

then

    yq '.a = .a * .b' sample.yml

will output

    a: !horse 6
    b: !goat 3

  Custom types: that are really maps

Custom tags will be maintained.

Given a sample.yml file of:

    a: !horse
      cat: meow
    b: !goat
      dog: woof

then

    yq '.a = .a * .b' sample.yml

will output

    a: !horse
      cat: meow
      dog: woof
    b: !goat
      dog: woof

  Custom types: clobber tags

Use the c option to clobber custom tags. Note that the second tag is now used

Given a sample.yml file of:

    a: !horse
      cat: meow
    b: !goat
      dog: woof

then

    yq '.a *=c .b' sample.yml

will output

    a: !goat
      cat: meow
      dog: woof
    b: !goat
      dog: woof

Parent

Parent simply returns the parent nodes of the matching nodes.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Simple example

Given a sample.yml file of:

    a:
      nested: cat

then

    yq '.a.nested | parent' sample.yml

will output

    nested: cat

  Parent of nested matches

Given a sample.yml file of:

    a:
      fruit: apple
      name: bob
    b:
      fruit: banana
      name: sam

then

    yq '.. | select(. == "banana") | parent' sample.yml

will output

    fruit: banana
    name: sam

  No parent

Given a sample.yml file of:

    {}

then

    yq 'parent' sample.yml

will output

Path

The path operator can be used to get the traversal paths of matching nodes in an expression. The path is returned as an array, which if traversed in order will lead to the matching node.

You can get the key/index of matching nodes by using the path operator to return the path array then piping that through .[-1] to get the last element of that array, the key.

Use setpath to set a value to the path array returned by path, and similarly delpaths for an array of path arrays.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Map path

Given a sample.yml file of:

    a:
      b: cat

then

    yq '.a.b | path' sample.yml

will output

    - a
    - b

  Get map key

Given a sample.yml file of:

    a:
      b: cat

then

    yq '.a.b | path | .[-1]' sample.yml

will output

    b

  Array path

Given a sample.yml file of:

    a:
      - cat
      - dog

then

    yq '.a.[] | select(. == "dog") | path' sample.yml

will output

    - a
    - 1

  Get array index

Given a sample.yml file of:

    a:
      - cat
      - dog

then

    yq '.a.[] | select(. == "dog") | path | .[-1]' sample.yml

will output

    1

  Print path and value

Given a sample.yml file of:

    a:
      - cat
      - dog
      - frog

then

    yq '.a[] | select(. == "*og") | [{"path":path, "value":.}]' sample.yml

will output

    - path:
        - a
        - 1
      value: dog
    - path:
        - a
        - 2
      value: frog

  Set path

Given a sample.yml file of:

    a:
      b: cat

then

    yq 'setpath(["a", "b"]; "things")' sample.yml

will output

    a:
      b: things

  Set on empty document

Running

    yq --null-input 'setpath(["a", "b"]; "things")'

will output

    a:
      b: things

  Set array path

Given a sample.yml file of:

    a:
      - cat
      - frog

then

    yq 'setpath(["a", 0]; "things")' sample.yml

will output

    a:
      - things
      - frog

  Set array path empty

Running

    yq --null-input 'setpath(["a", 0]; "things")'

will output

    a:
      - things

  Delete path

Notice delpaths takes an array of paths.

Given a sample.yml file of:

    a:
      b: cat
      c: dog
      d: frog

then

    yq 'delpaths([["a", "c"], ["a", "d"]])' sample.yml

will output

    a:
      b: cat

  Delete array path

Given a sample.yml file of:

    a:
      - cat
      - frog

then

    yq 'delpaths([["a", 0]])' sample.yml

will output

    a:
      - frog

  Delete - wrong parameter

delpaths does not work with a single path array

Given a sample.yml file of:

    a:
      - cat
      - frog

then

    yq 'delpaths(["a", 0])' sample.yml

will output

    Error: DELPATHS: expected entry [0] to be a sequence, but its a !!str. Note that delpaths takes an array of path arrays, e.g. [["a", "b"]]

Pick

Filter a map by the specified list of keys. Map is returned with the key in the order of the pick list.

Similarly, filter an array by the specified list of indices.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Pick keys from map

Note that the order of the keys matches the pick order and non existent keys are skipped.

Given a sample.yml file of:

    myMap:
      cat: meow
      dog: bark
      thing: hamster
      hamster: squeek

then

    yq '.myMap |= pick(["hamster", "cat", "goat"])' sample.yml

will output

    myMap:
      hamster: squeek
      cat: meow

  Pick indices from array

Note that the order of the indexes matches the pick order and non existent indexes are skipped.

Given a sample.yml file of:

    - cat
    - leopard
    - lion

then

    yq 'pick([2, 0, 734, -5])' sample.yml

will output

    - lion
    - cat

Pipe

Pipe the results of an expression into another. Like the bash operator.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Simple Pipe

Given a sample.yml file of:

    a:
      b: cat

then

    yq '.a | .b' sample.yml

will output

    cat

  Multiple updates

Given a sample.yml file of:

    a: cow
    b: sheep
    c: same

then

    yq '.a = "cat" | .b = "dog"' sample.yml

will output

    a: cat
    b: dog
    c: same

Recursive Descent (Glob)

This operator recursively matches (or globs) all children nodes given of a particular element, including that node itself. This is most often used to apply a filter recursively against all matches. It can be used in either the

  match values form ..

This will, like the jq equivalent, recursively match all value nodes. Use it to find/manipulate particular values.

For instance to set the style of all value nodes in a yaml doc, excluding map keys:

    yq '.. style= "flow"' file.yaml

  match values and map keys form ...

The also includes map keys in the results set. This is particularly useful in YAML as unlike JSON, map keys can have their own styling, tags and use anchors and aliases.

For instance to set the style of all nodes in a yaml doc, including the map keys:

    yq '... style= "flow"' file.yaml

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Recurse map (values only)

Given a sample.yml file of:

    a: frog

then

    yq '..' sample.yml

will output

    a: frog
    frog

  Recursively find nodes with keys

Note that this example has wrapped the expression in [] to show that there are two matches returned. You do not have to wrap in [] in your path expression.

Given a sample.yml file of:

    a:
      name: frog
      b:
        name: blog
        age: 12

then

    yq '[.. | select(has("name"))]' sample.yml

will output

    - name: frog
      b:
        name: blog
        age: 12
    - name: blog
      age: 12

  Recursively find nodes with values

Given a sample.yml file of:

    a:
      nameA: frog
      b:
        nameB: frog
        age: 12

then

    yq '.. | select(. == "frog")' sample.yml

will output

    frog
    frog

  Recurse map (values and keys)

Note that the map key appears in the results

Given a sample.yml file of:

    a: frog

then

    yq '...' sample.yml

will output

    a: frog
    a
    frog

  Aliases are not traversed

Given a sample.yml file of:

    a: &cat
      c: frog
    b: *cat

then

    yq '[..]' sample.yml

will output

    - a: &cat
        c: frog
      b: *cat
    - &cat
      c: frog
    - frog
    - *cat

  Merge docs are not traversed

Given a sample.yml file of:

    foo: &foo
      a: foo_a
      thing: foo_thing
      c: foo_c
    bar: &bar
      b: bar_b
      thing: bar_thing
      c: bar_c
    foobarList:
      b: foobarList_b
      !!merge <<:
        - *foo
        - *bar
      c: foobarList_c
    foobar:
      c: foobar_c
      !!merge <<: *foo
      thing: foobar_thing

then

    yq '.foobar | [..]' sample.yml

will output

    - c: foobar_c
      !!merge <<: *foo
      thing: foobar_thing
    - foobar_c
    - *foo
    - foobar_thing

Reduce

Reduce is a powerful way to process a collection of data into a new form.

    <exp> as $<name> ireduce (<init>; <block>)

e.g.

    .[] as $item ireduce (0; . + $item)

On the LHS we are configuring the collection of items that will be reduced <exp> as well as what each element will be called $<name>. Note that the array has been splatted into its individual elements.

On the RHS there is <init>, the starting value of the accumulator and <block>, the expression that will update the accumulator for each element in the collection. Note that within the block expression, . will evaluate to the current value of the accumulator.

  yq vs jq syntax

Reduce syntax in yq is a little different from jq - as yq (currently) isn’t as sophisticated as jq and its only supports infix notation (e.g. a + b, where the operator is in the middle of the two parameters) - where as jq uses a mix of infix notation with prefix notation (e.g. reduce a b is like writing + a b).

To that end, the reduce operator is called ireduce for backwards compatibility if a jq like prefix version of reduce is ever added.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Sum numbers

Given a sample.yml file of:

    - 10
    - 2
    - 5
    - 3

then

    yq '.[] as $item ireduce (0; . + $item)' sample.yml

will output

    20

  Merge all yaml files together

Given a sample.yml file of:

    a: cat

And another sample another.yml file of:

    b: dog

then

    yq eval-all '. as $item ireduce ({}; . * $item )' sample.yml another.yml

will output

    a: cat
    b: dog

  Convert an array to an object

Given a sample.yml file of:

    - name: Cathy
      has: apples
    - name: Bob
      has: bananas

then

    yq '.[] as $item ireduce ({}; .[$item | .name] = ($item | .has) )' sample.yml

will output

    Cathy: apples
    Bob: bananas

Reverse

Reverses the order of the items in an array

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Reverse

Given a sample.yml file of:

    - 1
    - 2
    - 3

then

    yq 'reverse' sample.yml

will output

    - 3
    - 2
    - 1

  Sort descending by string field

Use sort with reverse to sort in descending order.

Given a sample.yml file of:

    - a: banana
    - a: cat
    - a: apple

then

    yq 'sort_by(.a) | reverse' sample.yml

will output

    - a: cat
    - a: banana
    - a: apple

Select

Select is used to filter arrays and maps by a boolean expression.

  Related Operators

•   equals / not equals (==, !=) operators here (https://mikefarah.gitbook.io/yq/operators/equals)

•   comparison (>=, < etc) operators here (https://mikefarah.gitbook.io/yq/operators/compare)

•   boolean operators (and, or, any etc) here (https://mikefarah.gitbook.io/yq/operators/boolean-operators)

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Select elements from array using wildcard prefix

Given a sample.yml file of:

    - cat
    - goat
    - dog

then

    yq '.[] | select(. == "*at")' sample.yml

will output

    cat
    goat

  Select elements from array using wildcard suffix

Given a sample.yml file of:

    - go-kart
    - goat
    - dog

then

    yq '.[] | select(. == "go*")' sample.yml

will output

    go-kart
    goat

  Select elements from array using wildcard prefix and suffix

Given a sample.yml file of:

    - ago
    - go
    - meow
    - going

then

    yq '.[] | select(. == "*go*")' sample.yml

will output

    ago
    go
    going

  Select elements from array with regular expression

See more regular expression examples under the string operator docs.

Given a sample.yml file of:

    - this_0
    - not_this
    - nor_0_this
    - thisTo_4

then

    yq '.[] | select(test("[a-zA-Z]+_[0-9]$"))' sample.yml

will output

    this_0
    thisTo_4

  Select items from a map

Given a sample.yml file of:

    things: cat
    bob: goat
    horse: dog

then

    yq '.[] | select(. == "cat" or test("og$"))' sample.yml

will output

    cat
    dog

  Use select and with_entries to filter map keys

Given a sample.yml file of:

    name: bob
    legs: 2
    game: poker

then

    yq 'with_entries(select(.key | test("ame$")))' sample.yml

will output

    name: bob
    game: poker

  Select multiple items in a map and update

Note the brackets around the entire LHS.

Given a sample.yml file of:

    a:
      things: cat
      bob: goat
      horse: dog

then

    yq '(.a.[] | select(. == "cat" or . == "goat")) |= "rabbit"' sample.yml

will output

    a:
      things: rabbit
      bob: rabbit
      horse: dog

Sort Keys

The Sort Keys operator sorts maps by their keys (based on their string value). This operator does not do anything to arrays or scalars (so you can easily recursively apply it to all maps).

Sort is particularly useful for diffing two different yaml documents:

    yq -i -P 'sort_keys(..)' file1.yml
    yq -i -P 'sort_keys(..)' file2.yml
    diff file1.yml file2.yml

Note that yq does not yet consider anchors when sorting by keys - this may result in invalid yaml documents if your are using merge anchors.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Sort keys of map

Given a sample.yml file of:

    c: frog
    a: blah
    b: bing

then

    yq 'sort_keys(.)' sample.yml

will output

    a: blah
    b: bing
    c: frog

  Sort keys recursively

Note the array elements are left unsorted, but maps inside arrays are sorted

Given a sample.yml file of:

    bParent:
      c: dog
      array:
        - 3
        - 1
        - 2
    aParent:
      z: donkey
      x:
        - c: yum
          b: delish
        - b: ew
          a: apple

then

    yq 'sort_keys(..)' sample.yml

will output

    aParent:
      x:
        - b: delish
          c: yum
        - a: apple
          b: ew
      z: donkey
    bParent:
      array:
        - 3
        - 1
        - 2
      c: dog

Sort

Sorts an array. Use sort to sort an array as is, or sort_by(exp) to sort by a particular expression (e.g. subfield).

To sort by descending order, pipe the results through the reverse operator after sorting.

Note that at this stage, yq only sorts scalar fields.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Sort by string field

Given a sample.yml file of:

    - a: banana
    - a: cat
    - a: apple

then

    yq 'sort_by(.a)' sample.yml

will output

    - a: apple
    - a: banana
    - a: cat

  Sort descending by string field

Use sort with reverse to sort in descending order.

Given a sample.yml file of:

    - a: banana
    - a: cat
    - a: apple

then

    yq 'sort_by(.a) | reverse' sample.yml

will output

    - a: cat
    - a: banana
    - a: apple

  Sort array in place

Given a sample.yml file of:

    cool:
      - a: banana
      - a: cat
      - a: apple

then

    yq '.cool |= sort_by(.a)' sample.yml

will output

    cool:
      - a: apple
      - a: banana
      - a: cat

  Sort array of objects by key

Note that you can give sort_by complex expressions, not just paths

Given a sample.yml file of:

    cool:
      - b: banana
      - a: banana
      - c: banana

then

    yq '.cool |= sort_by(keys | .[0])' sample.yml

will output

    cool:
      - a: banana
      - b: banana
      - c: banana

  Sort is stable

Note the order of the elements in unchanged when equal in sorting.

Given a sample.yml file of:

    - a: banana
      b: 1
    - a: banana
      b: 2
    - a: banana
      b: 3
    - a: banana
      b: 4

then

    yq 'sort_by(.a)' sample.yml

will output

    - a: banana
      b: 1
    - a: banana
      b: 2
    - a: banana
      b: 3
    - a: banana
      b: 4

  Sort by numeric field

Given a sample.yml file of:

    - a: 10
    - a: 100
    - a: 1

then

    yq 'sort_by(.a)' sample.yml

will output

    - a: 1
    - a: 10
    - a: 100

  Sort, nulls come first

Given a sample.yml file of:

    - 8
    - 3
    - null
    - 6
    - true
    - false
    - cat

then

    yq 'sort' sample.yml

will output

    - null
    - false
    - true
    - 3
    - 6
    - 8
    - cat

Split into Documents

This operator splits all matches into separate documents

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Split empty

Running

    yq --null-input 'split_doc'

will output

  Split array

Given a sample.yml file of:

    - a: cat
    - b: dog

then

    yq '.[] | split_doc' sample.yml

will output

    a: cat
    ---
    b: dog

String Operators

  RegEx

This uses golangs native regex functions under the hood - See their docs (https://github.com/google/re2/wiki/Syntax) for the supported syntax.

Case insensitive tip: prefix the regex with (?i) - e.g. test("(?i)cats)".

  match(regEx)

This operator returns the substring match details of the given regEx.

  capture(regEx)

Capture returns named RegEx capture groups in a map. Can be more convenient than match depending on what you are doing.

  test(regEx)

Returns true if the string matches the RegEx, false otherwise.

  sub(regEx, replacement)

Substitutes matched substrings. The first parameter is the regEx to match substrings within the original string. The second is a what to replace those matches with. This can refer to capture groups from the first RegEx.

  String blocks, bash and newlines

Bash is notorious for chomping on precious trailing newline characters, making it tricky to set strings with newlines properly. In particular, the $( exp ) will trim trailing newlines.

For instance to get this yaml:

    a: |
      cat

Using $( exp ) wont work, as it will trim the trailing new line.

    m=$(echo "cat\n") yq -n '.a = strenv(m)'
    a: cat

However, using printf works:

    printf -v m "cat\n" ; m="$m" yq -n '.a = strenv(m)'
    a: |
      cat

As well as having multiline expressions:

    m="cat
    "  yq -n '.a = strenv(m)'
    a: |
      cat

Similarly, if you’re trying to set the content from a file, and want a trailing new line:

    IFS= read -rd '' output < <(cat my_file)
    output=$output ./yq '.data.values = strenv(output)' first.yml

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  To up (upper) case

Works with unicode characters

Given a sample.yml file of:

    água

then

    yq 'upcase' sample.yml

will output

    ÁGUA

  To down (lower) case

Works with unicode characters

Given a sample.yml file of:

    ÁgUA

then

    yq 'downcase' sample.yml

will output

    água

  Join strings

Given a sample.yml file of:

    - cat
    - meow
    - 1
    - null
    - true

then

    yq 'join("; ")' sample.yml

will output

    cat; meow; 1; ; true

  Trim strings

Given a sample.yml file of:

    - ' cat'
    - 'dog '
    - ' cow cow '
    - horse

then

    yq '.[] | trim' sample.yml

will output

    cat
    dog
    cow cow
    horse

  Match string

Given a sample.yml file of:

    foo bar foo

then

    yq 'match("foo")' sample.yml

will output

    string: foo
    offset: 0
    length: 3
    captures: []

  Match string, case insensitive

Given a sample.yml file of:

    foo bar FOO

then

    yq '[match("(?i)foo"; "g")]' sample.yml

will output

    - string: foo
      offset: 0
      length: 3
      captures: []
    - string: FOO
      offset: 8
      length: 3
      captures: []

  Match with global capture group

Given a sample.yml file of:

    abc abc

then

    yq '[match("(ab)(c)"; "g")]' sample.yml

will output

    - string: abc
      offset: 0
      length: 3
      captures:
        - string: ab
          offset: 0
          length: 2
        - string: c
          offset: 2
          length: 1
    - string: abc
      offset: 4
      length: 3
      captures:
        - string: ab
          offset: 4
          length: 2
        - string: c
          offset: 6
          length: 1

  Match with named capture groups

Given a sample.yml file of:

    foo bar foo foo  foo

then

    yq '[match("foo (?P<bar123>bar)? foo"; "g")]' sample.yml

will output

    - string: foo bar foo
      offset: 0
      length: 11
      captures:
        - string: bar
          offset: 4
          length: 3
          name: bar123
    - string: foo  foo
      offset: 12
      length: 8
      captures:
        - string: null
          offset: -1
          length: 0
          name: bar123

  Capture named groups into a map

Given a sample.yml file of:

    xyzzy-14

then

    yq 'capture("(?P<a>[a-z]+)-(?P<n>[0-9]+)")' sample.yml

will output

    a: xyzzy
    n: "14"

  Match without global flag

Given a sample.yml file of:

    cat cat

then

    yq 'match("cat")' sample.yml

will output

    string: cat
    offset: 0
    length: 3
    captures: []

  Match with global flag

Given a sample.yml file of:

    cat cat

then

    yq '[match("cat"; "g")]' sample.yml

will output

    - string: cat
      offset: 0
      length: 3
      captures: []
    - string: cat
      offset: 4
      length: 3
      captures: []

  Test using regex

Like jq’s equivalent, this works like match but only returns true/false instead of full match details

Given a sample.yml file of:

    - cat
    - dog

then

    yq '.[] | test("at")' sample.yml

will output

    true
    false

  Substitute / Replace string

This uses golang regex, described here (https://github.com/google/re2/wiki/Syntax) Note the use of |= to run in context of the current string value.

Given a sample.yml file of:

    a: dogs are great

then

    yq '.a |= sub("dogs", "cats")' sample.yml

will output

    a: cats are great

  Substitute / Replace string with regex

This uses golang regex, described here (https://github.com/google/re2/wiki/Syntax) Note the use of |= to run in context of the current string value.

Given a sample.yml file of:

    a: cat
    b: heat

then

    yq '.[] |= sub("(a)", "${1}r")' sample.yml

will output

    a: cart
    b: heart

  Custom types: that are really strings

When custom tags are encountered, yq will try to decode the underlying type.

Given a sample.yml file of:

    a: !horse cat
    b: !goat heat

then

    yq '.[] |= sub("(a)", "${1}r")' sample.yml

will output

    a: !horse cart
    b: !goat heart

  Split strings

Given a sample.yml file of:

    cat; meow; 1; ; true

then

    yq 'split("; ")' sample.yml

will output

    - cat
    - meow
    - "1"
    - ""
    - "true"

  Split strings one match

Given a sample.yml file of:

    word

then

    yq 'split("; ")' sample.yml

will output

    - word

Style

The style operator can be used to get or set the style of nodes (e.g. string style, yaml style)

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Update and set style of a particular node (simple)

Given a sample.yml file of:

    a:
      b: thing
      c: something

then

    yq '.a.b = "new" | .a.b style="double"' sample.yml

will output

    a:
      b: "new"
      c: something

  Update and set style of a particular node using path variables

Given a sample.yml file of:

    a:
      b: thing
      c: something

then

    yq 'with(.a.b ; . = "new" | . style="double")' sample.yml

will output

    a:
      b: "new"
      c: something

  Set tagged style

Given a sample.yml file of:

    a: cat
    b: 5
    c: 3.2
    e: true

then

    yq '.. style="tagged"' sample.yml

will output

    !!map
    a: !!str cat
    b: !!int 5
    c: !!float 3.2
    e: !!bool true

  Set double quote style

Given a sample.yml file of:

    a: cat
    b: 5
    c: 3.2
    e: true

then

    yq '.. style="double"' sample.yml

will output

    a: "cat"
    b: "5"
    c: "3.2"
    e: "true"

  Set double quote style on map keys too

Given a sample.yml file of:

    a: cat
    b: 5
    c: 3.2
    e: true

then

    yq '... style="double"' sample.yml

will output

    "a": "cat"
    "b": "5"
    "c": "3.2"
    "e": "true"

  Set single quote style

Given a sample.yml file of:

    a: cat
    b: 5
    c: 3.2
    e: true

then

    yq '.. style="single"' sample.yml

will output

    a: 'cat'
    b: '5'
    c: '3.2'
    e: 'true'

  Set literal quote style

Given a sample.yml file of:

    a: cat
    b: 5
    c: 3.2
    e: true

then

    yq '.. style="literal"' sample.yml

will output

    a: |-
      cat
    b: |-
      5
    c: |-
      3.2
    e: |-
      true

  Set folded quote style

Given a sample.yml file of:

    a: cat
    b: 5
    c: 3.2
    e: true

then

    yq '.. style="folded"' sample.yml

will output

    a: >-
      cat
    b: >-
      5
    c: >-
      3.2
    e: >-
      true

  Set flow quote style

Given a sample.yml file of:

    a: cat
    b: 5
    c: 3.2
    e: true

then

    yq '.. style="flow"' sample.yml

will output

    {a: cat, b: 5, c: 3.2, e: true}

  Reset style - or pretty print

Set empty (default) quote style, note the usage of ... to match keys too. Note that there is a --prettyPrint/-P short flag for this.

Given a sample.yml file of:

    a: cat
    "b": 5
    'c': 3.2
    "e": true

then

    yq '... style=""' sample.yml

will output

    a: cat
    b: 5
    c: 3.2
    e: true

  Set style relatively with assign-update

Given a sample.yml file of:

    a: single
    b: double

then

    yq '.[] style |= .' sample.yml

will output

    a: 'single'
    b: "double"

  Read style

Given a sample.yml file of:

    {a: "cat", b: 'thing'}

then

    yq '.. | style' sample.yml

will output

    flow
    double
    single

Subtract

You can use subtract to subtract numbers, as well as removing elements from an array.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Array subtraction

Running

    yq --null-input '[1,2] - [2,3]'

will output

    - 1

  Array subtraction with nested array

Running

    yq --null-input '[[1], 1, 2] - [[1], 3]'

will output

    - 1
    - 2

  Array subtraction with nested object

Note that order of the keys does not matter

Given a sample.yml file of:

    - a: b
      c: d
    - a: b

then

    yq '. - [{"c": "d", "a": "b"}]' sample.yml

will output

    - a: b

  Number subtraction - float

If the lhs or rhs are floats then the expression will be calculated with floats.

Given a sample.yml file of:

    a: 3
    b: 4.5

then

    yq '.a = .a - .b' sample.yml

will output

    a: -1.5
    b: 4.5

  Number subtraction - float

If the lhs or rhs are floats then the expression will be calculated with floats.

Given a sample.yml file of:

    a: 3
    b: 4.5

then

    yq '.a = .a - .b' sample.yml

will output

    a: -1.5
    b: 4.5

  Number subtraction - int

If both the lhs and rhs are ints then the expression will be calculated with ints.

Given a sample.yml file of:

    a: 3
    b: 4

then

    yq '.a = .a - .b' sample.yml

will output

    a: -1
    b: 4

  Decrement numbers

Given a sample.yml file of:

    a: 3
    b: 5

then

    yq '.[] -= 1' sample.yml

will output

    a: 2
    b: 4

  Date subtraction

You can subtract durations from dates. Assumes RFC3339 date time format, see date-time operators (https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information.

Given a sample.yml file of:

    a: 2021-01-01T03:10:00Z

then

    yq '.a -= "3h10m"' sample.yml

will output

    a: 2021-01-01T00:00:00Z

  Date subtraction - custom format

Use with_dtf to specify your datetime format. See date-time operators (https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information.

Given a sample.yml file of:

    a: Saturday, 15-Dec-01 at 6:00AM GMT

then

    yq 'with_dtf("Monday, 02-Jan-06 at 3:04PM MST", .a -= "3h1m")' sample.yml

will output

    a: Saturday, 15-Dec-01 at 2:59AM GMT

  Custom types: that are really numbers

When custom tags are encountered, yq will try to decode the underlying type.

Given a sample.yml file of:

    a: !horse 2
    b: !goat 1

then

    yq '.a -= .b' sample.yml

will output

    a: !horse 1
    b: !goat 1

Tag

The tag operator can be used to get or set the tag of nodes (e.g. !!str, !!int, !!bool).

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Get tag

Given a sample.yml file of:

    a: cat
    b: 5
    c: 3.2
    e: true
    f: []

then

    yq '.. | tag' sample.yml

will output

    !!map
    !!str
    !!int
    !!float
    !!bool
    !!seq

  type is an alias for tag

Given a sample.yml file of:

    a: cat
    b: 5
    c: 3.2
    e: true
    f: []

then

    yq '.. | type' sample.yml

will output

    !!map
    !!str
    !!int
    !!float
    !!bool
    !!seq

  Set custom tag

Given a sample.yml file of:

    a: str

then

    yq '.a tag = "!!mikefarah"' sample.yml

will output

    a: !!mikefarah str

  Find numbers and convert them to strings

Given a sample.yml file of:

    a: cat
    b: 5
    c: 3.2
    e: true

then

    yq '(.. | select(tag == "!!int")) tag= "!!str"' sample.yml

will output

    a: cat
    b: "5"
    c: 3.2
    e: true

Traverse (Read)

This is the simplest (and perhaps most used) operator, it is used to navigate deeply into yaml structures.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Simple map navigation

Given a sample.yml file of:

    a:
      b: apple

then

    yq '.a' sample.yml

will output

    b: apple

  Splat

Often used to pipe children into other operators

Given a sample.yml file of:

    - b: apple
    - c: banana

then

    yq '.[]' sample.yml

will output

    b: apple
    c: banana

  Optional Splat

Just like splat, but won’t error if you run it against scalars

Given a sample.yml file of:

    cat

then

    yq '.[]' sample.yml

will output

  Special characters

Use quotes with brackets around path elements with special characters

Given a sample.yml file of:

    "{}": frog

then

    yq '.["{}"]' sample.yml

will output

    frog

  Nested special characters

Given a sample.yml file of:

    a:
      "key.withdots":
        "another.key": apple

then

    yq '.a["key.withdots"]["another.key"]' sample.yml

will output

    apple

  Keys with spaces

Use quotes with brackets around path elements with special characters

Given a sample.yml file of:

    "red rabbit": frog

then

    yq '.["red rabbit"]' sample.yml

will output

    frog

  Dynamic keys

Expressions within [] can be used to dynamically lookup / calculate keys

Given a sample.yml file of:

    b: apple
    apple: crispy yum
    banana: soft yum

then

    yq '.[.b]' sample.yml

will output

    crispy yum

  Children don’t exist

Nodes are added dynamically while traversing

Given a sample.yml file of:

    c: banana

then

    yq '.a.b' sample.yml

will output

    null

  Optional identifier

Like jq, does not output an error when the yaml is not an array or object as expected

Given a sample.yml file of:

    - 1
    - 2
    - 3

then

    yq '.a?' sample.yml

will output

  Wildcard matching

Given a sample.yml file of:

    a:
      cat: apple
      mad: things

then

    yq '.a."*a*"' sample.yml

will output

    apple
    things

  Aliases

Given a sample.yml file of:

    a: &cat
      c: frog
    b: *cat

then

    yq '.b' sample.yml

will output

    *cat

  Traversing aliases with splat

Given a sample.yml file of:

    a: &cat
      c: frog
    b: *cat

then

    yq '.b[]' sample.yml

will output

    frog

  Traversing aliases explicitly

Given a sample.yml file of:

    a: &cat
      c: frog
    b: *cat

then

    yq '.b.c' sample.yml

will output

    frog

  Traversing arrays by index

Given a sample.yml file of:

    - 1
    - 2
    - 3

then

    yq '.[0]' sample.yml

will output

    1

  Traversing nested arrays by index

Given a sample.yml file of:

    [[], [cat]]

then

    yq '.[1][0]' sample.yml

will output

    cat

  Maps with numeric keys

Given a sample.yml file of:

    2: cat

then

    yq '.[2]' sample.yml

will output

    cat

  Maps with non existing numeric keys

Given a sample.yml file of:

    a: b

then

    yq '.[0]' sample.yml

will output

    null

  Traversing merge anchors

Given a sample.yml file of:

    foo: &foo
      a: foo_a
      thing: foo_thing
      c: foo_c
    bar: &bar
      b: bar_b
      thing: bar_thing
      c: bar_c
    foobarList:
      b: foobarList_b
      !!merge <<:
        - *foo
        - *bar
      c: foobarList_c
    foobar:
      c: foobar_c
      !!merge <<: *foo
      thing: foobar_thing

then

    yq '.foobar.a' sample.yml

will output

    foo_a

  Traversing merge anchors with override

Given a sample.yml file of:

    foo: &foo
      a: foo_a
      thing: foo_thing
      c: foo_c
    bar: &bar
      b: bar_b
      thing: bar_thing
      c: bar_c
    foobarList:
      b: foobarList_b
      !!merge <<:
        - *foo
        - *bar
      c: foobarList_c
    foobar:
      c: foobar_c
      !!merge <<: *foo
      thing: foobar_thing

then

    yq '.foobar.c' sample.yml

will output

    foo_c

  Traversing merge anchors with local override

Given a sample.yml file of:

    foo: &foo
      a: foo_a
      thing: foo_thing
      c: foo_c
    bar: &bar
      b: bar_b
      thing: bar_thing
      c: bar_c
    foobarList:
      b: foobarList_b
      !!merge <<:
        - *foo
        - *bar
      c: foobarList_c
    foobar:
      c: foobar_c
      !!merge <<: *foo
      thing: foobar_thing

then

    yq '.foobar.thing' sample.yml

will output

    foobar_thing

  Splatting merge anchors

Given a sample.yml file of:

    foo: &foo
      a: foo_a
      thing: foo_thing
      c: foo_c
    bar: &bar
      b: bar_b
      thing: bar_thing
      c: bar_c
    foobarList:
      b: foobarList_b
      !!merge <<:
        - *foo
        - *bar
      c: foobarList_c
    foobar:
      c: foobar_c
      !!merge <<: *foo
      thing: foobar_thing

then

    yq '.foobar[]' sample.yml

will output

    foo_c
    foo_a
    foobar_thing

  Traversing merge anchor lists

Note that the later merge anchors override previous

Given a sample.yml file of:

    foo: &foo
      a: foo_a
      thing: foo_thing
      c: foo_c
    bar: &bar
      b: bar_b
      thing: bar_thing
      c: bar_c
    foobarList:
      b: foobarList_b
      !!merge <<:
        - *foo
        - *bar
      c: foobarList_c
    foobar:
      c: foobar_c
      !!merge <<: *foo
      thing: foobar_thing

then

    yq '.foobarList.thing' sample.yml

will output

    bar_thing

  Splatting merge anchor lists

Given a sample.yml file of:

    foo: &foo
      a: foo_a
      thing: foo_thing
      c: foo_c
    bar: &bar
      b: bar_b
      thing: bar_thing
      c: bar_c
    foobarList:
      b: foobarList_b
      !!merge <<:
        - *foo
        - *bar
      c: foobarList_c
    foobar:
      c: foobar_c
      !!merge <<: *foo
      thing: foobar_thing

then

    yq '.foobarList[]' sample.yml

will output

    bar_b
    foo_a
    bar_thing
    foobarList_c

  Select multiple indices

Given a sample.yml file of:

    a:
      - a
      - b
      - c

then

    yq '.a[0, 2]' sample.yml

will output

    a
    c

Union

This operator is used to combine different results together.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Combine scalars

Running

    yq --null-input '1, true, "cat"'

will output

    1
    true
    cat

  Combine selected paths

Given a sample.yml file of:

    a: fieldA
    b: fieldB
    c: fieldC

then

    yq '.a, .c' sample.yml

will output

    fieldA
    fieldC

Unique

This is used to filter out duplicated items in an array. Note that the original order of the array is maintained.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Unique array of scalars (string/numbers)

Note that unique maintains the original order of the array.

Given a sample.yml file of:

    - 2
    - 1
    - 3
    - 2

then

    yq 'unique' sample.yml

will output

    - 2
    - 1
    - 3

  Unique nulls

Unique works on the node value, so it considers different representations of nulls to be different

Given a sample.yml file of:

    - ~
    - null
    - ~
    - null

then

    yq 'unique' sample.yml

will output

    - ~
    - null

  Unique all nulls

Run against the node tag to unique all the nulls

Given a sample.yml file of:

    - ~
    - null
    - ~
    - null

then

    yq 'unique_by(tag)' sample.yml

will output

    - ~

  Unique array object fields

Given a sample.yml file of:

    - name: harry
      pet: cat
    - name: billy
      pet: dog
    - name: harry
      pet: dog

then

    yq 'unique_by(.name)' sample.yml

will output

    - name: harry
      pet: cat
    - name: billy
      pet: dog

Variable Operators

Like the jq equivalents, variables are sometimes required for the more complex expressions (or swapping values between fields).

Note that there is also an additional ref operator that holds a reference (instead of a copy) of the path, allowing you to make multiple changes to the same path.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Single value variable

Given a sample.yml file of:

    a: cat

then

    yq '.a as $foo | $foo' sample.yml

will output

    cat

  Multi value variable

Given a sample.yml file of:

    - cat
    - dog

then

    yq '.[] as $foo | $foo' sample.yml

will output

    cat
    dog

  Using variables as a lookup

Example taken from jq (https://stedolan.github.io/jq/manual/#Variable/SymbolicBindingOperator:...as$identifier%7C...)

Given a sample.yml file of:

    "posts":
      - "title": First post
        "author": anon
      - "title": A well-written article
        "author": person1
    "realnames":
      "anon": Anonymous Coward
      "person1": Person McPherson

then

    yq '.realnames as $names | .posts[] | {"title":.title, "author": $names[.author]}' sample.yml

will output

    title: First post
    author: Anonymous Coward
    title: A well-written article
    author: Person McPherson

  Using variables to swap values

Given a sample.yml file of:

    a: a_value
    b: b_value

then

    yq '.a as $x  | .b as $y | .b = $x | .a = $y' sample.yml

will output

    a: b_value
    b: a_value

  Use ref to reference a path repeatedly

Note: You may find the with operator more useful.

Given a sample.yml file of:

    a:
      b: thing
      c: something

then

    yq '.a.b ref $x | $x = "new" | $x style="double"' sample.yml

will output

    a:
      b: "new"
      c: something

With

Use the with operator to conveniently make multiple updates to a deeply nested path, or to update array elements relatively to each other. The first argument expression sets the root context, and the second expression runs against that root context.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Update and style

Given a sample.yml file of:

    a:
      deeply:
        nested: value

then

    yq 'with(.a.deeply.nested; . = "newValue" | . style="single")' sample.yml

will output

    a:
      deeply:
        nested: 'newValue'

  Update multiple deeply nested properties

Given a sample.yml file of:

    a:
      deeply:
        nested: value
        other: thing

then

    yq 'with(.a.deeply; .nested = "newValue" | .other= "newThing")' sample.yml

will output

    a:
      deeply:
        nested: newValue
        other: newThing

  Update array elements relatively

The second expression runs with each element of the array as it’s contextual root. This allows you to make updates relative to the element.

Given a sample.yml file of:

    myArray:
      - a: apple
      - a: banana

then

    yq 'with(.myArray[]; .b = .a + " yum")' sample.yml

will output

    myArray:
      - a: apple
        b: apple yum
      - a: banana
        b: banana yum

JSON

Encode and decode to and from JSON. Supports multiple JSON documents in a single file (e.g. NDJSON).

Note that YAML is a superset of (single document) JSON - so you don’t have to use the JSON parser to read JSON when there is only one JSON document in the input. You will probably want to pretty print the result in this case, to get idiomatic YAML styling.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Parse json: simple

JSON is a subset of yaml, so all you need to do is prettify the output

Given a sample.json file of:

    {"cat": "meow"}

then

    yq -P '.' sample.json

will output

    cat: meow

  Parse json: complex

JSON is a subset of yaml, so all you need to do is prettify the output

Given a sample.json file of:

    {"a":"Easy! as one two three","b":{"c":2,"d":[3,4]}}

then

    yq -P '.' sample.json

will output

    a: Easy! as one two three
    b:
      c: 2
      d:
        - 3
        - 4

  Encode json: simple

Given a sample.yml file of:

    cat: meow

then

    yq -o=json '.' sample.yml

will output

    {
      "cat": "meow"
    }

  Encode json: simple - in one line

Given a sample.yml file of:

    cat: meow # this is a comment, and it will be dropped.

then

    yq -o=json -I=0 '.' sample.yml

will output

    {"cat":"meow"}

  Encode json: comments

Given a sample.yml file of:

    cat: meow # this is a comment, and it will be dropped.

then

    yq -o=json '.' sample.yml

will output

    {
      "cat": "meow"
    }

  Encode json: anchors

Anchors are dereferenced

Given a sample.yml file of:

    cat: &ref meow
    anotherCat: *ref

then

    yq -o=json '.' sample.yml

will output

    {
      "cat": "meow",
      "anotherCat": "meow"
    }

  Encode json: multiple results

Each matching node is converted into a json doc. This is best used with 0 indent (json document per line)

Given a sample.yml file of:

    things: [{stuff: cool}, {whatever: cat}]

then

    yq -o=json -I=0 '.things[]' sample.yml

will output

    {"stuff":"cool"}
    {"whatever":"cat"}

  Roundtrip NDJSON

Unfortunately the json encoder strips leading spaces of values.

Given a sample.json file of:

    {"this": "is a multidoc json file"}
    {"each": ["line is a valid json document"]}
    {"a number": 4}

then

    yq -p=json -o=json -I=0 sample.json

will output

    {"this":"is a multidoc json file"}
    {"each":["line is a valid json document"]}
    {"a number":4}

  Roundtrip multi-document JSON

The NDJSON parser can also handle multiple multi-line json documents in a single file!

Given a sample.json file of:

    {
        "this": "is a multidoc json file"
    }
    {
        "it": [
            "has",
            "consecutive",
            "json documents"
        ]
    }
    {
        "a number": 4
    }

then

    yq -p=json -o=json -I=2 sample.json

will output

    {
      "this": "is a multidoc json file"
    }
    {
      "it": [
        "has",
        "consecutive",
        "json documents"
      ]
    }
    {
      "a number": 4
    }

  Update a specific document in a multi-document json

Documents are indexed by the documentIndex or di operator.

Given a sample.json file of:

    {"this": "is a multidoc json file"}
    {"each": ["line is a valid json document"]}
    {"a number": 4}

then

    yq -p=json -o=json -I=0 '(select(di == 1) | .each ) += "cool"' sample.json

will output

    {"this":"is a multidoc json file"}
    {"each":["line is a valid json document","cool"]}
    {"a number":4}

  Find and update a specific document in a multi-document json

Use expressions as you normally would.

Given a sample.json file of:

    {"this": "is a multidoc json file"}
    {"each": ["line is a valid json document"]}
    {"a number": 4}

then

    yq -p=json -o=json -I=0 '(select(has("each")) | .each ) += "cool"' sample.json

will output

    {"this":"is a multidoc json file"}
    {"each":["line is a valid json document","cool"]}
    {"a number":4}

  Decode NDJSON

Given a sample.json file of:

    {"this": "is a multidoc json file"}
    {"each": ["line is a valid json document"]}
    {"a number": 4}

then

    yq -p=json sample.json

will output

    this: is a multidoc json file
    ---
    each:
        - line is a valid json document
    ---
    a number: 4

CSV

Encode/Decode/Roundtrip CSV and TSV files.

  Encode

Currently supports arrays of homogenous flat objects, that is: no nesting and it assumes the first object has all the keys required:

    - name: Bobo
      type: dog
    - name: Fifi
      type: cat

As well as arrays of arrays of scalars (strings/numbers/booleans):

    - [Bobo, dog]
    - [Fifi, cat]

  Decode

Decode assumes the first CSV/TSV row is the header row, and all rows beneath are the entries. The data will be coded into an array of objects, using the header rows as keys.

    name,type
    Bobo,dog
    Fifi,cat

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Encode CSV simple

Given a sample.yml file of:

    - [i, like, csv]
    - [because, excel, is, cool]

then

    yq -o=csv sample.yml

will output

    i,like,csv
    because,excel,is,cool

  Encode TSV simple

Given a sample.yml file of:

    - [i, like, csv]
    - [because, excel, is, cool]

then

    yq -o=tsv sample.yml

will output

    i   like    csv
    because excel   is  cool

  Encode array of objects to csv

Given a sample.yml file of:

    - name: Gary
      numberOfCats: 1
      likesApples: true
      height: 168.8
    - name: Samantha's Rabbit
      numberOfCats: 2
      likesApples: false
      height: -188.8

then

    yq -o=csv sample.yml

will output

    name,numberOfCats,likesApples,height
    Gary,1,true,168.8
    Samantha's Rabbit,2,false,-188.8

  Encode array of objects to custom csv format

Add the header row manually, then the we convert each object into an array of values - resulting in an array of arrays. Pick the columns and call the header whatever you like.

Given a sample.yml file of:

    - name: Gary
      numberOfCats: 1
      likesApples: true
      height: 168.8
    - name: Samantha's Rabbit
      numberOfCats: 2
      likesApples: false
      height: -188.8

then

    yq -o=csv '[["Name", "Number of Cats"]] +  [.[] | [.name, .numberOfCats ]]' sample.yml

will output

    Name,Number of Cats
    Gary,1
    Samantha's Rabbit,2

  Encode array of objects to csv - missing fields behaviour

First entry is used to determine the headers, and it is missing `likesApples', so it is not included in the csv. Second entry does not have `numberOfCats' so that is blank

Given a sample.yml file of:

    - name: Gary
      numberOfCats: 1
      height: 168.8
    - name: Samantha's Rabbit
      height: -188.8
      likesApples: false

then

    yq -o=csv sample.yml

will output

    name,numberOfCats,height
    Gary,1,168.8
    Samantha's Rabbit,,-188.8

  Parse CSV into an array of objects

First row is assumed to be the header row.

Given a sample.csv file of:

    name,numberOfCats,likesApples,height
    Gary,1,true,168.8
    Samantha's Rabbit,2,false,-188.8

then

    yq -p=csv sample.csv

will output

    - name: Gary
      numberOfCats: 1
      likesApples: true
      height: 168.8
    - name: Samantha's Rabbit
      numberOfCats: 2
      likesApples: false
      height: -188.8

  Parse TSV into an array of objects

First row is assumed to be the header row.

Given a sample.tsv file of:

    name    numberOfCats    likesApples height
    Gary    1   true    168.8
    Samantha's Rabbit   2   false   -188.8

then

    yq -p=tsv sample.tsv

will output

    - name: Gary
      numberOfCats: 1
      likesApples: true
      height: 168.8
    - name: Samantha's Rabbit
      numberOfCats: 2
      likesApples: false
      height: -188.8

  Round trip

Given a sample.csv file of:

    name,numberOfCats,likesApples,height
    Gary,1,true,168.8
    Samantha's Rabbit,2,false,-188.8

then

    yq -p=csv -o=csv '(.[] | select(.name == "Gary") | .numberOfCats) = 3' sample.csv

will output

    name,numberOfCats,likesApples,height
    Gary,3,true,168.8
    Samantha's Rabbit,2,false,-188.8

Properties

Encode/Decode/Roundtrip to/from a property file. Line comments on value nodes will be copied across.

By default, empty maps and arrays are not encoded - see below for an example on how to encode a value for these.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Encode properties

Note that empty arrays and maps are not encoded by default.

Given a sample.yml file of:

    # block comments don't come through
    person: # neither do comments on maps
        name: Mike Wazowski # comments on values appear
        pets:
        - cat # comments on array values appear
        food: [pizza] # comments on arrays do not
    emptyArray: []
    emptyMap: []

then

    yq -o=props sample.yml

will output

    # comments on values appear
    person.name = Mike Wazowski
    # comments on array values appear
    person.pets.0 = cat
    person.food.0 = pizza

  Encode properties: scalar encapsulation

Note that string values with blank characters in them are encapsulated with double quotes

Given a sample.yml file of:

    # block comments don't come through
    person: # neither do comments on maps
        name: Mike Wazowski # comments on values appear
        pets:
        - cat # comments on array values appear
        food: [pizza] # comments on arrays do not
    emptyArray: []
    emptyMap: []

then

    yq -o=props --unwrapScalar=false sample.yml

will output

    # comments on values appear
    person.name = "Mike Wazowski"
    # comments on array values appear
    person.pets.0 = cat
    person.food.0 = pizza

  Encode properties: no comments

Given a sample.yml file of:

    # block comments don't come through
    person: # neither do comments on maps
        name: Mike Wazowski # comments on values appear
        pets:
        - cat # comments on array values appear
        food: [pizza] # comments on arrays do not
    emptyArray: []
    emptyMap: []

then

    yq -o=props '... comments = ""' sample.yml

will output

    person.name = Mike Wazowski
    person.pets.0 = cat
    person.food.0 = pizza

  Encode properties: include empty maps and arrays

Use a yq expression to set the empty maps and sequences to your desired value.

Given a sample.yml file of:

    # block comments don't come through
    person: # neither do comments on maps
        name: Mike Wazowski # comments on values appear
        pets:
        - cat # comments on array values appear
        food: [pizza] # comments on arrays do not
    emptyArray: []
    emptyMap: []

then

    yq -o=props '(.. | select( (tag == "!!map" or tag =="!!seq") and length == 0)) = ""' sample.yml

will output

    # comments on values appear
    person.name = Mike Wazowski
    # comments on array values appear
    person.pets.0 = cat
    person.food.0 = pizza
    emptyArray =
    emptyMap =

  Decode properties

Given a sample.properties file of:

    # comments on values appear
    person.name = Mike Wazowski
    # comments on array values appear
    person.pets.0 = cat
    person.food.0 = pizza

then

    yq -p=props sample.properties

will output

    person:
        name: Mike Wazowski # comments on values appear
        pets:
            - cat # comments on array values appear
        food:
            - pizza

  Roundtrip

Given a sample.properties file of:

    # comments on values appear
    person.name = Mike Wazowski
    # comments on array values appear
    person.pets.0 = cat
    person.food.0 = pizza

then

    yq -p=props -o=props '.person.pets.0 = "dog"' sample.properties

will output

    # comments on values appear
    person.name = Mike Wazowski
    # comments on array values appear
    person.pets.0 = dog
    person.food.0 = pizza

XML

Encode and decode to and from XML. Whitespace is not conserved for round trips - but the order of the fields are.

Consecutive xml nodes with the same name are assumed to be arrays.

XML content data and attributes are created as fields. This can be controlled by the '--xml-attribute-prefix and --xml-content-name flags - see below for examples.

{% hint style=“warning” %} Note that versions prior to 4.18 require the `eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

  Parse xml: simple

Notice how all the values are strings, see the next example on how you can fix that.

Given a sample.xml file of:

    <?xml version="1.0" encoding="UTF-8"?>
    <cat>
      <says>meow</says>
      <legs>4</legs>
      <cute>true</cute>
    </cat>

then

    yq -p=xml '.' sample.xml

will output

    cat:
      says: meow
      legs: "4"
      cute: "true"

  Parse xml: number

All values are assumed to be strings when parsing XML, but you can use the from_yaml operator on all the strings values to autoparse into the correct type.

Given a sample.xml file of:

    <?xml version="1.0" encoding="UTF-8"?>
    <cat>
      <says>meow</says>
      <legs>4</legs>
      <cute>true</cute>
    </cat>

then

    yq -p=xml ' (.. | select(tag == "!!str")) |= from_yaml' sample.xml

will output

    cat:
      says: meow
      legs: 4
      cute: true

  Parse xml: array

Consecutive nodes with identical xml names are assumed to be arrays.

Given a sample.xml file of:

    <?xml version="1.0" encoding="UTF-8"?>
    <animal>cat</animal>
    <animal>goat</animal>

then

    yq -p=xml '.' sample.xml

will output

    animal:
      - cat
      - goat

  Parse xml: attributes

Attributes are converted to fields, with the default attribute prefix `+'. Use ’–xml-attribute-prefix` to set your own.

Given a sample.xml file of:

    <?xml version="1.0" encoding="UTF-8"?>
    <cat legs="4">
      <legs>7</legs>
    </cat>

then

    yq -p=xml '.' sample.xml

will output

    cat:
      +legs: "4"
      legs: "7"

  Parse xml: attributes with content

Content is added as a field, using the default content name of +content. Use --xml-content-name to set your own.

Given a sample.xml file of:

    <?xml version="1.0" encoding="UTF-8"?>
    <cat legs="4">meow</cat>

then

    yq -p=xml '.' sample.xml

will output

    cat:
      +content: meow
      +legs: "4"

  Parse xml: custom dtd

DTD entities are ignored.

Given a sample.xml file of:

    <?xml version="1.0"?>
    <!DOCTYPE root [
    <!ENTITY writer "Blah.">
    <!ENTITY copyright "Blah">
    ]>
    <root>
        <item>&writer;&copyright;</item>
    </root>

then

    yq -p=xml '.' sample.xml

will output

    root:
      item: '&writer;&copyright;'

  Parse xml: with comments

A best attempt is made to preserve comments.

Given a sample.xml file of:

    <!-- before cat -->
    <cat>
        <!-- in cat before -->
        <x>3<!-- multi
    line comment
    for x --></x>
        <!-- before y -->
        <y>
            <!-- in y before -->
            <d><!-- in d before -->z<!-- in d after --></d>

            <!-- in y after -->
        </y>
        <!-- in_cat_after -->
    </cat>
    <!-- after cat -->

then

    yq -p=xml '.' sample.xml

will output

    # before cat
    cat:
      # in cat before
      x: "3" # multi
      # line comment
      # for x
      # before y
      y:
        # in y before
        # in d before
        d: z # in d after
        # in y after
      # in_cat_after
    # after cat

  Parse xml: keep attribute namespace

Given a sample.xml file of:

    <?xml version="1.0"?>
    <map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url">
    </map>

then

    yq -p=xml -o=xml --xml-keep-namespace '.' sample.xml

will output

    <map xmlns="some-namespace" xmlns:xsi="some-instance" some-instance:schemaLocation="some-url"></map>

instead of

    <map xmlns="some-namespace" xsi="some-instance" schemaLocation="some-url"></map>

  Parse xml: keep raw attribute namespace

Given a sample.xml file of:

    <?xml version="1.0"?>
    <map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url">
    </map>

then

    yq -p=xml -o=xml --xml-keep-namespace --xml-raw-token '.' sample.xml

will output

    <map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url"></map>

instead of

    <map xmlns="some-namespace" xsi="some-instance" schemaLocation="some-url"></map>

  Encode xml: simple

Given a sample.yml file of:

    cat: purrs

then

    yq -o=xml '.' sample.yml

will output

    <cat>purrs</cat>

  Encode xml: array

Given a sample.yml file of:

    pets:
      cat:
        - purrs
        - meows

then

    yq -o=xml '.' sample.yml

will output

    <pets>
      <cat>purrs</cat>
      <cat>meows</cat>
    </pets>

  Encode xml: attributes

Fields with the matching xml-attribute-prefix are assumed to be attributes.

Given a sample.yml file of:

    cat:
      +name: tiger
      meows: true

then

    yq -o=xml '.' sample.yml

will output

    <cat name="tiger">
      <meows>true</meows>
    </cat>

  Encode xml: attributes with content

Fields with the matching xml-content-name is assumed to be content.

Given a sample.yml file of:

    cat:
      +name: tiger
      +content: cool

then

    yq -o=xml '.' sample.yml

will output

    <cat name="tiger">cool</cat>

  Encode xml: comments

A best attempt is made to copy comments to xml.

Given a sample.yml file of:

    # above_cat
    cat: # inline_cat
      # above_array
      array: # inline_array
        - val1 # inline_val1
        # above_val2
        - val2 # inline_val2
    # below_cat

then

    yq -o=xml '.' sample.yml

will output

    <!-- above_cat inline_cat --><cat><!-- above_array inline_array -->
      <array>val1<!-- inline_val1 --></array>
      <array><!-- above_val2 -->val2<!-- inline_val2 --></array>
    </cat><!-- below_cat -->

  Round trip: with comments

A best effort is made, but comment positions and white space are not preserved perfectly.

Given a sample.xml file of:

    <!-- before cat -->
    <cat>
        <!-- in cat before -->
        <x>3<!-- multi
    line comment
    for x --></x>
        <!-- before y -->
        <y>
            <!-- in y before -->
            <d><!-- in d before -->z<!-- in d after --></d>

            <!-- in y after -->
        </y>
        <!-- in_cat_after -->
    </cat>
    <!-- after cat -->

then

    yq -p=xml -o=xml '.' sample.xml

will output

    <!-- before cat --><cat><!-- in cat before -->
      <x>3<!-- multi
    line comment
    for x --></x><!-- before y -->
      <y><!-- in y before
    in d before -->
        <d>z<!-- in d after --></d><!-- in y after -->
      </y><!-- in_cat_after -->
    </cat><!-- after cat -->

AUTHORS
Mike Farah.