Merge lists with same key into a single list

[karamellpelle]

Consider the following YAML document:

train:
  locomotive:
    fruits:
      - banana
      - pear
      - apple
  wagon-1:
    fruits:
      - strawberry
      - apricot
  wagon-2:
    fruits:
      - papaya
      - plum
      - lime

fruits:
  - orange
  - lemon

airplane:
  fruits:
      - mango
      - pineapple
      - lemon
      - banana
      - grapefruit
      - kiwi
      - clementine

The key fruits appears at different places and levels, each having a value of type list. Is there an yq expression to merge all these lists into a single list, i.e a list of all the fruits in this document? And as a bonus question: Can we also strip the result into a list of unique values?

The closest to a solution I've gotten is this question, but still not the same.

I'm going to use yq in a patch for a vim plugin.

[mikefarah]

Sure:

yq '[.. | select(has("fruits")) | .fruits | .[]] | unique' file.yaml

Explanation:

• .. | select(has("fruits")) finds all the nodes that has a child 'fruits'
• .fruits gets the array
• .[] splats the array into its component fruit strings
• The whole thing is wrapped in an array, to make a single array of all the fruit
• This is piped into unique to remove all the dupes.
• Bonus, you can then pipe into sort as well

[karamellpelle]

Thank you so much! I have an idea to contribute to a neovim plugin for Haskell development (telescope_hoogle). Many Haskell projects are based on haskellstack, and they have package.yaml files that defines dependencies (and more, of course). For such projects, I want the plugin to only lookup in dependent packages.

BTW, the command above turned out to includes comments in the list, but I found that they can be removed like so:

yq '[... comments="" | .. | select(has("dependencies")) | .dependencies | .[]] | unique | sort' package.yaml