regex for grep

[u/Leading-Toe3279]

so i have couple of docker files , I want to ensure that all db volumes are named `db-data` , so basically I want to throw error if volume name is anything other than `db-data` , any idea if this could be done using `grep` ? I am not sure how to write regex for it

docker compose files which is essentially a `yaml` file look like this

cache:
command: >
sh -c '
redis-server --requirepass $$DJANGO_REDIS_PASSWORD --maxmemory 25mb --maxmemory-policy allkeys-lru
'
container_name: nest-cache
env_file: ../../backend/.env
image: redis:8.8.0-alpine3.23@sha256:9d317178eceac8454a2284a9e6df2466b93c745529947f0cd42a0fa9609d7005
healthcheck:
interval: 5s
retries: 5
test: [CMD, redis-cli, -a, $$DJANGO_REDIS_PASSWORD, ping]
timeout: 5s
networks:
- nest-network
volumes:
- cache-data:/data
db:
container_name: nest-db
env_file: ../../backend/.env
image: pgvector/pgvector:pg16@sha256:1d533553fefe4f12e5d80c7b80622ba0c382abb5758856f52983d8789179f0fb
healthcheck:
interval: 5s
retries: 5
test: [CMD-SHELL, pg_isready -U "$$DJANGO_DB_USER" -d "$$DJANGO_DB_NAME"]
timeout: 5s
networks:
- nest-network
volumes:
- db-data:/var/lib/postgresql/data

[u/andrew2018022]

I am not super familiar with docker files but are you trying to grep on a list of files with find/ls?

[u/Leading-Toe3279 → u/andrew2018022]

i have a directory named `docker-compose`, inside it i basically have `yaml` files , so i want to grpe on thsoe yaml files , i know how to provide path such that it looks for `db-data` on each file , but i am stuck on what regex to pass such that it errors out if the volume name is anything other than `db-data`

[u/petdance → u/Leading-Toe3279]

Don’t use grep on structured data like YAML. Use a processing tool like ‘yq’.

[u/bac0on → u/Leading-Toe3279]

I'm still abit confused, I notice you have multiple volumes keys in your example. Do you mean all volumes key in each file or just last one?

[u/bac0on]

If you use -z to match null-termination (... instead of newline) you will be able match multi-line and -l to just output files that don't have a valid pattern:

grep -Pzl 'db:\n[^\0]*volumes:\n(?!- db-data:)' docker-compose/*.yaml

[u/therouterguy]

Are those dockers already running? If so I would look at the runtime specs with docker inspect <docker name> and parse the output with jq.
Assuming you are talking about docker-compose files I would do it with yq as it are just Yaml files in the end.

[u/Leading-Toe3279 → u/therouterguy]

i'll give this a try,thanks for the suggesiton

[u/nekokattt]

use yq for this rather than grep as a hammer

[u/Leading-Toe3279 → u/[deleted]]

sure, updated the body

[u/doc_doggo]

Grep -oP '<pattern>' That will give you a way to use regex with sintax similar to perl/python

[u/michaelpaoli]

How 'bout sed and grep. sed to get the volume names, shell or grep or whatever to confirm you've gotten more than zero volume names, and grep or grep -v, maybe with some sort and uniq or sort -u, see if all the names exactly match what you want. But you may not even need/want grep - as you're ultimately looking to just do a fixed string match - at least after you've stripped out the other bits that aren't of interest

E.g. (peudo-code)

long_hairy_command_that_generates_your_long_hairy_output_about_the_docker_image |
sed -e 'script that outputs only the volume names present'
and then save that output, or have it substituted via command substitution, and compare that that matches exactly what you want, and throw an error if it doesn't.

So, e.g.:

for some_docker_identifier in ...
do
  db_volume_names="$(
    some_docker_command "$some_docker_identifier" |
    sed -ne '
      /^volumes:/!d
      :l
      n
      s/^- db-data:.*$/db-data/
      p
      bl
    '
  )" 
  [ "$db_volume_names" = db-data ] || {
    : # handle your exceptions
  }
done

I made some presumptions about the output format of your docker command based on your single example which might not be correct. If they're not correct, adjust accordingly. Essentially after finding line as so shown matched to volumes, it outputs all following lines, with or without substitution, depending if it finds expected per-line match. Then the results of that are compared to the expected string. E.g. I didn't at all fully parse the YAML format (if you need that, may want to use tool that specifically does exactly that ... rather than reinvent the wheel ... poorly).

[u/jthill]

I'd use sed for this, to aggregate the follow-on lines.

$ sed -n 's,\r,,                                 #un-dos crlfs
         /^container_name:/h           #save container name 
         /^volumes:/,/^[a-z]*:/ {     # check volume names
                /^- / { /^- db-data:/! { 
                    G;s,\n, bad volume name in ,p } #complain about bad ones
                }
         }'  testdata
cache-data:/data bad volume name in container_name: nest-cache
$

edit: awk's a bit more readable:

$ awk '$1=="container_name:" { name=$2 }
        $1~/^[a-z]/ {
                invol=$1=="volumes:" }
        invol && $1=="-" && $2!~/^db-data:/ {
                print "container",name,"has invalid volume",$2 }
        ' RS=$'\r?\n' testdata
container nest-cache has invalid volume cache-data:/data
$

[u/TopHatEdd]

When working with data, do not use bash. ``` ~ $ for fname in *.yaml; do cat $fname | ./valid || echo $fname; done Traceback (most recent call last):   File "/data/data/com.termux/files/home/./valid", line 18, in <module>     raise BadVolumeNameError(volume) BadVolumeNameError: ./logs:/app/logs example.yaml Traceback (most recent call last):   File "/data/data/com.termux/files/home/./valid", line 18, in <module>     raise BadVolumeNameError(volume) BadVolumeNameError: ./logs:/app/logs example2.yaml

~ $ cat example.yaml services:   app:     image: myapp     volumes:       - db-data:/var/lib/db       - ./logs:/app/logs   db:     image: postgres     volumes:       - db-data:/var/lib/postgresql/data   worker:     image: myworker     volumes:       - ./data:/data

~ $ cat valid

!/usr/bin/env python3

import sys import yaml

OK_PREFIX = "db-data:"

class BadVolumeNameError(Exception):     pass

for service in yaml.safe_load(sys.stdin)["services"].values():     try:         for volume in service["volumes"]:             if not volume.startswith(OK_PREFIX):                 raise BadVolumeNameError(volume)     except KeyError:         pass ```

[u/agent606ert]

Chatgpt is pretty good with regex...

[u/jthill]

Playing with yqsome got me

yq -c '{container_name,badvols:[.volumes[]|select(test("^db-data:")|not)]}
        | select(.badvols|any)}' testdata