feature request: support templated prepared queries in `"consul.hashicorp.com/connect-service-upstreams"` annotation

[dschaaff]

I'll try to describe my use case

I have number of environments for dev/testing that contain the same services in each one. Each environment runs in its kubernetes namespace in the same cluster. I want the upstream to only return copies of the connect service located in the same namespace. I can created a prepared query using templating that returns the desired result

{
	"name": "",
	"Template": {
    "Type": "name_prefix_match",
		"Regexp": "^(example-service)-(.*?)$"
	},
	"Service": {
    "Service": "${match(1)}",
    "Tags": ["${match(2)}"],
		"Connect": true
  }
}

Where match(0) is the name of the service and match(1) is the desired namespace. This works since I tag all the services with the namespace.

Please let me know if that makes sense, I'm happy to expand if desired.

[ishustava]

Hey @dschaaff,

You could create a prepared query in Consul and then reference it by name in the upstream (docs):

annotations:
  "consul.hashicorp.com/connect-service-upstreams": "prepared_query:[query name]:[port]"

Would that work for your use-case?

[dschaaff]

I do that currently. We have environments, each in their own k8s namespace, that get created for each branch in a project. Each service has the same name in each environment. This means "example-service" returns services across all of those namespaces. I'm looking to use prepared queries to allow the upstreams for "example-service" to only return those in the same namepace. To help me achieve this I register each connect service with a tag containing the namespace.

I tried to do this using a templated prepared query at first.

{
	"name": "",
	"Template": {
    "Type": "name_prefix_match",
		"Regexp": "^(example-service)-(.*?)$"
	},
	"Service": {
    "Service": "${match(1)}",
    "Tags": ["${match(2)}"],
		"Connect": true
  }
}

This query allows me lookup "example-service-my-namespace" and only receive the example service in "my-namespace". This works with the dns interface and I receive the expected results with dig example-service-my-namespace.query.consul. Unfortunately, this doesn't work with the annotations for upstreams. "consul.hashicorp.com/connect-service-upstreams": "prepared_query:example-service-my-namespace:1234" does not return any upstreams. I'd prefer a situation like this because I only need one templated query that would work for get any service in a any namespace.

My current workaround is running a small script before deploying the branch environments to create an explicit prepared query for each services.

create_consul_prepared_queries() {
  for s in "${services[@]}"
  do
    echo "creating prepared query for $s in consul to allow filtering by namespace"
    echo "rpc error making call: name $s-$NAMESPACE aliases an existing query name is expected when query already exists"
    curl --request POST \
      --url https://consul-ui.dev.cordialdev.com/v1/query \
      --header 'content-type: application/json' \
      --header "X-Consul-Token: ${CONSUL_HTTP_TOKEN}" \
      --data '{
        "name": "'"$s-$NAMESPACE"'",
        "Service": {
          "Service": "'"$s"'",
          "Tags": ["'"$NAMESPACE"'"],
          "Connect": true
        }
      }'
  done
}

I can then use the created prepared queries by name. When I tear down one of these environments I have another script that removes the prepared queries.

This workaround gets the job done but it'd be really nice to have a single templated query to maintain.