Armory Agent Configuration Options

Learn how to configure the Armory Agent based on installation mode and environment restrictions. This guide contains a detailed list of configuration options.

Proprietary

Where to configure the Agent

Set these options at the agent level in the kubesvc.yaml configuration file. If deploying as a non-SpinnakerTM service, you need to specify a clouddriver.grpc endpoint (e.g. grpc.spinnaker.example.com:443).

Kubernetes account

At a minimum you will need to add an account, give it a name, and set its Spinnaker permissions.

Spinnaker Service and Infrastructure modes

In these modes, you set up multiple accounts per agent. Your configuration should look like:

kubernetes:
  accounts:
    - name: account-01
      kubeconfigFile: /kubeconfigfiles/kubecfg-account01.yaml
    - ...  

If you are migrating accounts from Clouddriver, you can just copy the same block of configuration here. Unused properties are ignored.

Agent mode

In agent mode, your configuration should look like:

kubernetes:
  accounts:
    - name: account-01
      serviceAccount: true

Options

Settings Type Default Description
clouddriver.grpc string (hostname) spin-clouddriver-grpc:9091 Hostname of the Clouddriver or gRPC proxy endpoint
clouddriver.insecure boolean false If true, we’re connecting to a non TLS server
clouddriver.tls.serverName string none Server name on the remote certificate (override from the hostname)
clouddriver.tls.insecureSkipVerify boolean false Do not verify the endpoint’s certificate
clouddriver.tls.clientCertFile
clouddriver.tls.clientKeyFile
clouddriver.tls.clientKeyFilePassword
string
string
string
none
none
none
Client certificate file for mTLS
Client key file if not included in the certificate
Password the key file if needed
clouddriver.tls.cacertFile string none If provided, verify endpoint certificate with the trust store. Otherwise, the system trust store is used.
clouddriver.auth.token string none 0.3.0+ Optional bearer token added to each request back to the endpoint.
clouddriver.auth.tokenCommand.command
clouddriver.auth.tokenCommand.args
clouddriver.auth.tokenCommand.format
clouddriver.auth.tokenCommand.refreshIntervalSeconds
string
[]string
string
integer
none
none
[]
0
0.3.0+ Allows to invoke a command every refreshIntervalSeconds seconds that outputs either the token (format is raw) or a JSON object with an attribute of token if format is json or left empty. args is the optional list of parameters to the command.
clouddriver.noProxy boolean false 0.3.1+ Ignore the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables when connecting back to the control plane (Spinnaker)
logging.file string stdout if not defined File to save logs to
logging.level string INFO Log level. Can be any of (case insensitive):
panic, fatal , error, warn (or warning), info, debug, trace
kubernetes.noProxy boolean false 0.3.1+ Ignore the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables when connecting to any Kubernetes cluster
kubernetes.reconnectTimeoutMs integer 5000 How long to wait before reconnecting to Spinnaker
kubernetes.accounts[].name string none, required Name of the Kubernetes cluster in Spinnaker. Spinnaker still needs to accept that name.
kubernetes.accounts[].kubeConfigFile string none Path to the kubeconfig file if not using serviceAccount
kubernetes.accounts[].insecure boolean false Do not verify the TLS certificate of the Kubernetes API server
Don’t use without a good reason.
kubernetes.accounts[].context string empty If provided, will use the given context of the configured kubeconfig
kubernetes.accounts[].oAuthScopes []string empty List of OAuth scope when authenticating with gcp provider
https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl#authentication
kubernetes.accounts[].serviceAccount boolean false If true and the Agent runs in Kubernetes - use the current service account to call to the current API server. In that mode, you don’t need to provide a kubeconfig file.
kubernetes.accounts[].namespaces []string empty 0.4.0+ Whitelist of namespaces to monitor.
This comes at a greater cost of multiplying the resources by the number of namespaces.
kubernetes.accounts[].omitNamespaces []string empty Blacklist of namespaces
This comes at a greater cost of multiplying the resources by the number of namespaces.
NOT CURRENTLY IMPLEMENTED
kubernetes.accounts[].onlyNamespacedResources boolean false 0.4.0+ If true, the Agent will ignore non-namespaced resources; namespaces must be whitelisted with namespaces setting and CRDs with customResourceDefinitons.
kubernetes.accounts[].kinds []string empty If not empty, only kinds in the list will be cached. Use the format <kind>.<apiGroup> (e.g. Deployment.apps)
kubernetes.accounts[].omitKinds []string empty List of kinds not to cache.
kubernetes.accounts[].customResourceDefinitions []{kind: } empty 0.4.0+ List of CustomResourceDefinition to expose to Spinnaker. This is not needed if onlyNamespacedResources is left off. The format of kind is Kind.group.
kubernetes.accounts[].metrics boolean false When true, sends pod metrics back to Spinnaker every 20s
kubernetes.accounts[].permissions map empty Same meaning as permissions in Clouddriver: under READ and WRITE list of roles authorized.
kubernetes.accounts[].maxResumableResourceAgeMs integer 300000 (5m) When connecting to Spinnaker, the Agent asks Clouddriver for the latest resource version known per resource that is not older than that setting.

The resource version is used to resume the watch without first doing a list - saving memory and time. There’s no guarantee that the resource version is still known. If not “remembered” by the Kubernetes API server, a list call will be used.

https://kubernetes.io/docs/reference/using-api/api-concepts/#efficient-detection-of-changes
kubernetes.accounts[].onlySpinnakerManaged boolean false Only return Spinnaker managed resources
NOT IMPLEMENTED in the Agent but added to the plugin see kubesvc.runtime.defaults.onlySpinnakerManaged
kubernetes.accounts[].noProxy boolean false 0.3.1+ Ignore the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables when connecting to that Kubernetes cluster
server.host string localhost hostname of the server health check
server.port integer 8082 port of the server health check
server.ssl.enabled, server.ssl.certFile, server.ssl.keyFile, server.ssl.keyPassword, server.ssl.caCertFile, server.ssl.keyFilePassword, server.ssl.clientAuth Various options to control TLS config. Don’t bother, it’s just for the health endpoint.
prometheus.enabled boolean false Enable Prometheus handler
prometheus.port integer 8008 Port to expose Prometheus metrics on. Responds to both /metrics (standard) and /prometheus_metrics (Spinnaker default)
tasks.totalBudget integer 0 If > 0, limits the number of tasks that can be started. Tasks have different cost. Watches are considered free because they are part of the normal operations of the Agent.
tasks.budgetPerAccount integer 0 Same as above but per account. If both settings are provided, they’re both checked.
tasks.queueCheckFrequencyMs integer 2000 Frequency at which the Agent will check for new tasks to launch. Once launched a task is not stopped until explicitly requested (account unregistered or connection to Spinnaker lost)
pprof.enabled boolean false Enable pprof endpoint. Useful for troubleshooting, slowness, memory leaks, and more!
https://github.com/google/pprof/blob/master/doc/README.md
pprof.port integer 6060 Port on which to respond to pprof requests
secrets.vault.* object none Vault configuration

Restricted Environments

Network Access

The Agent needs access to its control plane (Spinnaker) as well to the various Kubernetes clusters it is configured to monitor. You can control which traffic should go through an HTTP proxy via the usual HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables.

A common case is to force the connection back to the control plane via a proxy but bypass it for Kubernetes clusters. In that case, define the environment variable HTTPS_PROXY=https://my.corporate.proxy and use the kubernetes.noProxy: true setting to not have to maintain the list of Kubernetes hosts in NO_PROXY.

Kubernetes Authorization

The Agent should be configured to access each Kubernetes cluster it monitors with a service account. You can limit what Spinnaker can do via the role you assign to that service account. For example, if you’d like Spinnaker to see NetworkPolicies but not deploy them:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: agent-role
rules:
- apiGroups: ["networking.k8s.io"]
  resources: ["networkpolicies"]
  verbs: [ "get", "list", "watch"]
...

Namespace Restrictions

You can limit the Agent to monitoring specific namespaces by listing them under namespaces. If you need to prevent the Agent from accessing cluster-wide (non-namespaced) resources, use the onlyNamespacedResources setting.

A side effect of disabling cluster-wide resources is that CustomResourceDefinitions won’t be known (and therefore deployable by Spinnaker). CustomResourceDefinitions are cluster-wide resources, but the custom resources themselves may be namespaced. To workaround the limitation, you can define customResourceDefinitions. Both namespaces and CRDs will be sent to Spinnaker as “synthetic” resources. They won’t be queried or watched, but their existence will be known by Spinnaker.

kubernetes:
    accounts:
        - name: production
          ...
          # Restricts the agent to namespaces `ns1` and `ns2`
          namespaces:
            - ns1
            - ns2
          # Prevents the Agent from querying non-namespaced resources
          onlyNamespacedResources: true
          # Whitelist CRDs so Spinnaker
          customResourceDefinitions:
            - kind: ServiceMonitor.monitoring.coreos.com
            - kind: SpinnakerService.spinnaker.armory.io


Last modified March 26, 2021: (43eca74)