Configuration

This page covers configuring the driver itself, the kubelet prerequisites required to run it, and the command-line flags currently being deprecated in favour of the configuration file.

Driver configuration

The driver supports two configuration mechanisms:

• The config file - the main, preferred way to configure the driver.
• Command-line flags — kept for a small set of important, mostly behavior altering settings and for backward compatibility. Some flags are being deprecated in favor of their config file equivalent (see Helm: deprecated args values vs driverConfig below).

If the same setting is provided both ways, the explicit command-line flag wins, so avoid mixing the two for the same field.

Configuration file

The config file is a YAML file passed to the driver via --config <path>.

When deploying with Helm, you don’t write this file yourself: set the driverConfig value in your values file and the chart serializes it to YAML, stores it in a ConfigMap, mounts it as /etc/dracpu/config.yaml inside the driver container and passes --config automatically. For example, with a values.yaml containing:

# values.yaml
# Driver configuration
driverConfig:
  cpuDeviceMode: grouped
  reservedCPUs: "0-1"
# Other tuning knobs of Helm chart
image:
  tag: v0.2.0
resources:
  requests:
    cpu: "200m"
    memory: "100Mi"
nodeSelector:
  kubernetes.io/os: linux

helm install dra-driver-cpu oci://registry.k8s.io/dra-driver-cpu/charts/dra-driver-cpu -f values.yaml

Individual fields can also be tuned with --set instead of a values file, e.g. to reserve CPUs 0-1:

helm install dra-driver-cpu oci://registry.k8s.io/dra-driver-cpu/charts/dra-driver-cpu \
    --set driverConfig.reservedCPUs="0-1"

driverConfig sub-fields

The config file is a flat YAML map - there are no nested groups. All fields are optional except where noted. Unknown fields are rejected at startup to catch typos early.

These fields only affect the driver’s own behavior (CPU allocation, hostname, sysfs path, etc.). Anything about the driver’s Pod itself - image, resources, node placement, and so on - is configured through other Helm values, not through this file.

apiVersion (string)

• When present, must be v1alpha1. Rejected otherwise.

cpuDeviceMode (string, default: grouped)

• individual: deprecated. It exposes each allocatable CPU as a separate device in the ResourceSlice. For scheduler-controlled selection of exact CPUs, follow Migrating from individual mode.
• grouped: exposes a single device representing a group of CPUs. This mode treats CPUs as a consumable capacity within the group, improving scalability by reducing the number of API objects.

groupBy (string, default: numanode)

• Grouping strategy used when cpuDeviceMode is grouped.
• numanode: groups CPUs by NUMA node.
• socket: groups CPUs by socket.
• machine: groups all allocatable node CPUs into a single machine-wide capacity device. Use it only when that allocator must select from the entire node.

allocator (string, default: cpumanager)

• cpumanager: selects CPUs with the driver’s built-in topology-aware allocator.
• external: works with every grouped groupBy value, requires an opaque cpuset from an external scheduler, and publishes the attributes that scheduler needs to choose CPUs. See Custom Opaque CPUSet Allocation Overrides.

reservedCPUs (string)

• CPUs excluded from allocation and from the ResourceSlice, given as a cpuset, e.g. "0-1". This has the same semantics as the kubelet’s static CPU Manager policy with strict-cpu-reservation enabled and reservedSystemCPUs set. For correct CPU accounting, the number of CPUs reserved here should match the sum of the kubelet’s kubeReserved and systemReserved settings, so that the kubelet subtracts the right number of CPUs from Node.Status.Allocatable.

hostnameOverride (string)

• Override the node hostname the driver registers under.

kubeconfig (string)

• Path to a kubeconfig file (for out-of-cluster use).

publishNodeAllocatableResourceMapping (bool, default: false)

• Publish nodeAllocatableResources mappings in ResourceSlice devices, so the scheduler and kubelet account claimed CPUs as node allocatable cpu.
• Requires the DRANodeAllocatableResources feature gate (alpha, 1.37+). See Workload Configuration Requirements.

Example

# values.yaml
driverConfig:
  apiVersion: v1alpha1
  cpuDeviceMode: grouped
  groupBy: numanode
  reservedCPUs: "0-3"

Versioning and backward compatibility

The schema is versioned via the optional apiVersion field (currently v1alpha1). The layout is intentionally flat for now. If a nested hierarchy is introduced in the future, the apiVersion field will be bumped so that older config files continue to be accepted or produce an error.

Helm: deprecated args values vs driverConfig

driverConfig is the Helm value that generates the config file described above (a single map covering all driver settings — there is no separate Helm value per field). args.* on the other hand exposes individual fields as explicit Helm values.

• args.cpuDeviceMode, args.groupBy, args.reservedCPUs, and args.hostnameOverride are deprecated: instead of being emitted as CLI flags, they are now folded into the generated driverConfig ConfigMap.
• Both reach the same driver settings; args.* takes priority when both are set for the same field.
• The intent is to eventually deprecate args.* entirely in favour of driverConfig as the single configuration mechanism. The driver logs the effective configuration at startup so you can verify which values are active.

Helm values validation and compatibility

The chart rejects unknown top-level values, so a misspelled value name fails validation before anything is rendered rather than being ignored. Helm’s reserved global table stays accepted, and values whose keys you pick, such as podLabels and resources.limits, are untouched.

If a chart value is removed in a later release, an upgrade that carries the previous release’s values forward can fail validation. Helm carries them when no new values are supplied at all, with --reuse-values, and with --reset-then-reuse-values. Validation runs before anything is rendered or applied, so the running release is untouched: drop the stale key and upgrade again, or pass --reset-values with the values you want in full.

Values that only your own tooling reads belong outside the file you hand to Helm. They were never chart values, so they are rejected rather than ignored.

Command-line flags

NOTE: Command-line flags are kept mainly for backward compatibility. Prefer the configuration file above for new deployments.

--cpu-device-mode, --group-by, --reserved-cpus, --hostname-override, --sysfs-overlay are deprecated in favour of their config file equivalents and will be removed in a future release (issue #245). Each is marked as deprecated in --help, and passing one explicitly logs a startup warning.

• --cpu-device-mode → cpuDeviceMode
• --group-by → groupBy
• --reserved-cpus → reservedCPUs
• --hostname-override → hostnameOverride
• --sysfs-overlay → sysfsOverlay

The remaining flags aren’t part of that deprecation:

• --config: path to the config file described above.
• --kubeconfig: path to a kubeconfig file, for out-of-cluster use. Also settable via the kubeconfig config field.
• --bind-address: address the metrics server listens on.
• --expose-pcie-roots: adds the resource.kubernetes.io/pcieRoot standard value to CPU devices, reporting the PCIe roots close to each device. Since it always reports values as a list, this option requires the cluster feature gate DRAListTypeAttributes (see KEP 5491) to be enabled. The driver cannot introspect the cluster feature gate, so enable the feature gate first and this option second. Unlike the flags above, it is not deprecated and has no config file equivalent — it is intentionally excluded from the config file (see driverConfig sub-fields above) and stays a standalone flag (or Helm args.exposePCIeRoots).
• --kubelet-root-dir: the kubelet’s own root directory (default /var/lib/kubelet). Running the binary directly, or writing the manifests by hand, the flag is the way to set the root, together with matching <root>/plugins and <root>/plugins_registry mounts. Like --expose-pcie-roots, it is not deprecated and has no config file equivalent: the chart owns this value, not driverConfig.

Kubelet configuration prerequisites

IMPORTANT: The kubelet’s CPUManager implements assignment of exclusive CPUs to workloads. The CPUManager and this DRA driver are mutually incompatible and only one can be enabled at a time on any given node. You need to disable the CPUManager on the nodes you wish to run this DRA driver.

1. The default settings of the kubelet are compatible with this DRA driver. If you never fine-tuned the kubelet, you are probably fine.
2. Make sure cpuManagerPolicy: "none" is set in the kubelet configuration file.
3. If you changed the kubelet configuration, restart the kubelet to take effect. NOTE: you may need to delete the CPUManager state file.
4. You may now proceed with deploying and configuring this DRA driver.

Using a custom kubelet root directory

If the kubelet runs with --root-dir set to something other than /var/lib/kubelet, set kubeletRootDir to that root’s absolute path (relative paths are refused, since a hostPath can’t follow the kubelet’s working directory). The path is cleaned, not resolved: .. and repeated separators are collapsed, and symlinks are left to the kubelet:

# values.yaml
kubeletRootDir: /mnt/data/kubelet

That one value becomes the driver’s --kubelet-root-dir and both hostPath mounts.

After path cleaning, the value may not contain $( or $$. The kubelet can reinterpret those in a container’s argument, rewriting $$ to a single $ and substituting $(VAR) where that variable is defined, while hostPath.path and volumeMount.mountPath are taken as written. A root still carrying one after cleaning can therefore reach the driver as one directory and the volumes as another.

Only those two sequences are refused, not the character. A lone $, as in /var/lib/$kubelet, is left alone by the kubelet and names the same directory everywhere, so it is accepted.

Cleaning comes first because that is the value the chart renders, so a sequence in a segment the cleaning removes is not refused: /var/lib/$(IGNORED)/../kubelet cleans to /var/lib/kubelet and is accepted. The chart does refuse both sequences outright rather than only the ones the kubelet would act on: an unresolvable $(VAR) is left alone, and the chart does not predict which variables a Pod will have.

Leaving kubeletRootDir out, or setting it to YAML null, selects /var/lib/kubelet. Helm drops a null key while coalescing values, so by the time the chart is rendered it looks the same as a release installed before this value existed, and both have to mean the standard root. An explicit empty string is refused instead.

The flag is newer than the chart’s appVersion, which still selects the v0.2.0 image, so a source-checkout install with a relocated root also needs an image built from a revision that knows the flag — otherwise every node exits on an unknown flag:

helm install dra-driver-cpu ./deployment/helm/dra-driver-cpu -n kube-system \
  --set-string kubeletRootDir=/mnt/data/kubelet \
  --set-string image.repository=REGISTRY/REPOSITORY \
  --set-string image.tag=IMAGE_TAG

A released chart’s appVersion already knows the flag, and a default install (root left unset) emits no flag at all — neither needs this.

A single release’s DaemonSet lands on every node it selects, so those nodes must agree on the kubelet root; splitting them across releases fails with invalid ownership metadata, since the chart’s DeviceClass is cluster-scoped under a fixed name. A cluster whose nodes disagree needs the driver deployed some other way until the chart can vary the root per node.