Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,13 @@ Self-service is an important part of CI/CD processes for containerization and mi

### Standard Kubernetes RBAC

$[prodname] implements the standard **Kubernetes RBAC Authorization APIs** with `Role` and `ClusterRole` types. The $[prodname] API server integrates with Kubernetes RBAC Authorization APIs as an extension API server.
$[prodname] implements the standard **Kubernetes RBAC Authorization APIs** with `Role` and `ClusterRole` types. In the default installation, the $[prodname] API server integrates with Kubernetes RBAC Authorization APIs as an extension API server. When using [native v3 CRDs](../../operations/native-v3-crds.mdx), tier RBAC is enforced via an admission webhook instead.

:::note

When using native `projectcalico.org/v3` CRDs, tier RBAC is enforced for **create, update, and delete** operations via the admission webhook. However, **GET, LIST, and WATCH** operations on tiered policies are not enforced because admission webhooks cannot intercept read requests. This is a known limitation.

:::

### RBAC for policies and tiers

Expand Down
180 changes: 180 additions & 0 deletions calico-enterprise/operations/native-v3-crds.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
---
description: Enable native projectcalico.org/v3 CRDs to use Calico resources directly as CRDs without the aggregation API server.
---

# Enable native v3 CRDs

:::note

This feature is tech preview. Tech preview features may be subject to significant changes before they become GA.

:::

## Big picture

Enable native `projectcalico.org/v3` CRDs so that Calico resources are backed directly by CRDs, eliminating the need for the Calico aggregation API server.

## Value

By default, $[prodname] uses an aggregation API server to serve `projectcalico.org/v3` APIs, storing resources internally as `crd.projectcalico.org/v1` CRDs. When using native `projectcalico.org/v3` CRDs, Calico resources are CRDs themselves, which provides several benefits:

- **Simpler architecture** — no aggregation API server to deploy and manage
- **GitOps-friendly** — no ordering dependencies between CRDs and the API server, so tools like ArgoCD and Flux can apply resources in any order
- **Less platform friction** — removes the need for host-network pods and other requirements of the aggregation API server
- **kubectl works directly** — manage `projectcalico.org/v3` resources with `kubectl` without installing the API server separately
- **Native Kubernetes validation and defaulting** — uses CEL validation rules embedded in the CRD schemas and MutatingAdmissionPolicies for defaulting, leveraging built-in Kubernetes mechanisms instead of a custom API server

## Concepts

### How native `projectcalico.org/v3` CRDs work

When using native `projectcalico.org/v3` CRDs:

- $[prodname] resources use the `projectcalico.org/v3` API group and are registered as native Kubernetes CRDs.
- The `APIServer` custom resource is still created, but instead of running the aggregation API server, it deploys a webhooks pod that handles validation and defaulting via admission policies.
- $[prodname] auto-detects the mode at startup based on which CRDs are installed on the cluster. If the `projectcalico.org/v3` CRDs are present, it uses them natively; if the `crd.projectcalico.org/v1` CRDs are present, it runs in API server mode.

### Validation and defaulting

When using native `projectcalico.org/v3` CRDs, resource validation and defaulting are handled by native CRD validation and defaulting, as well as ValidatingAdmissionPolicies and MutatingAdmissionPolicies. $[prodname] uses [MutatingAdmissionPolicies](https://kubernetes.io/docs/reference/access-authn-authz/validating-admission-policy/) for defaulting, which are currently a **beta** Kubernetes feature. You must ensure that the `MutatingAdmissionPolicy` feature gate is enabled on your Kubernetes API server before using native `projectcalico.org/v3` CRDs.

## Before you begin

- A Kubernetes cluster **without** $[prodname] installed, or a cluster where you are performing a fresh install. To migrate an existing cluster from API server mode, see [Migrate from API server to native CRDs](crd-migration.mdx).
- The `MutatingAdmissionPolicy` [feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) must be enabled on the Kubernetes API server. This feature is beta in Kubernetes and is not enabled by default.

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

## How to

### Install $[prodname] with native `projectcalico.org/v3` CRDs

Select the method below based on your preferred installation method.

<Tabs>
<TabItem label="Helm install" value="Helm install-0">

1. Add the $[prodname] Helm repo:

```bash
helm repo add projectcalico https://docs.tigera.io/calico/charts
```

1. Create the `tigera-operator` namespace:

```bash
kubectl create namespace tigera-operator
```

1. Install the v3 CRD chart instead of the default v1 CRD chart:

```bash
helm install calico-crds projectcalico/projectcalico.org.v3 --version $[releaseTitle] --namespace tigera-operator
```

:::note

This replaces the `crd.projectcalico.org.v1` chart used in the default installation. Do not install both CRD charts.

:::

1. Install the Tigera Operator:

```bash
helm install $[prodnamedash] projectcalico/tigera-operator --version $[releaseTitle] --namespace tigera-operator
```

If you have a `values.yaml` with custom configuration:

```bash
helm install $[prodnamedash] projectcalico/tigera-operator --version $[releaseTitle] -f values.yaml --namespace tigera-operator
```

</TabItem>
<TabItem label="Manifest install" value="Manifest install-1">

1. Install the v3 CRDs:

```bash
kubectl create -f $[manifestsUrl]/manifests/v3_projectcalico_org.yaml
```

:::note

This replaces the `v1_crd_projectcalico_org.yaml` manifest used in the default installation. Do not install both CRD manifests.

:::

1. Install the Tigera Operator and custom resources:

```bash
kubectl create -f $[manifestsUrl]/manifests/tigera-operator.yaml
```

</TabItem>
</Tabs>

After installing, complete the following steps:

1. Create the `APIServer` CR to deploy the webhooks pod. This does **not** run the aggregation API server — instead it deploys admission webhooks that handle validation and defaulting.

```bash
kubectl create -f - <<EOF
apiVersion: operator.tigera.io/v1
kind: APIServer
metadata:
name: default
spec: {}
EOF
```

1. Confirm that all pods are running:

```bash
watch kubectl get pods -n calico-system
```

1. Verify that `projectcalico.org` resources are available:

```bash
kubectl api-resources | grep '\sprojectcalico.org'
```

You can now use `kubectl` directly to manage $[prodname] resources, for example:

```bash
kubectl get ippools
kubectl get felixconfigurations
kubectl get bgpconfigurations
```

## Behavioral differences from API server mode

For most clients, the API should continue to behave as expected and consistently with existing behavior. There are a few small exceptions noted below.

### kubectl access

When using native `projectcalico.org/v3` CRDs, `kubectl` can manage Calico resources directly without installing the aggregation API server. There is no separate install step or cache-clearing required.

### IPPool CIDR overlap validation

In API server mode, creating an IPPool with a CIDR that overlaps an existing pool is rejected synchronously at creation time.

When using native `projectcalico.org/v3` CRDs, IPPool CIDR overlap validation is **asynchronous**. Pools with overlapping CIDRs are created successfully but receive a `Disabled` status condition. IPAM does not allocate addresses from disabled pools. Check the IPPool status to identify pools that have been disabled due to CIDR overlap:

```bash
kubectl get ippool <pool-name> -o yaml
```

### Tier RBAC enforcement

In both modes, tier-based RBAC uses the same `ClusterRole` and `RoleBinding` definitions with pseudo-resources like `tier.networkpolicies` and `tier.globalnetworkpolicies`.

In API server mode, tier RBAC is enforced for all operations (create, update, delete, get, list, watch) by the aggregation API server.

When using native `projectcalico.org/v3` CRDs, tier RBAC is enforced via the admission webhook for **create, update, and delete** operations. However, **GET, LIST, and WATCH** operations on tiered policies are **not enforced** because admission webhooks cannot intercept read operations. This is a known limitation.

## Known limitations

- **GET/LIST/WATCH tier RBAC not enforced** — Admission webhooks cannot intercept read operations, so tier-based RBAC for GET, LIST, and WATCH is not enforced when using native `projectcalico.org/v3` CRDs.
2 changes: 2 additions & 0 deletions calico-enterprise/reference/architecture/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,8 @@ The Linseed API uses mTLS to connect to clients, and provides an API to access E

**Main task**: Allows users to manage $[prodname] resources such as policies and tiers through `kubectl` or the Kubernetes API. `kubectl` has significant advantages over `calicoctl` including: audit logging, RBAC using Kubernetes Roles and RoleBindings, and not needing to provide privileged Kubernetes CRD access to anyone who needs to manage resources. [API server](../installation/api.mdx#apiserver).

In the default installation, this is an aggregation API server that translates between `projectcalico.org/v3` and internal CRD representations. When using [native v3 CRDs](../../operations/native-v3-crds.mdx), this component is not used — `kubectl` works directly with `projectcalico.org/v3` CRDs, and validation and defaulting are handled by admission policies instead.

### BIRD

**Main task**: Gets routes from Felix and distributes to BGP peers on the network for inter-host routing. Runs on each node that hosts a Felix agent. Open source, internet routing daemon. [BIRD](../component-resources/node/configuration.mdx#content-main).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,9 @@
- [Default felix configuration](../resources/felixconfig.mdx#spec)

:::note
If you customize felix configuration when you install $[prodname], the `v1 apiVersion` is used. However, when you apply
felix configuration customization after installation (when the calico-apiserver is running), use the `v3 apiVersion`.
If you customize Felix configuration when you install $[prodname], the `crd.projectcalico.org/v1` API group is used. However, when you apply
Felix configuration customization after installation (when the calico-apiserver is running), use the `projectcalico.org/v3` API group.

Check warning on line 25 in calico-enterprise/reference/installation/helm_customization.mdx

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [Vale.Terms] Use 'APIServer' instead of 'apiserver'. Raw Output: {"message": "[Vale.Terms] Use 'APIServer' instead of 'apiserver'.", "location": {"path": "calico-enterprise/reference/installation/helm_customization.mdx", "range": {"start": {"line": 25, "column": 71}}}, "severity": "WARNING"}
When using [native v3 CRDs](../../operations/native-v3-crds.mdx), only the `projectcalico.org/v3` API group is installed and should always be used.
:::

### Sample values.yaml
Expand Down
8 changes: 7 additions & 1 deletion calico-enterprise/reference/resources/ippool.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@

| Field | Description | Accepted Values | Schema | Default |
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------- | --------------------------------------------- |
| cidr | IP range to use for this pool. | A valid IPv4 or IPv6 CIDR. Subnet length must be at least big enough to fit a single block (by default `/26` for IPv4 or `/122` for IPv6). Must not overlap with the Link Local range `169.254.0.0/16` or `fe80::/10`. | string | |
| cidr | IP range to use for this pool. See [CIDR overlap validation](#cidr-overlap-validation) for details on overlap behavior. | A valid IPv4 or IPv6 CIDR. Subnet length must be at least big enough to fit a single block (by default `/26` for IPv4 or `/122` for IPv6). Must not overlap with the Link Local range `169.254.0.0/16` or `fe80::/10`. | string | |

Check warning on line 44 in calico-enterprise/reference/resources/ippool.mdx

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [Vale.Terms] Use 'CIDRs?' instead of 'cidr'. Raw Output: {"message": "[Vale.Terms] Use 'CIDRs?' instead of 'cidr'.", "location": {"path": "calico-enterprise/reference/resources/ippool.mdx", "range": {"start": {"line": 44, "column": 3}}}, "severity": "WARNING"}
| blockSize | The CIDR size of allocation blocks used by this pool. Blocks are allocated on demand to hosts and are used to aggregate routes. The value can only be set when the pool is created. | 20 to 32 (inclusive) for IPv4 and 116 to 128 (inclusive) for IPv6 | int | `26` for IPv4 pools and `122` for IPv6 pools. |
| ipipMode | The mode defining when IPIP will be used. Cannot be set at the same time as `vxlanMode`. | Always, CrossSubnet, Never | string | `Never` |
| vxlanMode | The mode defining when VXLAN will be used. Cannot be set at the same time as `ipipMode`. | Always, CrossSubnet, Never | string | `Never` |
Expand Down Expand Up @@ -82,6 +82,12 @@

$[prodname] supports Kubernetes [annotations that force the use of specific IP addresses](../component-resources/configuration.mdx#requesting-a-specific-ip-address). These annotations take precedence over the `allowedUses` field.

### CIDR overlap validation

By default (API server mode), creating an IPPool with a CIDR that overlaps an existing pool is rejected synchronously at creation time.

When using [native v3 CRDs](../../operations/native-v3-crds.mdx), CIDR overlap validation is **asynchronous**. Pools with overlapping CIDRs are created successfully but receive a `Disabled` status condition. IPAM does not allocate addresses from disabled pools. Check the IPPool status to identify any pools that have been disabled due to CIDR overlap.

### AWS-backed pools

$[prodname] supports IP pools that are backed by the AWS fabric. This feature was added in order
Expand Down
12 changes: 12 additions & 0 deletions calico/getting-started/kubernetes/helm.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,18 @@ For more information about configurable options via `values.yaml` please see [He
helm install calico-crds projectcalico/crd.projectcalico.org.v1 --version $[releaseTitle] --namespace tigera-operator
```

:::tip

To install with [native v3 CRDs](../../operations/native-v3-crds.mdx) (tech preview) instead, use the v3 CRD chart:

```bash
helm install calico-crds projectcalico/projectcalico.org.v3 --version $[releaseTitle] --namespace tigera-operator
```

Native v3 CRDs eliminate the need for the aggregation API server and allows `kubectl` to manage `projectcalico.org/v3` resources directly.

:::

1. Install the Tigera Operator using the Helm chart:

```bash
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,13 @@ Self-service is an important part of CI/CD processes for containerization and mi

### Standard Kubernetes RBAC

$[prodname] implements the standard **Kubernetes RBAC Authorization APIs** with `Role` and `ClusterRole` types. The $[prodname] API server integrates with Kubernetes RBAC Authorization APIs as an extension API server.
$[prodname] implements the standard **Kubernetes RBAC Authorization APIs** with `Role` and `ClusterRole` types. In the default installation, the $[prodname] API server integrates with Kubernetes RBAC Authorization APIs as an extension API server. When using [native v3 CRDs](../../operations/native-v3-crds.mdx), tier RBAC is enforced via an admission webhook instead.

:::note

When using native `projectcalico.org/v3` CRDs, tier RBAC is enforced for **create, update, and delete** operations via the admission webhook. However, **GET, LIST, and WATCH** operations on tiered policies are not enforced because admission webhooks cannot intercept read requests. This is a known limitation.

:::

### RBAC for policies and tiers

Expand Down
2 changes: 1 addition & 1 deletion calico/operations/calicoctl/install.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ should still be used to manage other Kubernetes resources.
:::note

If you would like to use `kubectl` to manage `projectcalico.org/v3` API resources, you can use the
[Calico API server](../install-apiserver.mdx).
[Calico API server](../install-apiserver.mdx). Alternatively, when using [native v3 CRDs](../native-v3-crds.mdx), `kubectl` can manage `projectcalico.org/v3` resources directly as native CRDs without needing the API server or calicoctl for resource management.

:::

Expand Down
Loading
Loading