Exadata Cloud@Customer Architecture: Infrastructure, VM Clusters, and Operational Model

Exadata Cloud@Customer (ExaCC) delivers the Oracle Cloud control plane experience on Exadata hardware that physically resides in the customer’s data centre. For organisations with data residency requirements, latency-sensitive applications that cannot tolerate public cloud networking, or regulatory constraints that prohibit off-premises data, ExaCC is the path to Autonomous Database and managed Exadata without public cloud.

Understanding ExaCC’s architecture — particularly the control plane / data plane separation and the VM Cluster model — is essential for architects sizing, operating, and troubleshooting the platform.

Physical Architecture

ExaCC is delivered as a factory-racked Exadata system. The current generation is Exadata X9M-2 or X10M. The physical components are identical to on-premises Exadata:

• Compute nodes (DB servers): 2+ nodes running Oracle Linux. Your databases run here.
• Storage cells: 3+ cells providing Exadata Smart Flash Cache, HCC compression, and Smart Scan offload.
• RDMA over Converged Ethernet (RoCE) interconnect: High-bandwidth, ultra-low-latency private network between compute and storage nodes.
• InfiniBand (pre-X8M) or RoCE (X8M+) cluster interconnect: Used by Oracle RAC for Cache Fusion.

ExaCC adds two additional management components:

• Exadata Control Plane Server (CPS): Two small x86 servers that act as the management bridge between the OCI control plane and the hardware. The CPS handles provisioning, patching, and monitoring signals.
• KVM switch and ILOM: Out-of-band management for hardware-level operations.

Control Plane vs Data Plane: The Critical Distinction

This is the most important architectural concept for ExaCC operations:

SQL and data never leave the customer’s network. Only management and metadata telemetry flow to OCI. This is the fundamental data residency guarantee.

The CPS requires outbound HTTPS connectivity to specific OCI endpoints. Work with your network team to whitelist these; ExaCC will not provision or receive patches without this connectivity.

# Verify CPS connectivity to OCI endpoints (run on CPS nodes)
curl -v https://iaas.eu-frankfurt-1.oraclecloud.com
curl -v https://telemetry-ingestion.eu-frankfurt-1.oraclecloud.com

VM Clusters: The Provisioning Unit

On ExaCC, databases do not run directly on bare metal compute nodes. Instead, compute nodes are divided into VM Clusters — groups of virtual machines, one per compute node, sharing a fraction of the compute node’s resources.

VM Cluster Design Considerations

CPU allocation: Each compute node has a fixed number of OCPU (Oracle CPUs). OCPUs are divided across VM Clusters. Example: an X9M-2 with 2 compute nodes, each with 96 OCPUs, gives 192 total. Allocate OCPUs across VM Clusters based on workload demand, leaving headroom for bursting.

Storage: VM Clusters share the storage cell pool. Storage is allocated at the Grid Infrastructure disk group level — not per VM Cluster. Plan ASM disk groups carefully:

+DATA   → Database data files (ASM normal or high redundancy)
+RECO   → RMAN backups, archive logs, FRA
+SPARSE → Snapshot clones (if using Test Master / Clone feature)

Memory: Each VM is provisioned with a fixed memory allocation. Ensure memory per VM is sufficient for the SGA + PGA + OS overhead of all planned databases on that VM.

Provisioning a VM Cluster

VM Clusters are provisioned via the OCI Console or OCI CLI:

oci db vm-cluster create \
  --compartment-id <compartment_ocid> \
  --display-name "prod-vm-cluster" \
  --exadata-infrastructure-id <exacc_infra_ocid> \
  --cpu-core-count 32 \
  --data-storage-size-in-tbs 20 \
  --ssh-public-keys "$(cat ~/.ssh/id_rsa.pub)" \
  --gi-version "19.0.0.0" \
  --db-servers '["<dbserver1_ocid>","<dbserver2_ocid>"]'

Grid Infrastructure and ASM on ExaCC

ExaCC VM Clusters use Oracle Grid Infrastructure (Clusterware + ASM) just as on-premises Exadata does. This means all the standard RAC and ASM operational procedures apply inside the VMs.

# On a compute node VM: check cluster status
crsctl stat res -t

# Check ASM disk group status
asmcmd
ASMCMD> lsdg

# Run exachk health check
/u01/app/grid/product/19.0.0/grid/bin/exachk

The key operational difference from on-premises Exadata: patching is initiated and orchestrated via OCI, not manually by DBAs. The OCI Console provides a patch list. You apply patches through the OCI API/Console, and Oracle’s automation handles the rolling patch application across compute nodes and storage cells.

Patching Model

Patching is one of the biggest operational differences between ExaCC and on-premises Exadata:

• Grid Infrastructure / OS patches: Applied via OCI Console → VM Cluster → Apply Patch. Rolling, one node at a time.
• Database Home patches: Applied per DB Home. Multiple DB Homes per VM Cluster are supported, enabling canary testing.
• Storage cell patches: Orchestrated by Oracle. You approve the maintenance window; Oracle applies the patch.

You cannot SSH into storage cells on ExaCC. cellcli commands are not available to customers — this is a departure from on-premises Exadata where storage-cell-level CLI access is routine. All storage cell operations go through OCI or through database-level views (V$CELL, V$CELL_METRIC).

-- Query cell metrics from database layer (no cellcli access on ExaCC)
SELECT cell_name, metric_name, metric_value, metric_unit
FROM v$cell_metric
WHERE metric_name IN (
    'CD_BY_FC_DIRTY',
    'CD_BY_FC_TOTAL',
    'CL_CPUT',
    'CL_IORMTHR'
)
ORDER BY cell_name, metric_name;

Backup Architecture on ExaCC

ExaCC integrates natively with OCI Object Storage for RMAN backups. Backups are transmitted from the data plane (on-premises) to OCI Object Storage over the customer’s internet or FastConnect link.

For customers where backup data must also remain on-premises:

• Use a local NFS target (configure RMAN SBT_TAPE or DISK channel to NFS).
• Or use Autonomous Recovery Service (Zero Data Loss Recovery Appliance in OCI) if connected via FastConnect.

# Configure RMAN channel for OCI Object Storage
rman target /
RMAN> CONFIGURE CHANNEL DEVICE TYPE SBT
PARMS 'SBT_LIBRARY=/opt/oracle/extapi/64/oss/libociopc.so,
       ENV=(OPC_PFILE=/etc/opc_exacc.pfile)';

RMAN> BACKUP DATABASE PLUS ARCHIVELOG;

ExaCC vs ExaCS vs On-Premises Exadata

ExaCC is the right choice when data must remain physically on-premises but you want Oracle to manage patching, provisioning, and lifecycle operations. If data can reside in OCI regions, ExaCS gives faster provisioning and lower operational overhead.