When multiple VM Clusters share the same Exadata storage cells on ExaCC, storage I/O becomes a shared resource that can interfere between workloads. Oracle’s I/O Resource Manager (IORM) is the mechanism for controlling how storage I/O is distributed across VM Clusters, databases, and consumer groups. Without IORM configuration, a single runaway workload on one VM Cluster can starve others.

IORM Architecture on ExaCC

IORM operates at two levels:

  1. Inter-VM Cluster IORM (configured via OCI): Controls storage I/O allocation between VM Clusters sharing the same physical Exadata infrastructure.
  2. Intra-database IORM (configured via DBCA or DBRM): Controls I/O between databases within a single VM Cluster and between consumer groups within a database.

Because ExaCC does not give customers cellcli access, inter-VM Cluster IORM is configured through the OCI Console or API — Oracle’s management layer pushes the configuration to the storage cells.

Configuring Inter-VM Cluster IORM via OCI

# OCI CLI: configure IORM for the Exadata infrastructure
oci db exadata-infrastructure update \
  --exadata-infrastructure-id <infra_ocid> \
  --db-server-version 19.3.0.0 \
  --maintenance-window '{"preference":"NO_PREFERENCE"}' \
  # IORM configuration is set via the Console or separate API call

# Update IORM plan for the infrastructure
oci db exadata-infrastructure update-exadata-infrastructure-exadata-infrastructure-update-details \
  --exadata-infrastructure-id <infra_ocid> \
  --db-plans '[{"dbName":"ALL", "share":1, "flashCacheLimit":"0G", "flashCacheMin":"0G", "allocation":null}]'

In practice, the OCI Console provides a cleaner interface for IORM plan configuration. Navigate to: Exadata Infrastructure → IORM Plan and assign share weights per database or VM Cluster.

Share-Based Allocation

IORM uses share-based allocation: higher share numbers receive proportionally more I/O when the system is under contention.

IORM Plan Example:
  VM Cluster: PROD    → Share: 8  (highest priority OLTP)
  VM Cluster: DWH     → Share: 4  (analytics batch)
  VM Cluster: DEV     → Share: 1  (development/test)

Under contention, the PROD cluster gets 8/(8+4+1) = ~62% of storage I/O. When the system is idle, all VM Clusters get full throughput regardless of share weights — IORM only throttles during resource contention.

Intra-Database IORM

Within a database, Oracle Database Resource Manager (DBRM) integrates with Exadata IORM to allocate I/O to consumer groups. Configure IORM directives via DBMS_RESOURCE_MANAGER:

-- Create an IORM plan
BEGIN
  DBMS_RESOURCE_MANAGER.CREATE_PENDING_AREA();

  DBMS_RESOURCE_MANAGER.CREATE_PLAN(
    plan    => 'PROD_IORM_PLAN',
    comment => 'Production IORM plan'
  );

  -- Directive: OLTP consumer group gets high I/O priority
  DBMS_RESOURCE_MANAGER.CREATE_PLAN_DIRECTIVE(
    plan                  => 'PROD_IORM_PLAN',
    group_or_subplan      => 'OLTP_GROUP',
    comment               => 'OLTP workloads',
    mgmt_p1               => 60,   -- 60% CPU
    asm_io_limit          => NULL  -- no I/O cap
  );

  -- Directive: Batch jobs capped on I/O
  DBMS_RESOURCE_MANAGER.CREATE_PLAN_DIRECTIVE(
    plan             => 'PROD_IORM_PLAN',
    group_or_subplan => 'BATCH_GROUP',
    comment          => 'Batch/analytics',
    mgmt_p1          => 20,
    asm_io_limit     => 500   -- cap at 500 MB/s
  );

  DBMS_RESOURCE_MANAGER.VALIDATE_PENDING_AREA();
  DBMS_RESOURCE_MANAGER.SUBMIT_PENDING_AREA();
END;
/

-- Activate the plan
ALTER SYSTEM SET RESOURCE_MANAGER_PLAN = 'PROD_IORM_PLAN' SCOPE=BOTH;

Monitoring I/O Performance and IORM Effectiveness

Database-Level I/O Statistics

-- Per-consumer group I/O wait statistics
SELECT consumer_group_name,
       io_requests,
       io_megabytes,
       io_wait_time / 1000000 AS io_wait_secs
FROM v$rsrcmgrmetric
WHERE consumer_group_name NOT IN ('SYS_GROUP', '_ORACLE_BACKGROUND_GROUP_')
ORDER BY io_wait_secs DESC;

Cell-Level Metrics via V$CELL_METRIC

-- Storage cell throughput by metric
SELECT cell_name,
       metric_name,
       ROUND(metric_value, 2) AS value,
       metric_unit
FROM v$cell_metric
WHERE metric_name IN (
    'CD_BY_FC_DIRTY',        -- Flash cache dirty bytes
    'CD_BY_FC_TOTAL',        -- Total flash cache utilisation
    'CL_IORMTHR',            -- IORM throttle percentage
    'DB_IO_RQ_LG_SEC',       -- Large I/O requests per second
    'DB_IO_RQ_SM_SEC',       -- Small I/O requests per second
    'DB_IO_BY_SEC'           -- Total I/O throughput (bytes/sec)
)
ORDER BY cell_name, metric_name;

Identifying IORM Throttling

When IORM is actively throttling, sessions accumulate on cell smart table scan or show extended cell single block physical read wait times with inflated latency. Query V$SESSION_EVENT to identify:

SELECT s.sid,
       s.username,
       s.service_name,
       s.sql_id,
       se.event,
       ROUND(se.time_waited / 100, 2) AS wait_secs
FROM v$session s
JOIN v$session_event se ON s.sid = se.sid
WHERE se.event LIKE 'cell%'
  AND s.status = 'ACTIVE'
  AND se.time_waited > 1000
ORDER BY wait_secs DESC;

Smart Flash Cache on ExaCC

The Exadata Smart Flash Cache (SFC) is the NVMe flash layer on each storage cell. It serves as a write-back cache for OLTP workloads and read cache for Smart Scan.

Flash Cache Usage

-- Flash cache hit ratio and utilisation
SELECT metric_name,
       ROUND(AVG(metric_value), 2) AS avg_value,
       metric_unit
FROM v$cell_metric_history
WHERE metric_name IN (
    'CD_BY_FC_TOTAL',
    'CD_IO_BY_R_FC_SEC',     -- Flash cache read throughput
    'CD_IO_BY_R_HD_SEC'      -- Disk read throughput (cache miss)
)
GROUP BY metric_name, metric_unit
ORDER BY metric_name;

A high ratio of CD_IO_BY_R_FC_SEC to CD_IO_BY_R_HD_SEC indicates the flash cache is effective — most reads are served from flash, not spinning disk.

Controlling Flash Cache Caching

For workloads that should not pollute the flash cache (large, single-use ETL scans), use the CELL_FLASH_CACHE storage clause to prevent cache eviction of hot OLTP data:

-- Modify table to not use flash cache (ETL staging tables)
ALTER TABLE staging_large_table STORAGE (CELL_FLASH_CACHE NONE);

-- Force a table to always be cached in flash
ALTER TABLE hot_lookup_table STORAGE (CELL_FLASH_CACHE KEEP);

HCC Compression Tiering on ExaCC

Hybrid Columnar Compression (HCC) is available on ExaCC just as on on-premises Exadata. The compression tier should match the access pattern:

-- High compression for archive (query only rarely)
ALTER TABLE sales_archive MOVE COMPRESS FOR ARCHIVE HIGH;

-- Query-optimised compression for analytics tables
ALTER TABLE sales_fact MOVE COMPRESS FOR QUERY HIGH;

-- Basic row compression for OLTP tables (HCC not appropriate here)
ALTER TABLE orders MOVE COMPRESS BASIC;

Monitor space savings:

SELECT table_name,
       compress_for,
       blocks,
       ROUND(blocks * 8192 / 1024 / 1024, 1) AS size_mb
FROM dba_tables
WHERE owner = 'SCHEMA1'
  AND compression = 'ENABLED'
ORDER BY blocks DESC;

Operational Summary

ExaCC storage tuning follows the same principles as on-premises Exadata, with two key differences:

  1. IORM inter-cluster plans are managed via OCI, not cellcli.
  2. Storage cell-level diagnostics are available via V$CELL_METRIC and V$CELL_METRIC_HISTORY — there is no direct cellcli access.

Focus areas for ExaCC storage performance:

  • Set IORM share weights appropriate to business priority of each VM Cluster.
  • Use CELL_FLASH_CACHE NONE on bulk ETL tables to protect flash cache for OLTP.
  • Apply HCC QUERY HIGH on read-mostly analytical tables to maximise storage efficiency.
  • Monitor CL_IORMTHR metric — any non-zero value indicates active throttling.