The Exadata X11M platform separates database compute and storage cells connected via an RDMA-capable RoCE fabric. This page summarizes hardware variants and provides practical command-line examples for common administration, troubleshooting and monitoring tasks.

Architecture Overview

Database Servers (Compute)

These servers run the database and client-facing workloads.

  • Standard Exadata X11M Database Server

    • CPU: 2x 96-core AMD EPYC 9J25 (192 physical cores)
    • Memory: 6400 MT/s DDR5, 512 GB–3 TB
    • Local storage: 2x 3.84 TB NVMe (OS / Oracle binaries)

  • Exadata X11M-Z Database Server (Entry)

    • CPU: 1x 32-core AMD EPYC 9J15
    • Memory: 768 GB or 1.125 TB DDR5

Storage Servers

Storage servers perform Smart Scans and offload I/O from database servers.

  • High Capacity (HC) — 12x 22 TB SAS + 4x 6.8 TB NVMe flash; 2x 32-core CPUs; XRMEM up to 1.25 TB
  • Extreme Flash (EF) — All-NVMe capacity-optimized: 4x 30.72 TB NVMe + flash cards; similar CPU/XRMEM
  • HC-Z — Entry HC variant with reduced disk/flash and smaller XRMEM

Networking

  • RoCE fabric (100 Gb/s per port, dual-port active-active → combined throughput) with PCIe 5.0 NICs

Administration & Useful Commands

Tips:

  • Run cellcli on storage cells for local configuration and hardware queries.
  • Use dcli from an administrative host to run commands across groups of cells or database nodes.

Common cellcli commands (run on a storage cell)

  • List physical disks and basic attributes:
cellcli -e "LIST PHYSICALDISK ATTRIBUTES name, disktype, physicalrpm, status"
  • Show logical disks and allocation:
cellcli -e "LIST LOGICALDISK"
  • List cells and status (from a cell):
cellcli -e "LIST CELL ATTRIBUTES name, status"
  • Show flash cache statistics:
cellcli -e "LIST METRICCURRENT WHERE name='FC_BY_USED'"
  • View alert history on the cell:
cellcli -e "LIST ALERTHISTORY"
  • Restart cell services safely:
cellcli -e "ALTER CELL RESTART SERVICES ALL"
  • Export cell configuration:
cellcli -e "LIST CELLCONFIG"

dcli examples (run from an admin host)

  • Run a single cellcli command across a group of cells (hostnames in cell_group):
dcli -g cell_group -l celladmin "cellcli -e 'LIST CELL ATTRIBUTES name, status'"
  • Collect alerthistory from all cells:
dcli -g cell_group -l celladmin "cellcli -e 'LIST ALERTHISTORY WHERE severity=\'critical\''"
  • Set up SSH user-equivalence (one-time, interactive):
dcli -g cell_group -l celladmin -k

VM / Grid Infrastructure quick checks

  • Query GI active version on database nodes:
dcli -g dbs_group -l grid "crsctl query crs activeversion"
  • Check Oracle Clusterware status:
dcli -g dbs_group -l grid "crsctl check crs"
  • OS release on DB nodes:
dcli -g dbs_group -l root "cat /etc/oracle-release"

VM hypervisor commands (run on hypervisors)

  • List VMs on hypervisors:
dcli -g hypervisor_group -l root "vm_maker --list-domains"
  • VM status for a named guest:
dcli -g hypervisor_group -l root "vm_maker --status-domain [vm_name]"

ASM / Storage utilities

  • asmcmd list ASM disks:
asmcmd lsdsk
  • Check Exadata health with exachk (run on a management host):
exachk --summary

Monitoring & Troubleshooting commands

  • Check I/O and CPU on a cell or host:
iostat -x 1 5
vmstat 1 5
top -b -n 1
  • Network statistics (RoCE link health):
ethtool -S <iface>
  • Quick disk SMART check (if supported):
smartctl -a /dev/sdX

Examples: find disks with predictive failure across all cells

dcli -g cell_group -l celladmin "cellcli -e 'LIST PHYSICALDISK WHERE predictiveFailure=TRUE'"

Best-practice notes

  • Always test cellcli commands on a single cell before executing cluster-wide via dcli.
  • Keep exachk reports and cellcli alert exports for post-mortem analysis.

If you’d like, I can now:

  • Start a local Hugo server to preview this page, or
  • Add more specific cellcli one-liners (flash, rebalance, smart_scan stats), or
  • Create a short cheatsheet file under content/ for admins.

Which would you prefer next?