Skip to main content
NetApp Stage KB

Why does the sum of all volume IOPs in an aggregate not match the aggregate IOPs?

Last Updated:

Applies to

  • OnCommand Unified Manager (OCUM)
  • Active IQ Unified Manager (AIQUM)
  • ONTAP 9
  • Clustered Data ONTAP 8.3


  • ONTAP has many different background tasks that take aggregate cycles in different quantities of IOPS than volume IOPS
  • Data that is initally read from disk will be cached, as ONTAP is optimized to keep most used and predicted next blocks in cache to avoid going to disk for each IOP for best performance
    • This means more volume IOPS will be counted than disk IOPS as the workload is heavily cached
    • The opposite can be true if excessive background operations are happening and little volume workload is happening
    • Example: If 10 blocks of data in a volume are being read repeatedly at a rate of 1000 IOPS, the first two blocks may be read, the next eight prefetched by readahead, and the blocks sit in cache (RAM) to service the 1000 IOPS


Why may aggregates have different op counts than the volume?
  • There are many background operations ONTAP does which decouple it from volume work
    • Background workloads such as SnapMirror, wafl scanners, deduplication, etc. all take disk IOPs but would not count as a user IOP
    • These are triggered by internal configurations or schedules
    • They are tracked in the internal qos statistics workloads
  • Workloads have different behaviors within ONTAP
    • IOPS get mixed together as requests come in
      • Example: One volume may have 4kB reads, the other 1 MB reads, and there may be write workloads which do not get written to disk until the Consistency Point
    • One IOP on the network may trigger several WAFL IOPS or it may take several IOPS on the network to make one WAFL IOP


How do different IOP types (read, write, other, and small or large IOPS) affect volume IOP counts compared to disk?
  • Reads that read the same data repeatedly are cached after the first hit
  • Readahead will fetch the blocks that are needed before requested, which causes more disk IOPS than volume IOPS
  • Large reads (> 64 kB) have one volume IOP but multiple disk IOPS are merged
  • Writes go through Consistency Points, and are not written to disk
  • Other IOPS that do not modify file system structures generally are in cache
    • Example: A volume may have 100,000 GETATTR IOPS but not use disk to read the data after initial load of workload
  • Other IOPs that do modify file system structures may require several disk IOPS to update metadata and data blocks
    • Example: A SETATTR that overwrites a file has to:
      1. Update metadata
      2. Mark the existing block free
      3. Write new data in Consistency Point on new spot on disk due to WAFL behavior


Should we use these outputs in AIQUM to see if more workload can be added

Additional Information

Example: The volume IOPs (less than 2000 total) in this aggregate do not total the aggregate IOPs in Active IQ Unified Manager


NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.