联系：QQ(5163721)
标题：使用Exadata的calibrate命令测试X5-2的IOPS和MBPS
作者：Lunar©版权所有[文章允许转载,但必须以链接方式注明源地址,否则追究法律责任.]
之前测试的X3的flashcard的IOPS大概是满配200w左右（具体参见 《Exadata上的IOPS和MBPS》）
使用Oracle 11.2的DBMS_RESOURCE_MANAGER.CALIBRATE_IO对Exadata X5（HC）进行测试
使用Exadata的calibrate命令测试时需要注意：
1，这个不能再业务高峰期操作，否则可能造成应用hang等。
2，一般测试的时候，是把cellsrv进程宕掉，否则就要带-force参数。
3，在老板上上，如果这个命令报错，还有一种可能的原因，就是它需要生成一些临时的lun配置文件，然后自己格式化。
如果报错，可以手工自己删除这些临时的lun文件。
新版本目前没有发发现这类问题，因为Oracle已经修改了脚本，所有的文件生成目录都是带日期的（启动测试的日期）。
因此，不会因为不能格式化lun等信息报错，如图
4，Exadata的calibrate命令底层采用orion的方法，根据自己预定义的lun组合，然后压测N轮
（不同的lun组合，压测好多次，自动的过程，无需手工干预）
5，Exadata的calibrate采用只读方式（保护数据安全）
6，不能再一个cell上并发两个calibrate，否则会报错的。
.
本次测试的机器是Exadata X5-2：
CPU型号：
[root@dm01cel02 cellos]# grep -E '^model name|^cpu MHz' /proc/cpuinfo cpu MHz : 2401.000 model name : Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz [root@dm01cel02 cellos]# [root@dm01db01 ibdiagtools]# grep -E '^model name|^cpu MHz' /proc/cpuinfo cpu MHz : 2301.000 model name : Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz [root@dm01db01 ibdiagtools]#
主机型号：
[root@dm01db01 ibdiagtools]# /opt/oracle.SupportTools/CheckHWnFWProfile -c loose [SUCCESS] The hardware and firmware matches supported profile for server=ORACLE_SERVER_X5-2 [root@dm01db01 ibdiagtools]# [root@dm01cel01 ~]# /opt/oracle.SupportTools/CheckHWnFWProfile -c loose [SUCCESS] The hardware and firmware matches supported profile for server=ORACLE_SERVER_X5-2L [root@dm01cel01 ~]#
估算flash card的IOPS：
[root@dm01db01 lunar]# cat a.txt |grep FLASH dm01cel01: LUN 1_1 on drive [FLASH_1_1] random read throughput: 3,227.00 MBPS, and 386910 IOPS dm01cel01: LUN 2_1 on drive [FLASH_2_1] random read throughput: 3,226.00 MBPS, and 357090 IOPS dm01cel01: LUN 4_1 on drive [FLASH_4_1] random read throughput: 3,227.00 MBPS, and 372007 IOPS dm01cel01: LUN 5_1 on drive [FLASH_5_1] random read throughput: 3,226.00 MBPS, and 399469 IOPS dm01cel02: LUN 1_1 on drive [FLASH_1_1] random read throughput: 3,226.00 MBPS, and 366062 IOPS dm01cel02: LUN 2_1 on drive [FLASH_2_1] random read throughput: 3,226.00 MBPS, and 401361 IOPS dm01cel02: LUN 4_1 on drive [FLASH_4_1] random read throughput: 3,226.00 MBPS, and 403004 IOPS dm01cel02: LUN 5_1 on drive [FLASH_5_1] random read throughput: 3,226.00 MBPS, and 408830 IOPS dm01cel03: LUN 1_1 on drive [FLASH_1_1] random read throughput: 3,226.00 MBPS, and 340622 IOPS dm01cel03: LUN 2_1 on drive [FLASH_2_1] random read throughput: 3,226.00 MBPS, and 339174 IOPS dm01cel03: LUN 4_1 on drive [FLASH_4_1] random read throughput: 3,227.00 MBPS, and 405781 IOPS dm01cel03: LUN 5_1 on drive [FLASH_5_1] random read throughput: 3,226.00 MBPS, and 408817 IOPS [root@dm01db01 lunar]#
因此，理论上，这台X5-2 quarter rack的flash lun的IOPS总和超过 450万
.
而在之前测试的X3的flashcard的IOPS大概是满配200w左右（具体参见 《Exadata上的IOPS和MBPS》）：
满配机器（14个cell）hard disk LUNs的MBPS推算： 1962*14个cell=27468
满配机器（14个cell）flash disk LUNs的MBPS推算： 4184*14个cell=58576
满配机器（14个cell）hard disk LUNs的IOPS推算： 4832*14个cell=67648
满配机器（14个cell）flash disk LUNs的IOPS推算：146413*14个cell=2049782
.
下面图是Oracle内部员工（凯耀）写的mon工具（底层基于开源的collect），在当前较新的版本上，貌似脚本还有点问题，我还没有调试。
因此这里只能看到磁盘的吞吐量，没有flash的。
磁盘的吞吐量最高大概3.5GB，CPU平均使用率在17%以下。
这个是执行过程中的一个瞬间的截图，可以看到这组LUN带的参数执行时，基本上是MBPS的较高值了：
这个图就是另外一组lun的orion，可以看到每个节点每次运行测试都是使用6个orion来压测的，这组lun显然MBPS较低：
具体测试如下：
测试时间，大概7分钟
测试方法，使用Exadata自带的calibrate命令：
[root@dm01db01 monitor]# dcli -g /opt/oracle.SupportTools/onecommand/cell_group -l root "cellcli -e calibrate force" dm01cel01: Calibration will take a few minutes... dm01cel01: Aggregate random read throughput across all hard disk LUNs: 1220 MBPS dm01cel01: Aggregate random read throughput across all flash disk LUNs: 12568 MBPS dm01cel01: Aggregate random read IOs per second (IOPS) across all hard disk LUNs: 2889 dm01cel01: Calibrating hard disks (read only) ... dm01cel01: LUN 0_0 on drive [8:0 ] random read throughput: 147.00 MBPS, and 256 IOPS dm01cel01: LUN 0_1 on drive [8:1 ] random read throughput: 142.00 MBPS, and 255 IOPS dm01cel01: LUN 0_10 on drive [8:10 ] random read throughput: 148.00 MBPS, and 244 IOPS dm01cel01: LUN 0_11 on drive [8:11 ] random read throughput: 145.00 MBPS, and 255 IOPS dm01cel01: LUN 0_2 on drive [8:2 ] random read throughput: 145.00 MBPS, and 255 IOPS dm01cel01: LUN 0_3 on drive [8:3 ] random read throughput: 146.00 MBPS, and 238 IOPS dm01cel01: LUN 0_4 on drive [8:4 ] random read throughput: 146.00 MBPS, and 252 IOPS dm01cel01: LUN 0_5 on drive [8:5 ] random read throughput: 146.00 MBPS, and 251 IOPS dm01cel01: LUN 0_6 on drive [8:6 ] random read throughput: 143.00 MBPS, and 255 IOPS dm01cel01: LUN 0_7 on drive [8:7 ] random read throughput: 150.00 MBPS, and 251 IOPS dm01cel01: LUN 0_8 on drive [8:8 ] random read throughput: 151.00 MBPS, and 256 IOPS dm01cel01: LUN 0_9 on drive [8:9 ] random read throughput: 145.00 MBPS, and 255 IOPS dm01cel01: Calibrating flash disks (read only, note that writes will be significantly slower) ... dm01cel01: LUN 1_1 on drive [FLASH_1_1] random read throughput: 3,227.00 MBPS, and 386910 IOPS dm01cel01: LUN 2_1 on drive [FLASH_2_1] random read throughput: 3,226.00 MBPS, and 357090 IOPS dm01cel01: LUN 4_1 on drive [FLASH_4_1] random read throughput: 3,227.00 MBPS, and 372007 IOPS dm01cel01: LUN 5_1 on drive [FLASH_5_1] random read throughput: 3,226.00 MBPS, and 399469 IOPS dm01cel01: CALIBRATE results are within an acceptable range. dm01cel01: Calibration has finished. dm01cel02: Calibration will take a few minutes... dm01cel02: Aggregate random read throughput across all hard disk LUNs: 1195 MBPS dm01cel02: Aggregate random read throughput across all flash disk LUNs: 12827 MBPS dm01cel02: Aggregate random read IOs per second (IOPS) across all hard disk LUNs: 2990 dm01cel02: Calibrating hard disks (read only) ... dm01cel02: LUN 0_0 on drive [8:0 ] random read throughput: 149.00 MBPS, and 254 IOPS dm01cel02: LUN 0_1 on drive [8:1 ] random read throughput: 138.00 MBPS, and 253 IOPS dm01cel02: LUN 0_10 on drive [8:10 ] random read throughput: 148.00 MBPS, and 255 IOPS dm01cel02: LUN 0_11 on drive [8:11 ] random read throughput: 148.00 MBPS, and 248 IOPS dm01cel02: LUN 0_2 on drive [8:2 ] random read throughput: 149.00 MBPS, and 250 IOPS dm01cel02: LUN 0_3 on drive [8:3 ] random read throughput: 150.00 MBPS, and 252 IOPS dm01cel02: LUN 0_4 on drive [8:4 ] random read throughput: 143.00 MBPS, and 252 IOPS dm01cel02: LUN 0_5 on drive [8:5 ] random read throughput: 148.00 MBPS, and 241 IOPS dm01cel02: LUN 0_6 on drive [8:6 ] random read throughput: 148.00 MBPS, and 255 IOPS dm01cel02: LUN 0_7 on drive [8:7 ] random read throughput: 152.00 MBPS, and 256 IOPS dm01cel02: LUN 0_8 on drive [8:8 ] random read throughput: 150.00 MBPS, and 255 IOPS dm01cel02: LUN 0_9 on drive [8:9 ] random read throughput: 150.00 MBPS, and 242 IOPS dm01cel02: Calibrating flash disks (read only, note that writes will be significantly slower) ... dm01cel02: LUN 1_1 on drive [FLASH_1_1] random read throughput: 3,226.00 MBPS, and 366062 IOPS dm01cel02: LUN 2_1 on drive [FLASH_2_1] random read throughput: 3,226.00 MBPS, and 401361 IOPS dm01cel02: LUN 4_1 on drive [FLASH_4_1] random read throughput: 3,226.00 MBPS, and 403004 IOPS dm01cel02: LUN 5_1 on drive [FLASH_5_1] random read throughput: 3,226.00 MBPS, and 408830 IOPS dm01cel02: CALIBRATE results are within an acceptable range. dm01cel02: Calibration has finished. dm01cel03: Calibration will take a few minutes... dm01cel03: Aggregate random read throughput across all hard disk LUNs: 1233 MBPS dm01cel03: Aggregate random read throughput across all flash disk LUNs: 11763 MBPS dm01cel03: Aggregate random read IOs per second (IOPS) across all hard disk LUNs: 3028 dm01cel03: Calibrating hard disks (read only) ... dm01cel03: LUN 0_0 on drive [8:0 ] random read throughput: 148.00 MBPS, and 253 IOPS dm01cel03: LUN 0_1 on drive [8:1 ] random read throughput: 152.00 MBPS, and 255 IOPS dm01cel03: LUN 0_10 on drive [8:10 ] random read throughput: 148.00 MBPS, and 251 IOPS dm01cel03: LUN 0_11 on drive [8:11 ] random read throughput: 154.00 MBPS, and 253 IOPS dm01cel03: LUN 0_2 on drive [8:2 ] random read throughput: 150.00 MBPS, and 255 IOPS dm01cel03: LUN 0_3 on drive [8:3 ] random read throughput: 152.00 MBPS, and 256 IOPS dm01cel03: LUN 0_4 on drive [8:4 ] random read throughput: 148.00 MBPS, and 253 IOPS dm01cel03: LUN 0_5 on drive [8:5 ] random read throughput: 147.00 MBPS, and 255 IOPS dm01cel03: LUN 0_6 on drive [8:6 ] random read throughput: 144.00 MBPS, and 252 IOPS dm01cel03: LUN 0_7 on drive [8:7 ] random read throughput: 153.00 MBPS, and 252 IOPS dm01cel03: LUN 0_8 on drive [8:8 ] random read throughput: 143.00 MBPS, and 256 IOPS dm01cel03: LUN 0_9 on drive [8:9 ] random read throughput: 142.00 MBPS, and 250 IOPS dm01cel03: Calibrating flash disks (read only, note that writes will be significantly slower) ... dm01cel03: LUN 1_1 on drive [FLASH_1_1] random read throughput: 3,226.00 MBPS, and 340622 IOPS dm01cel03: LUN 2_1 on drive [FLASH_2_1] random read throughput: 3,226.00 MBPS, and 339174 IOPS dm01cel03: LUN 4_1 on drive [FLASH_4_1] random read throughput: 3,227.00 MBPS, and 405781 IOPS dm01cel03: LUN 5_1 on drive [FLASH_5_1] random read throughput: 3,226.00 MBPS, and 408817 IOPS dm01cel03: CALIBRATE results are within an acceptable range. dm01cel03: Calibration has finished. [root@dm01db01 monitor]#
