/killer/a SCSI ID 1 opensolaris 132 /killer/b SCSI ID 2 tintri /killer/c SCSI ID 3 tegile *** Nexsan shipped their box and got it online on 3/8/2012 in the late *** afternoon, I have added their data below but I did not measure *** everything, just the bonnie++ and postmark output. 50 GiB volumes for each of them. OpenSolaris system built using 1 TiB Caviar Black drives built as 2 9 drive RAIDZ2 groups. Compression is 'on', dedupe is 'off'. System has 8 GiB of RAM and 2 single core AMD 240 processors. This system is slow in comparison to the appliances in the test. First test - brand new volumes, make filesystems, run bonnie++ against said volumes. Second test - fill volumes with random data then run bonnie++ against said volumes again. Nexsan was not tested. Third test - delete volumes and recreate from VMware vCenter, run postmark against said volumes. FreeBSD system: Feb 29 13:52:42 bm-f kernel: da1 at mpt0 bus 0 scbus0 target 1 lun 0 Feb 29 13:52:42 bm-f kernel: da1: Fixed Direct Access SCSI-2 device Feb 29 13:52:42 bm-f kernel: da1: 300.000MB/s transfers Feb 29 13:52:42 bm-f kernel: da1: Command Queueing enabled Feb 29 13:52:42 bm-f kernel: da1: 51200MB (104857600 512 byte sectors: 255H 63S/T 6527C) Feb 29 13:52:42 bm-f kernel: da2 at mpt0 bus 0 scbus0 target 2 lun 0 Feb 29 13:52:42 bm-f kernel: da2: Fixed Direct Access SCSI-2 device Feb 29 13:52:42 bm-f kernel: da2: 300.000MB/s transfers Feb 29 13:52:42 bm-f kernel: da2: Command Queueing enabled Feb 29 13:52:42 bm-f kernel: da2: 51200MB (104857600 512 byte sectors: 255H 63S/T 6527C) Feb 29 13:52:43 bm-f kernel: da3 at mpt0 bus 0 scbus0 target 3 lun 0 Feb 29 13:52:43 bm-f kernel: da3: Fixed Direct Access SCSI-2 device Feb 29 13:52:43 bm-f kernel: da3: 300.000MB/s transfers Feb 29 13:52:43 bm-f kernel: da3: Command Queueing enabled Feb 29 13:52:43 bm-f kernel: da3: 51200MB (104857600 512 byte sectors: 255H 63S/T 6527C) # time newfs -b 16384 -f 2048 -U /dev/da1s1a 0.042u 0.018s 0:00.51 9.8% 56+2078k 7+291io 0pf+0w # time newfs -b 16384 -f 2048 -U /dev/da2s1a 0.024u 0.031s 0:00.62 8.0% 50+1853k 7+291io 0pf+0w # time newfs -b 16384 -f 2048 -U /dev/da3s1a 0.042u 0.028s 0:00.60 10.0% 70+2810k 7+291io 0pf+0w _____ _ |_ _|_ _ ___| | __ ___ _ __ ___ | |/ _` / __| |/ / / _ \| '_ \ / _ \ | | (_| \__ \ < | (_) | | | | __/_ |_|\__,_|___/_|\_\ \___/|_| |_|\___(_) bonnie++ -u nobody -d /killer/a -m bm-f-os -n 10:102400:1024:1024 Using uid:65534, gid:65534. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP bm-f-os 4G 586 99 58555 33 32197 20 1066 99 44241 18 4972 251 Latency 17363us 1807ms 761ms 33654us 536ms 186ms Version 1.96 ------Sequential Create------ --------Random Create-------- bm-f-os -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 10:102400:1024/1024 1107 34 4066 92 +++++ +++ 1294 37 4046 94 +++++ +++ Latency 993ms 48985us 278us 31053us 13879us 1568us 1.96,1.96,bm-f-os,1,1330552257,4G,,586,99,58555,33,32197,20,1066,99,44241,18,4972,251,10,102400,1024,,1024,1107,34,4066,92,+++++,+++,1294,37,4046,94,+++++,+++,17363us,1807ms,761ms,33654us,536ms,186ms,993ms,48985us,278us,31053us,13879us,1568us bonnie++ -u nobody -d /killer/b -m bm-f-tintri -n 10:102400:1024:1024 Using uid:65534, gid:65534. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP bm-f-tintri 4G 586 99 107082 58 29040 18 1075 99 42986 18 3913 185 Latency 18151us 24788us 299ms 32470us 338ms 126ms Version 1.96 ------Sequential Create------ --------Random Create-------- bm-f-tintri -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 10:102400:1024/1024 1611 50 3903 92 +++++ +++ 1505 43 4020 94 +++++ +++ Latency 15635us 2453us 362us 222ms 3434us 422us 1.96,1.96,bm-f-tintri,1,1330552364,4G,,586,99,107082,58,29040,18,1075,99,42986,18,3913,185,10,102400,1024,,1024,1611,50,3903,92,+++++,+++,1505,43,4020,94,+++++,+++,18151us,24788us,299ms,32470us,338ms,126ms,15635us,2453us,362us,222ms,3434us,422us bonnie++ -u nobody -d /killer/c -m bm-f-tegile -n 10:102400:1024:1024 Using uid:65534, gid:65534. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP bm-f-tegile 4G 580 99 95185 52 31309 19 1068 99 51814 22 5151 236 Latency 17899us 87945us 520ms 35984us 559ms 129ms Version 1.96 ------Sequential Create------ --------Random Create-------- bm-f-tegile -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 10:102400:1024/1024 1427 44 3900 92 +++++ +++ 1444 41 4086 93 +++++ +++ Latency 498ms 1757us 5887us 76327us 8685us 605us 1.96,1.96,bm-f-tegile,1,1330550180,4G,,580,99,95185,52,31309,19,1068,99,51814,22,5151,236,10,102400,1024,,1024,1427,44,3900,92,+++++,+++,1444,41,4086,93,+++++,+++,17899us,87945us,520ms,35984us,559ms,129ms,498ms,1757us,5887us,76327us,8685us,605us bonnie++ -u nobody -d /killer/b -m bm-f-nexsan -n 10:102400:1024:1024Using uid:65534, gid:65534 Using uid:65534, gid:65534. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP bm-f-nexsan 4G 547 93 16504 10 23397 15 1063 99 56457 22 3644 149 Latency 17809us 649ms 675ms 9251us 528ms 973ms Version 1.96 ------Sequential Create------ --------Random Create-------- bm-f-nexsan -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 10:102400:1024/1024 320 10 4397 91 9601 31 334 9 4232 88 9022 20 Latency 1653ms 2817us 1600us 710ms 92203us 247us 1.96,1.96,bm-f-nexsan,1,1331343916,4G,,547,93,16504,10,23397,15,1063,99,56457,22,3644,149,10,102400,1024,,1024,320,10,4397,91,9601,31,334,9,4232,88,9022,20,17809us,649ms,675ms,9251us,528ms,973ms,1653ms,2817us,1600us,710ms,92203us,247us Ubuntu system: Feb 29 13:54:21 bm-l kernel: [169551.696117] vmw_pvscsi: msg type: 0x0 - MSG RING: 5/4 (5) Feb 29 13:54:21 bm-l kernel: [169551.696120] vmw_pvscsi: msg: device added at scsi0:1:0 Feb 29 13:54:21 bm-l kernel: [169551.696613] scsi 2:0:1:0: Direct-Access VMware Virtual disk 1.0 PQ: 0 ANSI: 2 Feb 29 13:54:21 bm-l kernel: [169551.696875] sd 2:0:1:0: Attached scsi generic sg2 type 0 Feb 29 13:54:21 bm-l kernel: [169551.697011] sd 2:0:1:0: [sdb] 104857600 512-byte logical blocks: (53.6 GB/50.0 GiB) Feb 29 13:54:21 bm-l kernel: [169551.697040] sd 2:0:1:0: [sdb] Write Protect is off Feb 29 13:54:21 bm-l kernel: [169551.697078] sd 2:0:1:0: [sdb] Cache data unavailable Feb 29 13:54:21 bm-l kernel: [169551.698801] sd 2:0:1:0: [sdb] Cache data unavailable Feb 29 13:54:21 bm-l kernel: [169551.700430] sdb: unknown partition table Feb 29 13:54:21 bm-l kernel: [169551.703407] sd 2:0:1:0: [sdb] Cache data unavailable Feb 29 13:54:21 bm-l kernel: [169551.704865] sd 2:0:1:0: [sdb] Attached SCSI disk Feb 29 13:54:21 bm-l kernel: [169551.772203] vmw_pvscsi: msg type: 0x0 - MSG RING: 6/5 (5) Feb 29 13:54:21 bm-l kernel: [169551.772206] vmw_pvscsi: msg: device added at scsi0:2:0 Feb 29 13:54:21 bm-l kernel: [169551.772212] vmw_pvscsi: device already exists Feb 29 13:54:21 bm-l kernel: [169551.856419] vmw_pvscsi: msg type: 0x0 - MSG RING: 7/6 (5) Feb 29 13:54:21 bm-l kernel: [169551.856422] vmw_pvscsi: msg: device added at scsi0:3:0 Feb 29 13:54:21 bm-l kernel: [169551.856545] scsi 2:0:3:0: Direct-Access VMware Virtual disk 1.0 PQ: 0 ANSI: 2 Feb 29 13:54:21 bm-l kernel: [169551.856747] sd 2:0:3:0: Attached scsi generic sg4 type 0 Feb 29 13:54:21 bm-l kernel: [169551.856789] sd 2:0:3:0: [sdd] 104857600 512-byte logical blocks: (53.6 GB/50.0 GiB) Feb 29 13:54:21 bm-l kernel: [169551.856822] sd 2:0:3:0: [sdd] Write Protect is off Feb 29 13:54:21 bm-l kernel: [169551.856850] sd 2:0:3:0: [sdd] Cache data unavailable Feb 29 13:54:21 bm-l kernel: [169551.862720] sd 2:0:3:0: [sdd] Cache data unavailable Feb 29 13:54:21 bm-l kernel: [169551.864162] sdd: unknown partition table Feb 29 13:54:21 bm-l kernel: [169551.865561] sd 2:0:3:0: [sdd] Cache data unavailable Feb 29 13:54:21 bm-l kernel: [169551.867094] sd 2:0:3:0: [sdd] Attached SCSI disk # time mkfs.ext4 /dev/sdb1 real 0m13.904s user 0m0.070s sys 0m1.120s # time mkfs.ext4 /dev/sdc1 real 0m9.195s user 0m0.060s sys 0m1.120s # time mkfs.ext4 /dev/sdd1 real 0m9.534s user 0m0.070s sys 0m1.090s bonnie++ -u nobody -d /killer/a -m bm-l-os -n 10:102400:1024:1024 Using uid:65534, gid:65534. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP bm-l-os 4G 647 98 72169 9 45789 6 2039 61 117149 6 8789 116 Latency 12645us 923ms 1421ms 478ms 522ms 2378us Version 1.96 ------Sequential Create------ --------Random Create-------- bm-l-os -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 10:102400:1024/1024 1324 9 +++++ +++ +++++ +++ 1352 10 +++++ +++ +++++ +++ Latency 144ms 366us 328us 541ms 272us 340us 1.96,1.96,bm-l-os,1,1330548672,4G,,647,98,72169,9,45789,6,2039,61,117149,6,8789,116,10,102400,1024,,1024,1324,9,+++++,+++,+++++,+++,1352,10,+++++,+++,+++++,+++,12645us,923ms,1421ms,478ms,522ms,2378us,144ms,366us,328us,541ms,272us,340us bonnie++ -u nobody -d /killer/b -m bm-l-tintri -n 10:102400:1024:1024 Using uid:65534, gid:65534. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP bm-l-tintri 4G 650 98 103964 13 48917 5 3325 99 118347 5 6522 79 Latency 13180us 391ms 412ms 3491us 112ms 3377us Version 1.96 ------Sequential Create------ --------Random Create-------- bm-l-tintri -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 10:102400:1024/1024 1720 13 +++++ +++ +++++ +++ 1751 13 +++++ +++ +++++ +++ Latency 541ms 282us 336us 74439us 334us 671us 1.96,1.96,bm-l-tintri,1,1330544676,4G,,650,98,103964,13,48917,5,3325,99,118347,5,6522,79,10,102400,1024,,1024,1720,13,+++++,+++,+++++,+++,1751,13,+++++,+++,+++++,+++,13180us,391ms,412ms,3491us,112ms,3377us,541ms,282us,336us,74439us,334us,671us bonnie++ -u nobody -d /killer/c -m bm-l-tegile -n 10:102400:1024:1024 Using uid:65534, gid:65534. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP bm-l-tegile 4G 655 98 104319 14 51875 6 3347 100 122586 6 8482 104 Latency 15254us 390ms 504ms 4094us 529ms 2082us Version 1.96 ------Sequential Create------ --------Random Create-------- bm-l-tegile -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 10:102400:1024/1024 1894 14 +++++ +++ +++++ +++ 1895 15 +++++ +++ +++++ +++ Latency 69804us 303us 384us 66833us 288us 519us 1.96,1.96,bm-l-tegile,1,1330545313,4G,,655,98,104319,14,51875,6,3347,100,122586,6,8482,104,10,102400,1024,,1024,1894,14,+++++,+++,+++++,+++,1895,15,+++++,+++,+++++,+++,15254us,390ms,504ms,4094us,529ms,2082us,69804us,303us,384us,66833us,288us,519us bonnie++ -u nobody -d /killer/b -m bm-l-nexsan -n 10:102400:1024:1024 Using uid:65534, gid:65534. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP bm-l-nexsan 4G 647 97 56788 7 41929 5 3178 96 119375 7 3672 48 Latency 13268us 688ms 510ms 4096us 413ms 29050us Version 1.96 ------Sequential Create------ --------Random Create-------- bm-l-nexsan -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 10:102400:1024/1024 1446 12 +++++ +++ 15500 49 1618 12 +++++ +++ 18132 65 Latency 141ms 825us 307us 142ms 293us 1210us 1.96,1.96,bm-l-nexsan,1,1331344515,4G,,647,97,56788,7,41929,5,3178,96,119375,7,3672,48,10,102400,1024,,1024,1446,12,+++++,+++,15500,49,1618,12,+++++,+++,18132,65,13268us,688ms,510ms,4096us,413ms,29050us,141ms,825us,307us,142ms,293us,1210us _____ _ _ |_ _|_ _ ___| | __ | |___ _____ | |/ _` / __| |/ / | __\ \ /\ / / _ \ | | (_| \__ \ < | |_ \ V V / (_) | |_|\__,_|___/_|\_\ \__| \_/\_/ \___(_) FreeBSD system: bonnie++ -u nobody -d /killer/a -m bm-f-rand-os -n 10:102400:1024:1024 Using uid:65534, gid:65534. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP bm-f-rand-os 4G 574 97 55142 30 20852 13 1065 99 44428 19 2138 106 Latency 131ms 915ms 906ms 54857us 116ms 217ms Version 1.96 ------Sequential Create------ --------Random Create-------- bm-f-rand-os -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 10:102400:1024/1024 718 23 4153 92 +++++ +++ 1534 45 4131 94 +++++ +++ Latency 944ms 1796us 450us 317ms 5083us 291us 1.96,1.96,bm-f-rand-os,1,1330555383,4G,,574,97,55142,30,20852,13,1065,99,44428,19,2138,106,10,102400,1024,,1024,718,23,4153,92,+++++,+++,1534,45,4131,94,+++++,+++,131ms,915ms,906ms,54857us,116ms,217ms,944ms,1796us,450us,317ms,5083us,291us bonnie++ -u nobody -d /killer/b -m bm-f-rand-tintri -n 10:102400:1024:1024 Using uid:65534, gid:65534. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP bm-f-rand-tintri 4G 581 99 103919 56 29251 18 1060 99 46156 19 3628 169 Latency 22956us 32795us 911ms 34024us 221ms 185ms Version 1.96 ------Sequential Create------ --------Random Create-------- bm-f-rand-tintri -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 10:102400:1024/1024 1409 43 3809 91 +++++ +++ 1497 42 3902 94 +++++ +++ Latency 591ms 2914us 431us 263ms 6713us 339us 1.96,1.96,bm-f-rand-tintri,1,1330556419,4G,,581,99,103919,56,29251,18,1060,99,46156,19,3628,169,10,102400,1024,,1024,1409,43,3809,91,+++++,+++,1497,42,3902,94,+++++,+++,22956us,32795us,911ms,34024us,221ms,185ms,591ms,2914us,431us,263ms,6713us,339us bonnie++ -u nobody -d /killer/c -m bm-f-rand-tegile -n 10:102400:1024:1024 Using uid:65534, gid:65534. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP bm-f-rand-tegile 4G 584 99 99541 55 33396 21 1069 99 56418 23 5035 216 Latency 17102us 253ms 622ms 34533us 417ms 134ms Version 1.96 ------Sequential Create------ --------Random Create-------- bm-f-rand-tegile -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 10:102400:1024/1024 1578 49 4520 92 +++++ +++ 1406 40 4211 94 +++++ +++ Latency 97028us 767us 225us 173ms 7218us 487us 1.96,1.96,bm-f-rand-tegile,1,1330563207,4G,,584,99,99541,55,33396,21,1069,99,56418,23,5035,216,10,102400,1024,,1024,1578,49,4520,92,+++++,+++,1406,40,4211,94,+++++,+++,17102us,253ms,622ms,34533us,417ms,134ms,97028us,767us,225us,173ms,7218us,487us Ubuntu system: bonnie++ -u nobody -d /killer/a -m bm-l-rand-os -n 10:102400:1024:1024 Using uid:65534, gid:65534. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP bm-l-rand-os 4G 648 98 77843 10 45231 5 3315 99 126932 6 8637 111 Latency 14888us 906ms 746ms 3551us 527ms 2739us Version 1.96 ------Sequential Create------ --------Random Create-------- bm-l-rand-os -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 10:102400:1024/1024 1391 10 +++++ +++ +++++ +++ 1256 9 +++++ +++ +++++ +++ Latency 144ms 715us 602us 64715us 244us 704us 1.96,1.96,bm-l-rand-os,1,1330557814,4G,,648,98,77843,10,45231,5,3315,99,126932,6,8637,111,10,102400,1024,,1024,1391,10,+++++,+++,+++++,+++,1256,9,+++++,+++,+++++,+++,14888us,906ms,746ms,3551us,527ms,2739us,144ms,715us,602us,64715us,244us,704us bonnie++ -u nobody -d /killer/b -m bm-l-rand-tintri -n 10:102400:1024:1024 Using uid:65534, gid:65534. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP bm-l-rand-tintri 4G 654 99 108418 13 48163 5 3307 99 133198 7 6575 88 Latency 13252us 394ms 342ms 3381us 113ms 3551us Version 1.96 ------Sequential Create------ --------Random Create-------- bm-l-rand-tintri -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 10:102400:1024/1024 1826 14 +++++ +++ +++++ +++ 1766 13 +++++ +++ +++++ +++ Latency 69003us 346us 324us 740ms 289us 329us 1.96,1.96,bm-l-rand-tintri,1,1330557027,4G,,654,99,108418,13,48163,5,3307,99,133198,7,6575,88,10,102400,1024,,1024,1826,14,+++++,+++,+++++,+++,1766,13,+++++,+++,+++++,+++,13252us,394ms,342ms,3381us,113ms,3551us,69003us,346us,324us,740ms,289us,329us bonnie++ -u nobody -d /killer/c -m bm-l-rand-tegile -n 10:102400:1024:1024 Using uid:65534, gid:65534. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP bm-l-rand-tegile 4G 650 97 106755 14 53793 7 3335 99 137501 7 8881 109 Latency 14042us 418ms 524ms 3047us 419ms 2590us Version 1.96 ------Sequential Create------ --------Random Create-------- bm-l-rand-tegile -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 10:102400:1024/1024 1832 14 +++++ +++ +++++ +++ 1839 15 +++++ +++ +++++ +++ Latency 69074us 263us 353us 67132us 313us 339us 1.96,1.96,bm-l-rand-tegile,1,1330558621,4G,,650,97,106755,14,53793,7,3335,99,137501,7,8881,109,10,102400,1024,,1024,1832,14,+++++,+++,+++++,+++,1839,15,+++++,+++,+++++,+++,14042us,418ms,524ms,3047us,419ms,2590us,69074us,263us,353us,67132us,313us,339us _____ _ _ _ |_ _|_ _ ___| | __ | |_| |__ _ __ ___ ___ | |/ _` / __| |/ / | __| '_ \| '__/ _ \/ _ \ | | (_| \__ \ < | |_| | | | | | __/ __/_ |_|\__,_|___/_|\_\ \__|_| |_|_| \___|\___(_) FreeBSD system OpenSolaris 132 bm-f# postmark PostMark v1.51 : 8/14/01 pm>set size 10000 10000000 pm>set number 2000 pm>set transactions 2500 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 1045 seconds total 822 seconds of transactions (3 per second) Files: 3263 created (3 per second) Creation alone: 2000 files (8 per second) Mixed with transactions: 1263 files (1 per second) 1229 read (1 per second) 1271 appended (1 per second) 3263 deleted (3 per second) Deletion alone: 2026 files (2026 per second) Mixed with transactions: 1237 files (1 per second) Data: 6337.51 megabytes read (6.06 megabytes per second) 18248.26 megabytes written (17.46 megabytes per second) Tintri bm-f# postmark PostMark v1.51 : 8/14/01 pm>set size 10000 10000000 pm>set number 2000 pm>set transactions 2500 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 463 seconds total 339 seconds of transactions (7 per second) Files: 3263 created (7 per second) Creation alone: 2000 files (16 per second) Mixed with transactions: 1263 files (3 per second) 1229 read (3 per second) 1271 appended (3 per second) 3263 deleted (7 per second) Deletion alone: 2026 files (2026 per second) Mixed with transactions: 1237 files (3 per second) Data: 6337.51 megabytes read (13.69 megabytes per second) 18248.26 megabytes written (39.41 megabytes per second) Tegile bm-f# postmark PostMark v1.51 : 8/14/01 pm>set size 10000 10000000 pm>set number 2000 pm>set transactions 2500 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 474 seconds total 364 seconds of transactions (6 per second) Files: 3263 created (6 per second) Creation alone: 2000 files (18 per second) Mixed with transactions: 1263 files (3 per second) 1229 read (3 per second) 1271 appended (3 per second) 3263 deleted (6 per second) Deletion alone: 2026 files (2026 per second) Mixed with transactions: 1237 files (3 per second) Data: 6337.51 megabytes read (13.37 megabytes per second) 18248.26 megabytes written (38.50 megabytes per second) Nexsan bm-f# postmark PostMark v1.51 : 8/14/01 pm>set size 10000 10000000 pm>set number 2000 pm>set transactions 2500 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 627 seconds total 366 seconds of transactions (6 per second) Files: 3263 created (5 per second) Creation alone: 2000 files (7 per second) Mixed with transactions: 1263 files (3 per second) 1229 read (3 per second) 1271 appended (3 per second) 3263 deleted (5 per second) Deletion alone: 2026 files (1013 per second) Mixed with transactions: 1237 files (3 per second) Data: 6337.51 megabytes read (10.11 megabytes per second) 18248.26 megabytes written (29.10 megabytes per second) Ubuntu system OpenSolaris 132 root@bm-l:/killer/a# postmark PostMark v1.51 : 8/14/01 pm>set size 10000 10000000 pm>set number 2000 pm>set transactions 2500 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 652 seconds total 452 seconds of transactions (5 per second) Files: 3263 created (5 per second) Creation alone: 2000 files (10 per second) Mixed with transactions: 1263 files (2 per second) 1229 read (2 per second) 1271 appended (2 per second) 3263 deleted (5 per second) Deletion alone: 2026 files (2026 per second) Mixed with transactions: 1237 files (2 per second) Data: 6337.51 megabytes read (9.72 megabytes per second) 18248.26 megabytes written (27.99 megabytes per second) Tintri root@bm-l:/killer/b# postmark PostMark v1.51 : 8/14/01 pm>set size 10000 10000000 pm>set number 2000 pm>set transactions 2500 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 263 seconds total 167 seconds of transactions (14 per second) Files: 3263 created (12 per second) Creation alone: 2000 files (21 per second) Mixed with transactions: 1263 files (7 per second) 1229 read (7 per second) 1271 appended (7 per second) 3263 deleted (12 per second) Deletion alone: 2026 files (2026 per second) Mixed with transactions: 1237 files (7 per second) Data: 6337.51 megabytes read (24.10 megabytes per second) 18248.26 megabytes written (69.39 megabytes per second) Tegile root@bm-l:/killer/c# postmark PostMark v1.51 : 8/14/01 pm>set size 10000 10000000 pm>set number 2000 pm>set transactions 2500 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 279 seconds total 175 seconds of transactions (14 per second) Files: 3263 created (11 per second) Creation alone: 2000 files (19 per second) Mixed with transactions: 1263 files (7 per second) 1229 read (7 per second) 1271 appended (7 per second) 3263 deleted (11 per second) Deletion alone: 2026 files (2026 per second) Mixed with transactions: 1237 files (7 per second) Data: 6337.51 megabytes read (22.72 megabytes per second) 18248.26 megabytes written (65.41 megabytes per second) Nexsan root@bm-l:/killer/b# postmark PostMark v1.51 : 8/14/01 pm>set size 10000 10000000 pm>set number 2000 pm>set transactions 2500 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 275 seconds total 178 seconds of transactions (14 per second) Files: 3263 created (11 per second) Creation alone: 2000 files (20 per second) Mixed with transactions: 1263 files (7 per second) 1229 read (6 per second) 1271 appended (7 per second) 3263 deleted (11 per second) Deletion alone: 2026 files (2026 per second) Mixed with transactions: 1237 files (6 per second) Data: 6337.51 megabytes read (23.05 megabytes per second) 18248.26 megabytes written (66.36 megabytes per second)