Home


Dennis Lang
lang.dennis @ comcast.net

Measure Various File Input/Output Methods
Updated: 31-Aug-2009

The following table shows the performance (MegaBytes/Second) of various Input/Output methods. The following methods are measured:

  • FILE (buffered i/o) fopen/fread/fwrite
  • Hnd (low level) open/read/write
  • MMap (memory mapped)
  • Stream (c++ ioStream) fstream
  • Windows createfile, readfile, writefile

    • Test run on HP 4 dual-core 2.8GHz, Windows XP32, with 750 GB Segate ST3750640AS drive, measured average transfer rate of 66MB/s using HDTune.

    The test is performed by first creating two 100 Megabyte files. One file is used to flush the OS file cache between each test. The flush is done by sequencially reading the entire file. Each test uses the specified i/o methods to read or write a specified block size buffer. The sequencial test runs across the entire 100MB file. The random test only touches 50% of the file. The random access test is shorten because I don't think the entire file needs to be randomly read to get the effective random access transfer rate. The time to run the test is measured as well as the total number of bytes procssed. The resuls are saved to a log file. The following graphs were created by using Excel and a Pivot table of the data.

    The following two graphs include both Sequencial and Random file access results in one graph. Please ignore the false transition between the two graphs. I plotted them together to make it easier to compare. NOTE: HDTune measured the average raw transfer rate of the disk drive at 66MB/sec, yet most of the test results exceed that rate by 2x. This difference reflects Windows XP's ability to buffer disk access to improve performance during Input (Read) tests. The Output (write) performance results are lower than the hardware transfer rate as expected because they don't get a cache benefit.

    The first graph shows the Input performance. Note Windows file input is poor for small block sizes with FILE being the best overall solution..

    The second graph shows the Output performance. I removed the Memory map results because I am unsure the code is actually updating the disk file. This graph is harder to pick a winner, but I would pick Stream as the best overall solution.

    The following table shows the raw results from the FileIO test program using the "-k 100mb -z" switches.

    * Click on column headers to sort (javascript provided by Gennadiy Shvets)
    Style Access Inp/Out MBytes BlockSiz Seconds MBytes/sec
    FILE Seq Out 100.00 64 1.872 53.431
    Hnd Seq Out 100.00 64 4.623 21.632
    MMap Seq Out 100.00 64 0.644 155.392
    Stream Seq Out 100.00 64 1.653 60.511
    Win Seq Out 100.00 64 4.431 22.568
    FILE Seq Out 100.00 256 1.726 57.945
    Hnd Seq Out 100.00 256 1.796 55.686
    MMap Seq Out 100.00 256 0.581 172.258
    Stream Seq Out 100.00 256 2.614 38.256
    Win Seq Out 100.00 256 1.851 54.025
    FILE Seq Out 100.00 1024 2.510 39.846
    Hnd Seq Out 100.00 1024 2.867 34.885
    MMap Seq Out 100.00 1024 0.577 173.178
    Stream Seq Out 100.00 1024 1.535 65.150
    Win Seq Out 100.00 1024 1.798 55.631
    FILE Seq Out 100.00 4096 1.899 52.656
    Hnd Seq Out 100.00 4096 3.068 32.595
    MMap Seq Out 100.00 4096 0.569 175.770
    Stream Seq Out 100.00 4096 1.885 53.061
    Win Seq Out 100.00 4096 2.817 35.501
    FILE Seq Out 100.00 16384 1.923 51.992
    Hnd Seq Out 100.00 16384 1.877 53.267
    MMap Seq Out 100.00 16384 0.625 160.113
    Stream Seq Out 100.00 16384 1.593 62.793
    Win Seq Out 100.00 16384 1.842 54.282
    FILE Seq Out 100.00 65536 3.426 29.191
    Hnd Seq Out 100.00 65536 1.879 53.206
    MMap Seq Out 100.00 65536 0.579 172.619
    Stream Seq Out 100.00 65536 1.609 62.132
    Win Seq Out 100.00 65536 2.203 45.391
    FILE Seq Out 100.00 262144 2.693 37.138
    Hnd Seq Out 100.00 262144 2.273 44.004
    MMap Seq Out 100.00 262144 0.581 171.993
    Stream Seq Out 100.00 262144 1.557 64.241
    Win Seq Out 100.00 262144 3.465 28.862
    FILE Seq Inp 100.00 64 0.819 122.096
    Hnd Seq Inp 100.00 64 3.841 26.037
    MMap Seq Inp 100.00 64 0.714 140.009
    Stream Seq Inp 100.00 64 0.907 110.293
    Win Seq Inp 100.00 64 3.802 26.302
    FILE Seq Inp 100.00 256 0.638 156.665
    Hnd Seq Inp 100.00 256 1.336 74.853
    MMap Seq Inp 100.00 256 0.624 160.313
    Stream Seq Inp 100.00 256 0.671 149.002
    Win Seq Inp 100.00 256 1.293 77.335
    FILE Seq Inp 100.00 1024 0.616 162.464
    Hnd Seq Inp 100.00 1024 0.729 137.108
    MMap Seq Inp 100.00 1024 0.611 163.629
    Stream Seq Inp 100.00 1024 0.626 159.653
    Win Seq Inp 100.00 1024 0.723 138.386
    FILE Seq Inp 100.00 4096 0.562 178.082
    Hnd Seq Inp 100.00 4096 0.559 178.799
    MMap Seq Inp 100.00 4096 0.623 160.532
    Stream Seq Inp 100.00 4096 0.603 165.883
    Win Seq Inp 100.00 4096 0.557 179.506
    FILE Seq Inp 100.00 16384 0.521 191.873
    Hnd Seq Inp 100.00 16384 0.521 191.818
    MMap Seq Inp 100.00 16384 0.649 154.188
    Stream Seq Inp 100.00 16384 0.606 164.986
    Win Seq Inp 100.00 16384 0.536 186.510
    FILE Seq Inp 100.00 65536 0.506 197.446
    Hnd Seq Inp 100.00 65536 0.506 197.495
    MMap Seq Inp 100.00 65536 0.631 158.593
    Stream Seq Inp 100.00 65536 0.569 175.627
    Win Seq Inp 100.00 65536 0.506 197.643
    FILE Seq Inp 100.00 262144 0.503 198.664
    Hnd Seq Inp 100.00 262144 0.502 199.140
    MMap Seq Inp 100.00 262144 0.668 149.766
    Stream Seq Inp 100.00 262144 0.587 170.406
    Win Seq Inp 100.00 262144 0.504 198.254
    FILE Rand Out 50.00 64 17.768 2.814
    Hnd Rand Out 50.00 64 19.315 2.589
    MMap Rand Out 50.00 64 0.477 104.829
    Stream Rand Out 50.00 64 19.818 2.523
    Win Rand Out 50.00 64 19.837 2.521
    FILE Rand Out 50.00 256 16.772 2.981
    Hnd Rand Out 50.00 256 15.543 3.217
    MMap Rand Out 50.00 256 0.420 119.011
    Stream Rand Out 50.00 256 13.417 3.727
    Win Rand Out 50.00 256 15.249 3.279
    FILE Rand Out 50.00 1024 2.666 18.753
    Hnd Rand Out 50.00 1024 2.043 24.468
    MMap Rand Out 50.00 1024 0.629 79.541
    Stream Rand Out 50.00 1024 2.438 20.511
    Win Rand Out 50.00 1024 2.427 20.600
    FILE Rand Out 50.00 4096 5.999 8.334
    Hnd Rand Out 50.00 4096 6.733 7.427
    MMap Rand Out 50.00 4096 0.374 133.566
    Stream Rand Out 50.00 4096 11.307 4.422
    Win Rand Out 50.00 4096 5.385 9.286
    FILE Rand Out 50.00 16384 5.325 9.391
    Hnd Rand Out 50.00 16384 3.992 12.525
    MMap Rand Out 50.00 16384 1.256 39.823
    Stream Rand Out 50.00 16384 3.870 12.919
    Win Rand Out 50.00 16384 3.180 15.724
    FILE Rand Out 50.00 65536 3.531 14.160
    Hnd Rand Out 50.00 65536 5.277 9.474
    MMap Rand Out 50.00 65536 0.795 62.926
    Stream Rand Out 50.00 65536 4.925 10.152
    Win Rand Out 50.00 65536 4.150 12.047
    FILE Rand Out 50.00 262144 1.487 33.628
    Hnd Rand Out 50.00 262144 2.723 18.361
    MMap Rand Out 50.00 262144 3.204 15.604
    Stream Rand Out 50.00 262144 2.441 20.485
    Win Rand Out 50.00 262144 1.339 37.337
    FILE Rand Inp 50.00 64 3.949 12.660
    Hnd Rand Inp 50.00 64 2.679 18.661
    MMap Rand Inp 50.00 64 12.111 4.129
    Stream Rand Inp 50.00 64 4.873 10.261
    Win Rand Inp 50.00 64 2.549 19.619
    FILE Rand Inp 50.00 256 1.329 37.636
    Hnd Rand Inp 50.00 256 0.880 56.797
    MMap Rand Inp 50.00 256 3.190 15.676
    Stream Rand Inp 50.00 256 1.576 31.725
    Win Rand Inp 50.00 256 0.874 57.239
    FILE Rand Inp 50.00 1024 0.506 98.865
    Hnd Rand Inp 50.00 1024 0.426 117.414
    MMap Rand Inp 50.00 1024 1.995 25.064
    Stream Rand Inp 50.00 1024 0.575 86.913
    Win Rand Inp 50.00 1024 0.448 111.692
    FILE Rand Inp 50.00 4096 0.340 146.952
    Hnd Rand Inp 50.00 4096 0.330 151.525
    MMap Rand Inp 50.00 4096 1.406 35.568
    Stream Rand Inp 50.00 4096 0.378 132.147
    Win Rand Inp 50.00 4096 0.338 148.092
    FILE Rand Inp 50.00 16384 0.320 156.011
    Hnd Rand Inp 50.00 16384 0.319 156.981
    MMap Rand Inp 50.00 16384 1.422 35.151
    Stream Rand Inp 50.00 16384 0.361 138.467
    Win Rand Inp 50.00 16384 0.327 152.829
    FILE Rand Inp 50.00 65536 0.313 159.956
    Hnd Rand Inp 50.00 65536 0.320 156.319
    MMap Rand Inp 50.00 65536 1.962 25.488
    Stream Rand Inp 50.00 65536 0.355 140.955
    Win Rand Inp 50.00 65536 0.312 160.374
    FILE Rand Inp 50.00 262144 0.310 161.176
    Hnd Rand Inp 50.00 262144 0.310 161.356
    MMap Rand Inp 50.00 262144 2.536 19.718
    Stream Rand Inp 50.00 262144 0.352 142.100
    Win Rand Inp 50.00 262144 0.310 161.452


  • Dowload source code: Dennis Lang's FileIO.zip (Linux and Windows)

    • Top
    Home