Hi,

I open the file, then call mmap() on the whole file and get pointer,
then I work with this pointer.  I expect that page should be only once
touched to get it into the memory (disk cache?), but this doesn't work!

I wrote the test (attached) and ran it for the 1G file generated from
/dev/random, the result is the following:

Prepare file:
# swapoff -a
# newfs /dev/ada0b
# mount /dev/ada0b /mnt
# dd if=/dev/random of=/mnt/random-1024 bs=1m count=1024

Purge cache:
# umount /mnt
# mount /dev/ada0b /mnt

Run test:
$ ./mmap /mnt/random-1024 30
mmap:  1 pass took:   7.431046 (none: 262112; res:     32; super:
0; other:      0)
mmap:  2 pass took:   7.356670 (none: 261648; res:    496; super:
0; other:      0)
mmap:  3 pass took:   7.307094 (none: 260521; res:   1623; super:
0; other:      0)
mmap:  4 pass took:   7.350239 (none: 258904; res:   3240; super:
0; other:      0)
mmap:  5 pass took:   7.392480 (none: 257286; res:   4858; super:
0; other:      0)
mmap:  6 pass took:   7.292069 (none: 255584; res:   6560; super:
0; other:      0)
mmap:  7 pass took:   7.048980 (none: 251142; res:  11002; super:
0; other:      0)
mmap:  8 pass took:   6.899387 (none: 247584; res:  14560; super:
0; other:      0)
mmap:  9 pass took:   7.190579 (none: 242992; res:  19152; super:
0; other:      0)
mmap: 10 pass took:   6.915482 (none: 239308; res:  22836; super:
0; other:      0)
mmap: 11 pass took:   6.565909 (none: 232835; res:  29309; super:
0; other:      0)
mmap: 12 pass took:   6.423945 (none: 226160; res:  35984; super:
0; other:      0)
mmap: 13 pass took:   6.315385 (none: 208555; res:  53589; super:
0; other:      0)
mmap: 14 pass took:   6.760780 (none: 192805; res:  69339; super:
0; other:      0)
mmap: 15 pass took:   5.721513 (none: 174497; res:  87647; super:
0; other:      0)
mmap: 16 pass took:   5.004424 (none: 155938; res: 106206; super:
0; other:      0)
mmap: 17 pass took:   4.224926 (none: 135639; res: 126505; super:
0; other:      0)
mmap: 18 pass took:   3.749608 (none: 117952; res: 144192; super:
0; other:      0)
mmap: 19 pass took:   3.398084 (none:  99066; res: 163078; super:
0; other:      0)
mmap: 20 pass took:   3.029557 (none:  74994; res: 187150; super:
0; other:      0)
mmap: 21 pass took:   2.379430 (none:  55231; res: 206913; super:
0; other:      0)
mmap: 22 pass took:   2.046521 (none:  40786; res: 221358; super:
0; other:      0)
mmap: 23 pass took:   1.152797 (none:  30311; res: 231833; super:
0; other:      0)
mmap: 24 pass took:   0.972617 (none:  16196; res: 245948; super:
0; other:      0)
mmap: 25 pass took:   0.577515 (none:   8286; res: 253858; super:
0; other:      0)
mmap: 26 pass took:   0.380738 (none:   3712; res: 258432; super:
0; other:      0)
mmap: 27 pass took:   0.253583 (none:   1193; res: 260951; super:
0; other:      0)
mmap: 28 pass took:   0.157508 (none:      0; res: 262144; super:
0; other:      0)
mmap: 29 pass took:   0.156169 (none:      0; res: 262144; super:
0; other:      0)
mmap: 30 pass took:   0.156550 (none:      0; res: 262144; super:
0; other:      0)

If I ran this:
$ cat /mnt/random-1024 > /dev/null
before test, when result is the following:

$ ./mmap /mnt/random-1024 5
mmap:  1 pass took:   0.337657 (none:      0; res: 262144; super:
0; other:      0)
mmap:  2 pass took:   0.186137 (none:      0; res: 262144; super:
0; other:      0)
mmap:  3 pass took:   0.186132 (none:      0; res: 262144; super:
0; other:      0)
mmap:  4 pass took:   0.186535 (none:      0; res: 262144; super:
0; other:      0)
mmap:  5 pass took:   0.190353 (none:      0; res: 262144; super:
0; other:      0)

This is what I expect.  But why this doesn't work without reading file
manually?

I've also never seen super pages, how to make them work?

I've been playing with madvise and posix_fadvise but no luck.  BTW,
posix_fadvise(POSIX_FADV_WILLNEED) does nothing as the commentary says,
shouldn't this be documented in the manual page?

All tests were run under 9.0-STABLE (r233744).

--
Andrey Zonov