I forgot to attach my test program.

On 04.04.2012 13:36, Andrey Zonov wrote:
> On 04.04.2012 11:17, Konstantin Belousov wrote:
>>
>> Calling madvise(MADV_RANDOM) fixes the issue, because the code to
>> deactivate/cache the pages is turned off. On the other hand, it also
>> turns of read-ahead for faulting, and the first loop becomes eternally
>> long.
>
> Now it takes 5 times longer. Anyway, thanks for explanation.
>
>>
>> Doing MADV_WILLNEED does not fix the problem indeed, since willneed
>> reactivates the pages of the object at the time of call. To use
>> MADV_WILLNEED, you would need to call it between faults/memcpy.
>>
>
> I played with it, but no luck so far.
>
>>>
>>> I've also never seen super pages, how to make them work?
>> They just work, at least for me. Look at the output of procstat -v
>> after enough loops finished to not cause disk activity.
>>
>
> The problem was in my test program. I fixed it, now I see super pages
> but I'm still not satisfied. There are several tests below:
>
> 1. With madvise(MADV_RANDOM) I see almost all super pages:
> $ ./mmap /mnt/random-1024 5
> mmap: 1 pass took: 26.438535 (none: 0; res: 262144; super: 511; other: 0)
> mmap: 2 pass took: 0.187311 (none: 0; res: 262144; super: 511; other: 0)
> mmap: 3 pass took: 0.184953 (none: 0; res: 262144; super: 511; other: 0)
> mmap: 4 pass took: 0.186007 (none: 0; res: 262144; super: 511; other: 0)
> mmap: 5 pass took: 0.185790 (none: 0; res: 262144; super: 511; other: 0)
>
> Should it be 512?
>
> 2. Without madvise(MADV_RANDOM):
> $ ./mmap /mnt/random-1024 50
> mmap: 1 pass took: 7.629745 (none: 262112; res: 32; super: 0; other: 0)
> mmap: 2 pass took: 7.301720 (none: 261202; res: 942; super: 0; other: 0)
> mmap: 3 pass took: 7.261416 (none: 260226; res: 1918; super: 1; other: 0)
> [skip]
> mmap: 49 pass took: 0.155368 (none: 0; res: 262144; super: 323; other: 0)
> mmap: 50 pass took: 0.155438 (none: 0; res: 262144; super: 323; other: 0)
>
> Only 323 pages.
>
> 3. If I just re-run test I don't see super pages with any size of "block".
>
> $ ./mmap /mnt/random-1024 5 $((1<<30))
> mmap: 1 pass took: 1.013939 (none: 0; res: 262144; super: 0; other: 0)
> mmap: 2 pass took: 0.267082 (none: 0; res: 262144; super: 0; other: 0)
> mmap: 3 pass took: 0.270711 (none: 0; res: 262144; super: 0; other: 0)
> mmap: 4 pass took: 0.268940 (none: 0; res: 262144; super: 0; other: 0)
> mmap: 5 pass took: 0.269634 (none: 0; res: 262144; super: 0; other: 0)
>
> 4. If I activate madvise(MADV_WILLNEDD) in the copy loop and re-run test
> then I see super pages only if I use "block" greater than 2Mb.
>
> $ ./mmap /mnt/random-1024 1 $((1<<21))
> mmap: 1 pass took: 0.299722 (none: 0; res: 262144; super: 0; other: 0)
> $ ./mmap /mnt/random-1024 1 $((1<<22))
> mmap: 1 pass took: 0.271828 (none: 0; res: 262144; super: 170; other: 0)
> $ ./mmap /mnt/random-1024 1 $((1<<23))
> mmap: 1 pass took: 0.333188 (none: 0; res: 262144; super: 258; other: 0)
> $ ./mmap /mnt/random-1024 1 $((1<<24))
> mmap: 1 pass took: 0.339250 (none: 0; res: 262144; super: 303; other: 0)
> $ ./mmap /mnt/random-1024 1 $((1<<25))
> mmap: 1 pass took: 0.418812 (none: 0; res: 262144; super: 324; other: 0)
> $ ./mmap /mnt/random-1024 1 $((1<<26))
> mmap: 1 pass took: 0.360892 (none: 0; res: 262144; super: 335; other: 0)
> $ ./mmap /mnt/random-1024 1 $((1<<27))
> mmap: 1 pass took: 0.401122 (none: 0; res: 262144; super: 342; other: 0)
> $ ./mmap /mnt/random-1024 1 $((1<<28))
> mmap: 1 pass took: 0.478764 (none: 0; res: 262144; super: 345; other: 0)
> $ ./mmap /mnt/random-1024 1 $((1<<29))
> mmap: 1 pass took: 0.607266 (none: 0; res: 262144; super: 346; other: 0)
> $ ./mmap /mnt/random-1024 1 $((1<<30))
> mmap: 1 pass took: 0.901269 (none: 0; res: 262144; super: 347; other: 0)
>
> 5. If I activate madvise(MADV_WILLNEED) immediately after mmap() then I
> see some number of super pages (the number from test #2).
>
> $ ./mmap /mnt/random-1024 5
> mmap: 1 pass took: 0.178666 (none: 0; res: 262144; super: 323; other: 0)
> mmap: 2 pass took: 0.158889 (none: 0; res: 262144; super: 323; other: 0)
> mmap: 3 pass took: 0.157229 (none: 0; res: 262144; super: 323; other: 0)
> mmap: 4 pass took: 0.156895 (none: 0; res: 262144; super: 323; other: 0)
> mmap: 5 pass took: 0.162938 (none: 0; res: 262144; super: 323; other: 0)
>
> 6. If I read file manually before test then I don't see super pages with
> any size of "block" and madvise(MADV_WILLNEED) doesn't help.
>
> $ ./mmap /mnt/random-1024 5 $((1<<30))
> mmap: 1 pass took: 0.996767 (none: 0; res: 262144; super: 0; other: 0)
> mmap: 2 pass took: 0.311129 (none: 0; res: 262144; super: 0; other: 0)
> mmap: 3 pass took: 0.317430 (none: 0; res: 262144; super: 0; other: 0)
> mmap: 4 pass took: 0.314437 (none: 0; res: 262144; super: 0; other: 0)
> mmap: 5 pass took: 0.310757 (none: 0; res: 262144; super: 0; other: 0)
>
>

--
Andrey Zonov