I read the how-should-i-benchmark-small-functions section in FAQ. I change the example and test it. But got confused.

I add 2 bench cases:

• mix-inline, which does multiple operations;
• mix-func, which dose the same work, but by a function mix().

I thought these 2 cases should cost same time. But the latter is much faster, even same with the single addition case. (314.17 ps vs 319.95 ps)

fn mix(i: &i64) -> i64 {
    (i + 100) / (i - 3) * (i % 7)
}

fn bench_int(c: &mut Criterion) {
    let mut group = c.benchmark_group("small");

    group.bench_with_input("single", &10, |b, i| b.iter(|| i + 10));  // 319.95 ps
    group.bench_with_input("mix-inline", &10, |b, i| {
        b.iter(|| (i + 100) / (i - 3) * (i % 7)) // same with function mix()  // 886.59 ps
    });
    group.bench_with_input("mix-func", &10, |b, i| b.iter(|| mix(i)));  // 314.17 ps ??

    group.finish();
}

Raw output at MacOS/M1:

Benchmarking small/single: Collecting 100 samples in estimated 5.0000 s (16B iteratio
small/single            time:   [314.23 ps 319.95 ps 331.63 ps]
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe
Benchmarking small/mix-inline: Collecting 100 samples in estimated 5.0000 s (5.7B ite
small/mix-inline        time:   [881.21 ps 886.59 ps 894.05 ps]
Found 8 outliers among 100 measurements (8.00%)
  4 (4.00%) high mild
  4 (4.00%) high severe
Benchmarking small/mix-func: Collecting 100 samples in estimated 5.0000 s (16B iterat
small/mix-func          time:   [312.72 ps 314.17 ps 316.79 ps]
Found 14 outliers among 100 measurements (14.00%)
  3 (3.00%) high mild
  11 (11.00%) high severe

Raw output at Linux/x86_64:

small/single            time:   [632.20 ps 632.49 ps 632.81 ps]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high severe
small/mix-inline        time:   [3.8265 ns 3.8360 ns 3.8477 ns]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
small/mix-func          time:   [317.10 ps 317.54 ps 317.99 ps]
Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe

Do you know why the mix-func case so fast?

Thanks in advance

===
I post an issue at Criterion.

3 posts - 2 participants

Read full topic