[PATCH v2 0/3] landlock: Refactor layer masks

[Günther Noack]

Hello!

This patch set "transposes" the layer masks matrix, which was
previously modeled as a access-max-sized array of layer masks, and
changes it to be a layer-max-sized array of access masks instead.
(It is a pure refactoring, there are no user-visible changes.)

This unlocks a few code simplifications and in multiple places it
removes the need for loops and branches that deal with individual
bits.  Instead, the changed data structure now lends itself for more
bitwise operations.  The underlying hypothesis for me was that by
using more bitwise operations and fewer branches, we would get an
overall speedup even when the data structure size increases slightly
in some cases.

Tentative results with and without this patch set show that the
hypothesis likely holds true.  The benchmark I used exercises a "worst
case" scenario that attempts to be bottlenecked on the affected code:
constructs a large number of nested directories, with one "path
beneath" rule each and then tries to open the innermost directory many
times.  The benchmark is intentionally unrealistic to amplify the
amount of time used for the path walk logic and forces Landlock to
walk the full path (eventually failing the open syscall).  (I'll send
the benchmark program in a reply to this mail for full transparency.)

Measured with the benchmark program, the patch set results in a
speedup of about -10%.  The benchmark results are only tentative and
have been produced in Qemu:

With the patch, the benchmark runs in 6046 clocks (measured with
times(3)):

*** Benchmark ***
10000 dirs, 100000 iterations, with landlock
*** Benchmark concluded ***
System: 6046 clocks
User  : 1 clocks
Clocks per second: 1000000

Without the patch, we get 6713 clocks, which is 11% more

*** Benchmark ***
10000 dirs, 100000 iterations, with landlock
*** Benchmark concluded ***
System: 6713 clocks
User  : 0 clocks
Clocks per second: 1000000

The base revision used for benchmarking was commit 7a51784da76d
("tools/sched_ext: update scx_show_state.py for scx_aborting change")

In real-life scenarios, the speed improvement from this patch set will
be less pronounced than in the artificial benchmark, as people do not
usually stack directories that deeply and attach so many rules to
them, and the EACCES error should also be the exception rather than
the norm.

I am looking forward to your feedback.

P.S.: I am open to suggestions on what the "layer masks" variables
should be called, because the name "layer masks" might be less
appropriate after this change.  I have not fixed up the name
everywhere because fixing up the code took priority for now.

---
Changes since previous versions:

V2: (This patch set)

* Remove the refactoring around the deny_mask_t type,
  it is better to send that as a separate patch (mic review)
* Added the benchmark program to the selftests
* Fix unused variable report for "access_dom":
  https://lore.kernel.org/all/202601200900.wonk9M0m-lkp@intel.com/
* Use size_t and ARRAY_SIZE to loop over the layers (mic review)
* Documentation
  * Fixing up and adding back documentaiton (mic review)
  * Documented landlock_unmask_layers()
  * Fixed up kernel docs in a place where it was improperly updated
    (Spotted by Randy Dunlap
    https://lore.kernel.org/all/20260123025121.3713403-1-rdunlap@infradead.org/)
* Minor
  * Const, some newlines (mic review)

V1: (Initial version)

https://lore.kernel.org/all/20251230103917.10549-3-gnoack3000@gmail.com/

Günther Noack (3):
  selftests/landlock: Add filesystem access benchmark
  landlock: access_mask_subset() helper
  landlock: transpose the layer masks data structure

 security/landlock/access.h                  |  16 +-
 security/landlock/audit.c                   |  84 ++---
 security/landlock/audit.h                   |   3 +-
 security/landlock/domain.c                  |  45 +--
 security/landlock/domain.h                  |   4 +-
 security/landlock/fs.c                      | 354 +++++++++-----------
 security/landlock/net.c                     |  11 +-
 security/landlock/ruleset.c                 |  88 ++---
 security/landlock/ruleset.h                 |  21 +-
 tools/testing/selftests/landlock/.gitignore |   1 +
 tools/testing/selftests/landlock/Makefile   |   1 +
 tools/testing/selftests/landlock/fs_bench.c | 161 +++++++++
 12 files changed, 444 insertions(+), 345 deletions(-)
 create mode 100644 tools/testing/selftests/landlock/fs_bench.c

-- 
2.52.0