[RFC PATCH 2/2] landlock: transpose the layer masks data structure

[Justin Suess]

On 1/20/26 19:26, Tingmao Wang wrote:
> On 12/30/25 10:39, Günther Noack wrote:
>> The layer masks data structure tracks the requested but unfulfilled
>> access rights during an operations security check.  It stores one bit
>> for each combination of access right and layer index.  If the bit is
>> set, that access right is not granted (yet) in the given layer and we
>> have to traverse the path further upwards to grant it.
>>
>> Previously, the layer masks were stored as arrays mapping from access
>> right indices to layer_mask_t.  The layer_mask_t value then indicates
>> all layers in which the given access right is still (tentatively)
>> denied.
>>
>> This patch introduces struct layer_access_masks instead: This struct
>> contains an array with the access_mask_t of each (tentatively) denied
>> access right in that layer.
>>
>> The hypothesis of this patch is that this simplifies the code enough
>> so that the resulting code will run faster:
>>
>> * We can use bitwise operations in multiple places where we previously
>>   looped over bits individually with macros.  (Should require less
>>   branch speculation)
>>
>> * Code is ~160 lines smaller.
>>
>> Other noteworthy changes:
>>
>> * Clarify deny_mask_t and the code assembling it.
>>   * Document what that value looks like
>>   * Make writing and reading functions specific to file system rules.
>>     (It only worked for FS rules before as well, but going all the way
>>     simplifies the code logic more.)
> In the original commit message that added this type [1] there was this
> statement:
>
>> Implementing deny_masks_t with a bitfield instead of a struct enables a
>> generic implementation to store and extract layer levels.
> At some point when looking at this I was wondering why this wasn't a
> struct with 2 u8:4 fields, but rather, a u8 with bit manipulation code.
> While it is possible that I might have just misunderstood it, reading the
> above statement my take-away was that a struct would have forced us to
> address the indices with specific names, e.g. it would need to be defined
> like
>
> struct deny_masks_t {
>     u8 ioctl:4;
>     u8 truncate:4;
> }
>
> And it would thus not be possible to manipulate the indices in a generic
> way (e.g. the way it was implemented before, given
> all_existing_optional_access and access_bit, read and write the right
> bits).
>
> However, since we're now removing that generic-ability, should we consider
> turning it into a struct?  (If later on we have different access types
> that also have optional accesses, we could use a union of structs)
>
>
> btw, since this causes conflicts with the quiet flag series and Mickaël
> has indicated that this should be merged first, I will probably have to
> make my series based on top of this.  Will watch this series to see if
> there are more changes.
Likewise for my NO_INHERIT series, which will need some rebase work as
well. (my series is built on the quiet flag series, to reuse the similar "bubble up"
flag collection logic).

I'll keep an eye on your tree Tingmao and start rebasing my NO_INHERIT
on your patches if you put your work there. (Otherwise I'll do it when you
send it on the mailing list)