[RFC PATCH -mm 0/4] mm, security, bpf: Fine-grained control over memory policy adjustments with lsm bpf

[Casey Schaufler]

On 11/11/2023 11:34 PM, Yafang Shao wrote:
> Background
> ==========
>
> In our containerized environment, we've identified unexpected OOM events
> where the OOM-killer terminates tasks despite having ample free memory.
> This anomaly is traced back to tasks within a container using mbind(2) to
> bind memory to a specific NUMA node. When the allocated memory on this node
> is exhausted, the OOM-killer, prioritizing tasks based on oom_score,
> indiscriminately kills tasks. This becomes more critical with guaranteed
> tasks (oom_score_adj: -998) aggravating the issue.

Is there some reason why you can't fix the callers of mbind(2)?
This looks like an user space configuration error rather than a
system security issue.

>
> The selected victim might not have allocated memory on the same NUMA node,
> rendering the killing ineffective. This patch aims to address this by
> disabling MPOL_BIND in container environments.
>
> In the container environment, our aim is to consolidate memory resource
> control under the management of kubelet. If users express a preference for
> binding their memory to a specific NUMA node, we encourage the adoption of
> a standardized approach. Specifically, we recommend configuring this memory
> policy through kubelet using cpuset.mems in the cpuset controller, rather
> than individual users setting it autonomously. This centralized approach
> ensures that NUMA nodes are globally managed through kubelet, promoting
> consistency and facilitating streamlined administration of memory resources
> across the entire containerized environment.

Changing system behavior for a single use case doesn't seem prudent.
You're introducing a bunch of kernel code to avoid fixing a broken
user space configuration.

>
> Proposed Solutions
> =================
>
> - Introduce Capability to Disable MPOL_BIND
>   Currently, any task can perform MPOL_BIND without specific capabilities.
>   Enforcing CAP_SYS_RESOURCE or CAP_SYS_NICE could be an option, but this
>   may have unintended consequences. Capabilities, being broad, might grant
>   unnecessary privileges. We should explore alternatives to prevent
>   unexpected side effects.
>
> - Use LSM BPF to Disable MPOL_BIND
>   Introduce LSM hooks for syscalls such as mbind(2), set_mempolicy(2), and
>   set_mempolicy_home_node(2) to disable MPOL_BIND. This approach is more
>   flexibility and allows for fine-grained control without unintended
>   consequences. A sample LSM BPF program is included, demonstrating
>   practical implementation in a production environment.
>
> Future Considerations
> =====================
>
> In addition, there's room for enhancement in the OOM-killer for cases
> involving CONSTRAINT_MEMORY_POLICY. It would be more beneficial to
> prioritize selecting a victim that has allocated memory on the same NUMA
> node. My exploration on the lore led me to a proposal[0] related to this
> matter, although consensus seems elusive at this point. Nevertheless,
> delving into this specific topic is beyond the scope of the current
> patchset.
>
> [0]. https://lore.kernel.org/lkml/20220512044634.63586-1-ligang.bdlg@bytedance.com/ 
>
> Yafang Shao (4):
>   mm, security: Add lsm hook for mbind(2)
>   mm, security: Add lsm hook for set_mempolicy(2)
>   mm, security: Add lsm hook for set_mempolicy_home_node(2)
>   selftests/bpf: Add selftests for mbind(2) with lsm prog
>
>  include/linux/lsm_hook_defs.h                      |  8 +++
>  include/linux/security.h                           | 26 +++++++
>  mm/mempolicy.c                                     | 13 ++++
>  security/security.c                                | 19 ++++++
>  tools/testing/selftests/bpf/prog_tests/mempolicy.c | 79 ++++++++++++++++++++++
>  tools/testing/selftests/bpf/progs/test_mempolicy.c | 29 ++++++++
>  6 files changed, 174 insertions(+)
>  create mode 100644 tools/testing/selftests/bpf/prog_tests/mempolicy.c
>  create mode 100644 tools/testing/selftests/bpf/progs/test_mempolicy.c
>