[PATCH RESEND v3 bpf-next 00/14] BPF token

[Andrii Nakryiko]

On Thu, Jun 29, 2023 at 4:15 PM Toke Høiland-Jørgensen <toke at redhat.com> wrote:
>
> Andrii Nakryiko <andrii at kernel.org> writes:
>
> > This patch set introduces new BPF object, BPF token, which allows to delegate
> > a subset of BPF functionality from privileged system-wide daemon (e.g.,
> > systemd or any other container manager) to a *trusted* unprivileged
> > application. Trust is the key here. This functionality is not about allowing
> > unconditional unprivileged BPF usage. Establishing trust, though, is
> > completely up to the discretion of respective privileged application that
> > would create a BPF token, as different production setups can and do achieve it
> > through a combination of different means (signing, LSM, code reviews, etc),
> > and it's undesirable and infeasible for kernel to enforce any particular way
> > of validating trustworthiness of particular process.
> >
> > The main motivation for BPF token is a desire to enable containerized
> > BPF applications to be used together with user namespaces. This is currently
> > impossible, as CAP_BPF, required for BPF subsystem usage, cannot be namespaced
> > or sandboxed, as a general rule. E.g., tracing BPF programs, thanks to BPF
> > helpers like bpf_probe_read_kernel() and bpf_probe_read_user() can safely read
> > arbitrary memory, and it's impossible to ensure that they only read memory of
> > processes belonging to any given namespace. This means that it's impossible to
> > have namespace-aware CAP_BPF capability, and as such another mechanism to
> > allow safe usage of BPF functionality is necessary. BPF token and delegation
> > of it to a trusted unprivileged applications is such mechanism. Kernel makes
> > no assumption about what "trusted" constitutes in any particular case, and
> > it's up to specific privileged applications and their surrounding
> > infrastructure to decide that. What kernel provides is a set of APIs to create
> > and tune BPF token, and pass it around to privileged BPF commands that are
> > creating new BPF objects like BPF programs, BPF maps, etc.
>
> So a colleague pointed out today that the Seccomp Notify functionality
> would be a way to achieve your stated goal of allowing unprivileged
> containers to (selectively) perform bpf() syscall operations. Christian
> Brauner has a pretty nice writeup of the functionality here:
> https://people.kernel.org/brauner/the-seccomp-notifier-new-frontiers-in-unprivileged-container-development
>
> In fact he even mentions allowing unprivileged access to bpf() as a
> possible use case (in the second-to-last paragraph).
>
> AFAICT this would enable your use case without adding any new kernel
> functionality or changing the BPF-using applications, while allowing the
> privileged userspace daemon to make case-by-case decisions on each
> operation instead of granting blanket capabilities (which is my main
> objection to the token proposal, as we discussed on the last iteration
> of the series).

It's not "blanket" capabilities. You control types or maps and
programs that could be created. And again, CAP_SYS_ADMIN guarded.
Please, don't give CAP_SYS_ADMIN/root permissions to applications you
can't be sure won't do something stupid and blame kernel API for it.

After all, the root process can setuid() any file and make it run with
elevated permissions, right? Doesn't get more "blanket" than that.

>
> So I'm curious whether you considered this as an alternative to
> BPF_TOKEN? And if so, what your reason was for rejecting it?
>

Yes, I'm aware, Christian has a follow up short blog post specifically
for using this for proxying BPF from privileged process ([0]).

So, in short, I think it's not a good generic solution. It's very
fragile and high-maintenance. It's still proxying BPF UAPI (except
application does preserve illusion of using BPF syscall, yes, that
part is good) with all the implications: needing to replicate all of
UAPI (fetching all those FDs from another process, following all the
pointers from another process' memory, etc), and also writing back all
the correct things (into another process' memory): log content,
log_true_size (out param), any other output parameters. What do we do
when an application uses a newer version of bpf_attr that is supported
by proxy? And honestly, I'm like 99% sure there are lots of less
obvious issues one runs into when starting implementing something like
this.

This sounds like a hack and nightmare to implement and support.
Perhaps that indirectly is supported by the fact that even Christian
half-jokingly calls this a crazy approach. That code basically is
unchanged for the last three years, with only one fix from Christian
one year after initial introduction ([1]) to fix a quirky issue
related to the limitation of pidfd working only for thread group
leaders. It also still supports only BPF_PROG_TYPE_CGROUP_DEVICE
program loading, it doesn't support a bunch of newer BPF_PROG_LOAD
fields and functionality, etc, etc.

So as a technical curiosity it's pretty cool and perhaps is the right
tool for the job for very narrow specific use cases. But as a
realistic generic approach that could be used by industry at large for
safe BPF usage from namespaced containers -- not so much.


  [0] https://brauner.io/2020/08/07/seccomp-notify-intercepting-the-bpf-syscall.html
  [1] https://github.com/lxc/lxd/commit/566d0a3b3cbe288787886c2f3bf5b250ceb930b0


> -Toke
>