Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 23 additions & 1 deletion docs/learning-more/multi-machine-tracing.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,25 @@ UNIX path to a TCP listener that remote machines can reach.
`--enable-relay-endpoint` makes that socket accept `traced_relay`
connections in addition to ordinary local producers.

For deployments that need to keep a local AF_UNIX producer socket
listening alongside the TCP relay socket — so host-side unprivileged
producers and remote `traced_relay` clients use separate sockets —
list both in `PERFETTO_PRODUCER_SOCK_NAME` and use the narrower
`--enable-relay-endpoint-on` form, which names the socket that should
carry `RelayPort`:

```bash
PERFETTO_PRODUCER_SOCK_NAME=/tmp/perfetto-producer,0.0.0.0:20001 \
tracebox traced --enable-relay-endpoint-on=0.0.0.0:20001
```

In this form the local UNIX socket continues to serve unprivileged
producers and never accepts relay calls, which keeps the `RelayPort`
service unreachable from local apps. The flag selects from the producer
sockets already listed in `PERFETTO_PRODUCER_SOCK_NAME` — it does not
introduce a new endpoint, so the named socket must appear in the env
var.

Leave this process running.

### Step 2: Start `traced_probes` on the host
Expand All @@ -69,7 +88,10 @@ PERFETTO_PRODUCER_SOCK_NAME=127.0.0.1:20001 \
The same env var that rebound `traced`'s listener also tells local
producers where to connect — without it, `traced_probes` would still try
the default UNIX socket and fail. `sudo -E` preserves the env var across
the privilege escalation needed for ftrace.
the privilege escalation needed for ftrace. (If you used the
`--enable-relay-endpoint-on` form in Step 1 instead, omit the env var:
the default UNIX producer socket is still up, and `traced_probes` will
connect to it without any extra configuration.)

### Step 3: Start `traced_relay` on the guest

Expand Down
21 changes: 18 additions & 3 deletions docs/reference/traced.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,12 +83,27 @@ Entities interact with `traced` primarily through two channels:
<prod_group>:<prod_mode>:<cons_group>:<cons_mode>`: Sets the group ownership
and permission mode for the producer and consumer sockets. This is important
for controlling which users and processes can connect to `traced`.
* `--enable-relay-endpoint`: Enables an endpoint for
[multi-machine tracing](/docs/deployment/multi-machine-architecture.md) via
`traced_relay`. See
* `--enable-relay-endpoint`: Exposes the `RelayPort` service used by
[multi-machine tracing](/docs/deployment/multi-machine-architecture.md)
on every producer socket named by `PERFETTO_PRODUCER_SOCK_NAME` (or the
default producer socket). This is the standard switch for multi-machine
setups; see
[Multi-machine recording](/docs/learning-more/multi-machine-tracing.md)
for the host-side setup.

> **Security note:** when the producer socket list mixes a local
> AF_UNIX socket with a remote-capable one, this flag exposes
> `RelayPort` on the local socket too. Deployments that combine
> local and remote producer sockets should use
> `--enable-relay-endpoint-on` below to keep the relay port off the
> local socket.
* `--enable-relay-endpoint-on <socket>`: Like `--enable-relay-endpoint`,
but exposes the `RelayPort` service only on the named producer socket.
`<socket>` must be one of the entries in `PERFETTO_PRODUCER_SOCK_NAME`
(or the default producer socket): the flag selects which existing
producer sockets get `RelayPort`; it does not introduce new endpoints.
May be repeated.

## Built-in Producer

On Android, `traced` also includes a built-in producer with several key
Expand Down
3 changes: 0 additions & 3 deletions include/perfetto/ext/tracing/core/tracing_service.h
Original file line number Diff line number Diff line change
Expand Up @@ -313,9 +313,6 @@ struct PERFETTO_EXPORT_COMPONENT TracingServiceInitOpts {
using CompressorFn = void (*)(std::vector<TracePacket>*);
CompressorFn compressor_fn = nullptr;

// Whether the relay endpoint is enabled on producer transport(s).
bool enable_relay_endpoint = false;

// An (optional) list of proto extension descriptors to dump into each trace
// recorded. This is to support injecting protos that are known by the
// embedder (e.g. the Android vendor image) but not by the upstream perfetto.
Expand Down
5 changes: 5 additions & 0 deletions include/perfetto/ext/tracing/ipc/service_ipc_host.h
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,11 @@ struct ListenEndpoint {
std::string sock_name;
base::ScopedSocketHandle sock_handle;
std::unique_ptr<ipc::Host> ipc_host;

// If true, the RelayPort IPC service is also exposed on this endpoint.
// Should only be set on endpoints reserved for cross-machine peers (e.g.
// vsock, TCP), not on local sockets shared with unprivileged producers.
bool expose_relay_endpoint = false;
};

// Creates an instance of the service (business logic + UNIX socket transport).
Expand Down
74 changes: 58 additions & 16 deletions src/traced/service/service.cc
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@

#include <stdio.h>
#include <algorithm>
#include <set>

#include "perfetto/base/status.h"
#include "perfetto/ext/base/android_utils.h"
Expand Down Expand Up @@ -57,9 +58,19 @@ Options and arguments
<prod_mode> is the mode bits (e.g. 0660) for chmod the produce socket,
<cons_group> is the group name for chgrp the consumer socket, and
<cons_mode> is the mode bits (e.g. 0660) for chmod the consumer socket.
--enable-relay-endpoint : enables the relay endpoint on producer socket(s)
for traced_relay to communicate with traced in a multiple-machine
tracing session.
--enable-relay-endpoint : exposes the RelayPort service (used by
traced_relay in multi-machine tracing) on every producer socket named
by PERFETTO_PRODUCER_SOCK_NAME / the default producer socket. This
is the standard switch for multi-machine setups.
--enable-relay-endpoint-on <socket> : narrower variant of the above,
intended for security-conscious deployments. Exposes the RelayPort
service only on the named producer socket. <socket> must match one
of the entries in PERFETTO_PRODUCER_SOCK_NAME (or the default
producer socket); the flag selects which producer sockets get
RelayPort, it does NOT introduce new endpoints. May be repeated.
Useful when the producer socket list mixes local AF_UNIX sockets
with remote-capable ones and the relay port must not be reachable
on the local socket.

Example:
%s --set-socket-permissions traced-producer:0660:traced-consumer:0660
Expand All @@ -76,11 +87,13 @@ int PERFETTO_EXPORT_ENTRYPOINT ServiceMain(int argc, char** argv) {
OPT_VERSION = 1000,
OPT_SET_SOCKET_PERMISSIONS = 1001,
OPT_BACKGROUND,
OPT_ENABLE_RELAY_ENDPOINT
OPT_ENABLE_RELAY_ENDPOINT,
OPT_ENABLE_RELAY_ENDPOINT_ON
};

bool background = false;
bool enable_relay_endpoint = false;
std::set<std::string> relay_endpoint_sockets;

static const option long_options[] = {
{"background", no_argument, nullptr, OPT_BACKGROUND},
Expand All @@ -89,6 +102,8 @@ int PERFETTO_EXPORT_ENTRYPOINT ServiceMain(int argc, char** argv) {
OPT_SET_SOCKET_PERMISSIONS},
{"enable-relay-endpoint", no_argument, nullptr,
OPT_ENABLE_RELAY_ENDPOINT},
{"enable-relay-endpoint-on", required_argument, nullptr,
OPT_ENABLE_RELAY_ENDPOINT_ON},
{nullptr, 0, nullptr, 0}};

std::string producer_socket_group, consumer_socket_group,
Expand Down Expand Up @@ -121,6 +136,9 @@ int PERFETTO_EXPORT_ENTRYPOINT ServiceMain(int argc, char** argv) {
case OPT_ENABLE_RELAY_ENDPOINT:
enable_relay_endpoint = true;
break;
case OPT_ENABLE_RELAY_ENDPOINT_ON:
relay_endpoint_sockets.emplace(optarg);
break;
default:
PrintUsage(argv[0]);
return 1;
Expand Down Expand Up @@ -154,17 +172,14 @@ int PERFETTO_EXPORT_ENTRYPOINT ServiceMain(int argc, char** argv) {
}
#endif

std::string relay_producer_socket;
std::string android_relay_socket;
#if PERFETTO_BUILDFLAG(PERFETTO_OS_ANDROID)
relay_producer_socket = base::GetAndroidProp("traced.relay_producer_port");
// If a guest producer port is defined, then the relay endpoint should be
// enabled regardless. This is used in cases where perf data is passed
// from guest machines or the hypervisor to Android.
if (!relay_producer_socket.empty())
init_opts.enable_relay_endpoint = true;
// On Android, a relay-only producer socket can be configured via system
// property — e.g. when perf data is forwarded from guest VMs or the
// hypervisor into the host's traced. The named socket is appended to the
// producer endpoint list below and marked relay-capable.
android_relay_socket = base::GetAndroidProp("traced.relay_producer_port");
#endif
if (enable_relay_endpoint)
init_opts.enable_relay_endpoint = true;
svc = ServiceIPCHost::CreateInstance(&task_runner, init_opts);

// When built as part of the Android tree, the two socket are created and
Expand All @@ -180,18 +195,45 @@ int PERFETTO_EXPORT_ENTRYPOINT ServiceMain(int argc, char** argv) {
#else
ListenEndpoint consumer_ep(base::ScopedFile(atoi(env_cons)));
std::list<ListenEndpoint> producer_eps;
// The init-bound FD is the local producer socket. The relay endpoint, if
// any, is the separate socket named by traced.relay_producer_port (added
// below), never this one.
producer_eps.emplace_back(ListenEndpoint(base::ScopedFile(atoi(env_prod))));
if (!relay_producer_socket.empty()) {
producer_eps.emplace_back(ListenEndpoint(relay_producer_socket));
if (!android_relay_socket.empty()) {
ListenEndpoint relay_ep(android_relay_socket);
relay_ep.expose_relay_endpoint = true;
producer_eps.emplace_back(std::move(relay_ep));
}
started = svc->Start(std::move(producer_eps), std::move(consumer_ep));
#endif
} else {
std::list<ListenEndpoint> producer_eps;
auto producer_socket_names = TokenizeProducerSockets(GetProducerSocket());
std::set<std::string> seen_relay_selectors;
for (const auto& producer_socket_name : producer_socket_names) {
remove(producer_socket_name.c_str());
producer_eps.emplace_back(ListenEndpoint(producer_socket_name));
ListenEndpoint ep(producer_socket_name);
// RelayPort is exposed on this socket if either:
// --enable-relay-endpoint was passed (applies to all
// PERFETTO_PRODUCER_SOCK_NAME sockets), OR
// --enable-relay-endpoint-on=<this socket> was passed (selects
// specific producer sockets).
bool selected = relay_endpoint_sockets.count(producer_socket_name) > 0;
if (selected)
seen_relay_selectors.insert(producer_socket_name);
ep.expose_relay_endpoint = enable_relay_endpoint || selected;
producer_eps.emplace_back(std::move(ep));
}
for (const auto& selector : relay_endpoint_sockets) {
if (seen_relay_selectors.count(selector) == 0) {
PERFETTO_ELOG(
"--enable-relay-endpoint-on=%s does not match any socket in "
"PERFETTO_PRODUCER_SOCK_NAME / the default producer socket "
"(%s). The flag selects from existing producer sockets; it does "
"not introduce new endpoints.",
selector.c_str(), GetProducerSocket());
return 1;
}
}
remove(GetConsumerSocket());
started = svc->Start(std::move(producer_eps),
Expand Down
24 changes: 12 additions & 12 deletions src/tracing/ipc/service/service_ipc_host_impl.cc
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,9 @@ bool ServiceIPCHostImpl::Start(std::list<ListenEndpoint> producer_sockets,

// Initialize the IPC transport.
for (auto& sock : producer_sockets) {
producer_ipc_ports_.emplace_back(
CreateIpcHost(task_runner_, std::move(sock)));
bool expose_relay = sock.expose_relay_endpoint;
producer_ipc_ports_.push_back(
{CreateIpcHost(task_runner_, std::move(sock)), expose_relay});
}
consumer_ipc_port_ = CreateIpcHost(task_runner_, std::move(consumer_socket));

Expand All @@ -98,10 +99,9 @@ bool ServiceIPCHostImpl::DoStart() {
init_opts_);

if (producer_ipc_ports_.empty() || !consumer_ipc_port_ ||
std::any_of(producer_ipc_ports_.begin(), producer_ipc_ports_.end(),
[](const std::unique_ptr<ipc::Host>& port) {
return port == nullptr;
})) {
std::any_of(
producer_ipc_ports_.begin(), producer_ipc_ports_.end(),
[](const ProducerIPCPort& port) { return port.host == nullptr; })) {
Shutdown();
return false;
}
Expand All @@ -115,20 +115,20 @@ bool ServiceIPCHostImpl::DoStart() {
// consumer port ipcs might exhaust the send buffer under normal operation
// due to large messages such as ReadBuffersResponse.
for (auto& producer_ipc_port : producer_ipc_ports_)
producer_ipc_port->SetSocketSendTimeoutMs(kProducerSocketTxTimeoutMs);
producer_ipc_port.host->SetSocketSendTimeoutMs(kProducerSocketTxTimeoutMs);

// TODO(fmayer): add a test that destroys the ServiceIPCHostImpl soon after
// Start() and checks that no spurious callbacks are issued.
for (auto& producer_ipc_port : producer_ipc_ports_) {
bool producer_service_exposed = producer_ipc_port->ExposeService(
bool producer_service_exposed = producer_ipc_port.host->ExposeService(
std::unique_ptr<ipc::Service>(new ProducerIPCService(svc_.get())));
PERFETTO_CHECK(producer_service_exposed);

if (!init_opts_.enable_relay_endpoint)
// Optionally also expose the RelayPort service on this port, when the
// caller has explicitly opted in for cross-machine relay traffic.
if (!producer_ipc_port.expose_relay_endpoint)
continue;
// Expose a secondary service for sync with remote relay service
// if requested.
bool relay_service_exposed = producer_ipc_port->ExposeService(
bool relay_service_exposed = producer_ipc_port.host->ExposeService(
std::unique_ptr<ipc::Service>(new RelayIPCService(svc_.get())));
PERFETTO_CHECK(relay_service_exposed);
}
Expand Down
8 changes: 7 additions & 1 deletion src/tracing/ipc/service/service_ipc_host_impl.h
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,13 @@ class ServiceIPCHostImpl : public ServiceIPCHost {
// Note that there can be multiple producer sockets if it's specified in the
// producer socket name (e.g. for listening both on vsock for VMs and AF_UNIX
// for processes on the same machine).
std::vector<std::unique_ptr<ipc::Host>> producer_ipc_ports_;
// The `expose_relay_endpoint` bit gates whether the RelayIPCService is
// exposed on that specific port (see ListenEndpoint).
struct ProducerIPCPort {
std::unique_ptr<ipc::Host> host;
bool expose_relay_endpoint = false;
};
std::vector<ProducerIPCPort> producer_ipc_ports_;

// As above, but for the Consumer port.
std::unique_ptr<ipc::Host> consumer_ipc_port_;
Expand Down
16 changes: 10 additions & 6 deletions test/test_helper.h
Original file line number Diff line number Diff line change
Expand Up @@ -132,10 +132,8 @@ class ServiceThread {
{"PERFETTO_PRODUCER_SOCK_NAME", "PERFETTO_CONSUMER_SOCK_NAME"});
runner_ = base::ThreadTaskRunner::CreateAndStart("perfetto.svc");
runner_->PostTaskAndWaitForTesting([this]() {
TracingService::InitOpts init_opts = {};
if (enable_relay_endpoint_)
init_opts.enable_relay_endpoint = true;
svc_ = ServiceIPCHost::CreateInstance(runner_->get(), init_opts);
svc_ = ServiceIPCHost::CreateInstance(runner_->get(),
TracingService::InitOpts{});
auto producer_sockets = TokenizeProducerSockets(producer_socket_.c_str());
for (const auto& producer_socket : producer_sockets) {
// In some cases the socket is a TCP or abstract unix.
Expand All @@ -152,8 +150,14 @@ class ServiceThread {
}
base::SetEnv("PERFETTO_PRODUCER_SOCK_NAME", producer_socket_);
base::SetEnv("PERFETTO_CONSUMER_SOCK_NAME", consumer_socket_);
bool res =
svc_->Start(producer_socket_.c_str(), consumer_socket_.c_str());
std::list<ListenEndpoint> producer_eps;
for (const auto& producer_socket : producer_sockets) {
ListenEndpoint ep(producer_socket);
ep.expose_relay_endpoint = enable_relay_endpoint_;
producer_eps.emplace_back(std::move(ep));
}
bool res = svc_->Start(std::move(producer_eps),
ListenEndpoint(consumer_socket_));
if (!res) {
PERFETTO_FATAL("Failed to start service listening on %s and %s",
producer_socket_.c_str(), consumer_socket_.c_str());
Expand Down
Loading