Update profiles specification README.md and pprof.md

CHANGELOG.md

@@ -28,6 +28,9 @@ release.
 
 ### Profiles
 
+- Remove duplicate information from and extend Profiles documentation (README.md, pprof.md).
+  ([#4932](https://github.com/open-telemetry/opentelemetry-specification/pull/4932))
+
 ### Resource
 
 ### Entities

specification/profiles/README.md

@@ -6,47 +6,80 @@ path_base_for_github_subdir:
 
 # OpenTelemetry Profiles
 
+**Status**: [Development](../document-status.md)
+
+> [!NOTE]
+>
+> The profiles signal is still experimental and under active development.
+> Breaking changes may be introduced in future versions.
+
 <details>
 <summary>Table of Contents</summary>
 
 <!-- toc -->
 
 - [Overview](#overview)
+- [Design goals](#design-goals)
+- [Data Model](#data-model)
 - [Known values](#known-values)
+- [Specifications](#specifications)
+- [References](#references)
 
 <!-- tocstop -->
 
 </details>
 
 ## Overview
 
-Profiles are emerging as the fourth essential signal of observability, alongside
-logs, metrics, and traces. They offer unparalleled insights into system and
-application behavior, often uncovering performance bottlenecks overlooked by
-other signals.
-
-Profiles provide granular, time-based views of resource consumption and
-code execution, encompassing:
-
-* **Application-level profiling**: Reveals how software functions consume CPU,
-memory, and other resources, highlighting slow or inefficient code.
-
-* **System-level profiling**: Offers a holistic view of the infrastructure,
-pinpointing issues in operating system calls, kernel operations, and I/O.
-
-This performance picture can lead to:
-
-* **Faster Root Cause Analysis**: Quickly identifies the exact cause of
-performance degradation.
-* **Proactive Optimization**: Identifies potential issues before user impact.
-* **Improved Resource Utilization**: Optimizes infrastructure for cost savings
-and efficiency.
-* **Enhanced Developer Productivity**: Helps developers validate code performance
-and prevent regressions.
-
-In essence, while logs, metrics, and traces show "what" and "how much/where"
-profiles explain "why" and "how efficiently" making them indispensable in modern
-observability.
+A **profile** is a collection of stack traces with associated values
+representing resource consumption, collected from a running program. Profiling
+is the process of collecting such profiles, typically by sampling program state
+at regular intervals.
+
+For a general introduction to OpenTelemetry profiling and how it complements other
+observability signals, see [Profiles concepts](https://opentelemetry.io/docs/concepts/signals/profiles/).
+
+## Design goals
+
+The profiles signal is designed with the following goals in mind:
+
+- **Low overhead**: Enable profiling agents to operate continuously in production
+  environments without materially impacting application performance.
+- **Efficient representation**: Reduces volume of data stored and transmitted,
+  by using dictionary tables to deduplicate repeated information across samples.
+- **Compatibility with existing formats**: The data model is a superset of
+  established profiling formats such as [pprof](https://github.com/google/pprof)
+  and in most cases supports lossless conversions to and from these formats.
+  If that's not possible (e.g. custom extensions), the `original_payload_format`
+  field can be used to transmit the original information for future lossless
+  export or reinterpretation.
+- **Correlation with other signals**: Profiles MUST be linkable to logs, metrics
+  and traces through shared resource context and, where applicable,
+  direct trace/span references. This is the primary differentiator of OpenTelemetry
+  profiles compared to standalone profiling solutions.
+
+## Data model
+
+The OpenTelemetry profiles data model is defined [here](./data-model.md).
+It builds on the [pprof protobuf format](https://github.com/google/pprof/tree/main/proto)
+and extends it with:
+
+- **Resource and scope context**: Each batch of profiles is associated with a
+  [Resource](../resource/README.md) and an [InstrumentationScope](../common/instrumentation-scope.md),
+  consistent with logs, metrics and traces.
+- **Generalized dictionary**: Deduplicates not only strings but also other messages
+  that exhibit redundancy.
+- **Generalized attributes**: Most messages can carry attributes following the
+  same conventions as other signals, augmented with Unit information (`KeyValueAndUnit`).
+- **Span context references**: Samples MAY include a `Link` (span ID and trace ID),
+  enabling direct linking between a profile sample and the trace/span during
+  which it was captured.
+
+For details on the required attributes for `Mapping` messages and the custom
+hashing scheme for build id generation, see [Mappings](./mappings.md).
+
+For more information on compatibility with [pprof](https://github.com/google/pprof),
+see [pprof](./pprof.md).
 
 ## Known values
 
@@ -61,3 +94,16 @@ tools, known values are utilized.
 | Profile field | Known values |
 | ------------- | ------------ |
 | original_payload_format | [pprof](https://github.com/google/pprof/tree/main/proto), [jfr](https://en.wikipedia.org/wiki/JDK_Flight_Recorder) or [linux_perf](https://perfwiki.github.io/) |
+
+## Specifications
+
+* [Profiles Data Model](./data-model.md)
+* [Profiles Mappings Attributes](./mappings.md)
+* [Profiles Pprof Compatibility](./pprof.md)
+
+## References
+
+- [Profiles Concepts](https://opentelemetry.io/docs/concepts/signals/profiles/)
+- [Profiles Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/general/profiles/)
+- [OTEP0212 OpenTelemetry Profiles Vision](../../oteps/profiles/0212-profiling-vision.md)
+- [OTEP0239 OpenTelemetry Profiles Data Model](../../oteps/profiles/0239-profiles-data-model.md)

specification/profiles/pprof.md

@@ -10,10 +10,11 @@
 
 ## Compatibility
 
-Original [pprof](https://github.com/google/pprof/tree/main/proto) is forward
-compatible with OpenTelemetry Profiles in a sense that it can be transformed into
-OpenTelemetry Profiles and again into [pprof](https://github.com/google/pprof/tree/main/proto)
-without data loss.
+Original [pprof](https://github.com/google/pprof/tree/main/proto) is compatible
+with OpenTelemetry Profiles in that it can be transformed into OpenTelemetry Profiles
+and back again into [pprof](https://github.com/google/pprof/tree/main/proto) without
+data loss (convertibility but not wire compatibility).
+
+To enable this compatibility through explicit conversion, OpenTelemetry provides pprof-specific [guidelines](https://opentelemetry.io/docs/specs/semconv/general/profiles/#compatibility-with-pprof)
+in [Semantic Conventions](https://opentelemetry.io/docs/specs/semconv).
 
-For this compatibility OpenTelemetry also provides a `pprof` namespace in
-Semantic Conventions.