Nominate a specific, canonical extension for pprof files

[mhansen]

Hi, over on internal bug b/487062289 I'm working on benchmarks that output a .pprof file into sponge.

I'd like to have a button where you can click from the benchmark results to go to the pprof web UI. We already have this for Perfetto, it's nice.

I need to match for pprof files. For Perfetto, we have a PerfettoMatchUtil.java:

  private static final ImmutableSet<String> PERFETTO_EXTENSIONS =
      ImmutableSet.of(
          ".pftrace.gz", // UTP Perfetto Plugin
          ".pftrace",
          "_perfetto.pb.gz",
          "_perfetto.pb",
          ".perfetto-trace.gz", // Extension used in Perfetto docs
          ".perfetto-trace");

It's a bit of a mess (indicating they could have chosen a canonical extension earlier), but it works.

I'd like a canonical, distinguishing file extension for pprof profiles. But I need a file extension matcher that doesn't give me too many false positives.

There's a lot of talk in pprof about outputting .pb and .pb.gz. These are perhaps canonical, but not distinct from the myriad other wireformat protocol buffer formats.

Not distinct enough to offer a 'open in pprof' link from generic testing infrastructure; they'd probably have too many false positives.

I propose we just declare that pprof files canonically use extensions:

• foo.pprof
• foo.pprof.gz

And we update the docs that talk about .pb .pb.gz to use these as examples.

Thoughts?

[aalexand]

I'm somewhat skeptical about the value of this. There isn't a lot of precedent for this internally at Google and the most common extension is *.pb.gz. Also, the amount of the documentation and the code here and there that produces pprof files is going to be massive, so my worry is that the change / agreement itself is simple but it's the pprof team that will end up owning the eventual cleanup.

Is the Sponge case of double-clicking on a file (and Sponge routing that by file name pattern, not by file content sniffing) the main motivation or is there more? (Besides "it's cleaner this way")

[mhansen]

There isn't a lot of precedent for this internally at Google and the most common extension is *.pb.gz.

Right, not much precedent yet. I could make a precedent - go ahead and route in sponge based on .pprof or pprof.gz, but I wanted to ask pprof team first, lest you want some other extension and then we end up with 6 regexes like Perfetto has.

Is the Sponge case of double-clicking on a file (and Sponge routing that by file name pattern, not by file content sniffing) the main motivation or is there more?

Yes, that's my motivation today. More use cases I can imagine (not my use case today, but just to make the argument for how this'd be more broadly useful):

• Route buganizer attachments by file extension
• OS file association by extension to open a file in a profile reader
• HTML file upload forms could restrict by extension
• When dragging a file into a web tool, it might not have to do content sniffing, may be able to route based on the extension.

I don't think we have the option to do content-sniffing: (1) Sponge doesn't read the file first, and (2) the content is a generic wire-encoded protobuf with no magic number, it's quite involved to figure out if it matches a schema (well, it's possible but requires reading a lot of the file - internal tool gqui does this but it's slow).

There's an alternative here: I could leave it as .pb and .pb.gz and offer a context menu link to "Open in pprof" that the user can opt into. I suppose, like right-clicking a text file and choosing Open in Notepad or Open in Vim. The challenge would be if we have so many false-positives from other types of .pb files. In particular, people may get confused by perfetto vs pprof profiles which are both software performance profiles ending in .pb. We'd have lots of false positives within Google, probably fewer outside.

Also, the amount of the documentation and the code here and there that produces pprof files is going to be massive, so my worry is that the change / agreement itself is simple but it's the pprof team that will end up owning the eventual cleanup.

I thought the documentation might not be so much. The code that produces pprof is fairly large, yes. I was hoping that we could document something like: "pprof's canonical extension is .pprof or .pprof.gz"; and leave old producers in-place, allow adoption of the new filename in new tools that want to benefit from file extension routing.

I'm open to the idea that having multiple extensions (.pb and .pprof) isn't worth it, and that we can live with the collisions. I suppose there are also collisions in DOS-style 3-letter file extensions too, and we got by. But I do think there'd be some benefits to uniquely, distinctly identifying pprof files that we could optionally allow people to do; and if we don't do this, we might end up with a proliferation of different extensions.