Skip to content

yaml2obj: Implement Coverage mapping format #129472

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
212 changes: 212 additions & 0 deletions llvm/include/llvm/ObjectYAML/CovMap.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
//===- CovMap.h - ObjectYAML Interface for coverage map ---------*- C++ -*-===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
//
// - llvm::coverage::yaml
//
// Describes binary file formats and YAML structures of coverage map.
//
// - llvm::yaml
//
// Attachments for YAMLTraits.
//
// - llvm::covmap
//
// Provides YAML encoder for coverage map.
//
//===----------------------------------------------------------------------===//

#ifndef LLVM_OBJECTYAML_COVMAP_H
#define LLVM_OBJECTYAML_COVMAP_H

#include "llvm/ADT/StringRef.h"
#include "llvm/ObjectYAML/ELFYAML.h"
#include "llvm/Support/Endian.h"
#include "llvm/Support/YAMLTraits.h"
#include <array>
#include <cstdint>
#include <memory>
#include <optional>
#include <string>
#include <utility>
#include <vector>

namespace llvm {
class raw_ostream;
} // namespace llvm

namespace llvm::coverage::yaml {

/// Base Counter, corresponding to coverage::Counter.
struct CounterTy {
enum TagTy : uint8_t {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why the name "TagTy"?

this refers to the type of operation, correct?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make more sense to call this "OpTy" or something?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
enum TagTy : uint8_t {
enum OperationTy : uint8_t {

Zero = 0,
Ref,
Sub,
Add,
};

TagTy Tag;
uint64_t Val;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Val is the value of the counter? Can you document that?


virtual ~CounterTy() {}

virtual void mapping(llvm::yaml::IO &IO);

void encode(raw_ostream &OS) const;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a doxygen comment for this? e.g.

Encodes \p OS using ULEB128.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, would it possibly be useful to return the unsigned from ULEB128?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
void encode(raw_ostream &OS) const;
/// Encodes \p OS using ULEB128.
void encode(raw_ostream &OS) const;

};

/// Holds a pair of both hands but doesn't hold ops(add or sub).
/// Ops is stored in CounterTy::Tag.
using ExpressionTy = std::array<CounterTy, 2>;

/// {True, False}
using BranchTy = std::array<CounterTy, 2>;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be OK to use a struct here?

struct BranchTy {
  CounterTy True;
  CounterTy False;
};

Then later, when you're encoding, it would be easer to read, because you could encode True and False directly rather than [0] and [1].

If it is considerably more performant to use std::array, then it's fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to make yaml text compact.


/// {ID, TrueID, FalseID}
/// Note: This has +1 offset unlike mcdc::ConditionID.
using MCDCBranchTy = std::array<uint16_t, 3>;

struct DecisionTy {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you write a doxygen comment for this struct?

uint64_t BIdx; ///< Bitmap index
uint64_t NC; ///< NumConds

void mapping(llvm::yaml::IO &IO);

void encode(raw_ostream &OS) const;
};

/// {LineStart, ColumnStart, LineEnd, ColumnEnd}
using LocTy = std::array<uint64_t, 4>;

/// Region record.
/// CounterTy is enhanced if Tag is Zero and Val is not zero.
struct RecTy : CounterTy {
enum ExtTagTy : uint8_t {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a doxygen comment for this?

Considering we already have something called TagTy, this seems to be related to opcodes from a first glance at the code. However, this is information about coverage.

Would it be possible to come up with a name that makes that very clear for this type?

It could be something like DecisionTy.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
enum ExtTagTy : uint8_t {
enum DecisionTy : uint8_t {

Skip = 2,
Branch = 4,
Decision = 5,
MCDCBranch = 6,
};

/// This is optional in detailed view.
std::optional<ExtTagTy> ExtTag;

// Options for extensions.
std::optional<uint64_t> Expansion; ///< Doesn't have ExtTag.
std::optional<BranchTy> BranchOpt; ///< Optionally has MCDC.
std::optional<MCDCBranchTy> MCDC;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be used optionally.

std::optional<DecisionTy> DecisionOpt;

/// True or None.
/// Stored in ColumnEnd:31.
std::optional<bool> isGap;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain what a Gap is in this comment?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
std::optional<bool> isGap;
std::optional<bool> IsGap;


LocTy dLoc; ///< Differential line numbers.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
LocTy dLoc; ///< Differential line numbers.
LocTy DLoc; ///< Differential line numbers.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LLVM naming conventions


void mapping(llvm::yaml::IO &IO) override;

void encode(raw_ostream &OS) const;
};

/// {NumRecs, Recs...}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NumRecs not present?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, can you add a comment that describes what this does in the context of the code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to mention here, NumRecs is in encoded file. Is this noisy?

struct FileRecsTy {
std::vector<RecTy> Recs;

void mapping(llvm::yaml::IO &IO);
};

/// An element of CovFun array.
struct CovFunTy {
llvm::yaml::Hex64 NameRef; ///< Hash value of the symbol.
llvm::yaml::Hex64 FuncHash; ///< Signature of this function.
llvm::yaml::Hex64 FilenamesRef; ///< Pointer to CovMap
std::vector<unsigned> FileIDs; ///< Resolved by CovMap
std::vector<ExpressionTy> Expressions;
std::vector<FileRecsTy> Files; ///< 2-dimension array of Recs.

void mapping(llvm::yaml::IO &IO);

void encode(raw_ostream &OS, endianness Endianness) const;
};

/// An element of CovMap array.
struct CovMapTy {
/// This is the key of CovMap but not present in the file
/// format. Calculate and store with Filenames.
llvm::yaml::Hex64 FilenamesRef;

uint32_t Version;

/// Raw Filenames (and storage of Files)
std::vector<std::string> Filenames;

void mapping(llvm::yaml::IO &IO);

/// Encode Filenames. This is mostly used just to obtain FilenamesRef.
std::pair<uint64_t, std::string> encodeFilenames(bool Compress = false) const;

void encode(raw_ostream &OS, endianness Endianness) const;
};

} // namespace llvm::coverage::yaml

LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::coverage::yaml::CovMapTy)
LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::coverage::yaml::CovFunTy)
LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::coverage::yaml::ExpressionTy)
LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::coverage::yaml::RecTy)
LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::coverage::yaml::FileRecsTy)
LLVM_YAML_IS_FLOW_SEQUENCE_VECTOR(llvm::coverage::yaml::CounterTy)

#define LLVM_COVERAGE_YAML_ELEM_MAPPING(Ty) \
namespace llvm::yaml { \
template <> struct MappingTraits<llvm::coverage::yaml::Ty> { \
static void mapping(IO &IO, llvm::coverage::yaml::Ty &Obj) { \
Obj.mapping(IO); \
} \
}; \
}

/// `Flow` is used for emission of a compact oneliner for RecTy.
#define LLVM_COVERAGE_YAML_ELEM_MAPPING_FLOW(Ty) \
namespace llvm::yaml { \
template <> struct MappingTraits<llvm::coverage::yaml::Ty> { \
static void mapping(IO &IO, llvm::coverage::yaml::Ty &Obj) { \
Obj.mapping(IO); \
(void)flow; \
} \
static const bool flow = true; \
}; \
}

#define LLVM_COVERAGE_YAML_ENUM(Ty) \
namespace llvm::yaml { \
template <> struct ScalarEnumerationTraits<llvm::coverage::yaml::Ty> { \
static void enumeration(IO &IO, llvm::coverage::yaml::Ty &Value); \
}; \
}

LLVM_COVERAGE_YAML_ENUM(CounterTy::TagTy)
LLVM_COVERAGE_YAML_ENUM(RecTy::ExtTagTy)
LLVM_COVERAGE_YAML_ELEM_MAPPING_FLOW(CounterTy)
LLVM_COVERAGE_YAML_ELEM_MAPPING_FLOW(DecisionTy)
LLVM_COVERAGE_YAML_ELEM_MAPPING_FLOW(RecTy)
LLVM_COVERAGE_YAML_ELEM_MAPPING(FileRecsTy)
LLVM_COVERAGE_YAML_ELEM_MAPPING(CovFunTy)
LLVM_COVERAGE_YAML_ELEM_MAPPING(CovMapTy)

namespace llvm::covmap {

/// Returns whether Name is interested.
bool nameMatches(StringRef Name);

/// Returns a new ELFYAML Object.
std::unique_ptr<ELFYAML::CovMapSectionBase> make_unique(StringRef Name);

} // namespace llvm::covmap

#endif // LLVM_OBJECTYAML_COVMAP_H
14 changes: 14 additions & 0 deletions llvm/include/llvm/ObjectYAML/ELFYAML.h
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,7 @@ struct Chunk {
DependentLibraries,
CallGraphProfile,
BBAddrMap,
CovMapBase,

// Special chunks.
SpecialChunksStart,
Expand Down Expand Up @@ -398,6 +399,19 @@ struct RawContentSection : Section {
std::optional<std::vector<uint8_t>> ContentBuf;
};

struct CovMapSectionBase : Section {
std::optional<llvm::yaml::Hex64> Info;

CovMapSectionBase() : Section(ChunkKind::CovMapBase) {}

virtual void mapping(yaml::IO &IO) = 0;
virtual Error encode(raw_ostream &OS, endianness Endianness) const = 0;

static bool classof(const Chunk *S) {
return S->Kind == ChunkKind::CovMapBase;
}
};

struct NoBitsSection : Section {
NoBitsSection() : Section(ChunkKind::NoBits) {}

Expand Down
3 changes: 3 additions & 0 deletions llvm/lib/ObjectYAML/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ add_llvm_component_library(LLVMObjectYAML
CodeViewYAMLTypes.cpp
COFFEmitter.cpp
COFFYAML.cpp
CovMap.cpp
DWARFEmitter.cpp
DWARFYAML.cpp
DXContainerEmitter.cpp
Expand Down Expand Up @@ -34,7 +35,9 @@ add_llvm_component_library(LLVMObjectYAML

LINK_COMPONENTS
BinaryFormat
Coverage
Object
ProfileData
Support
TargetParser
DebugInfoCodeView
Expand Down
Loading