Skip to content

[GOFF] Refactor writing GOFF records #93855

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

[GOFF] Refactor writing GOFF records #93855

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
da97d3d
[GOFF] Refactor writing GOFF records
redstar May 13, 2024
d850899
Replace combination of sed and bash expression with awk in LIT test
redstar May 31, 2024
ebd2444
Change awk statement in LIT test again
redstar May 31, 2024
a778e25
Changes based on reviewer comments.
redstar Jun 7, 2024
6f26aa8
Remove additional braces around argument list
redstar Jun 7, 2024
641f644
Small changes
redstar Jun 10, 2024
020589f
Updates based on reviewer comments
redstar Jun 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions llvm/include/llvm/ObjectYAML/GOFFYAML.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,40 +33,40 @@ LLVM_YAML_STRONG_TYPEDEF(uint8_t, GOFF_ENDFLAGS)
// forward references to records, etc.). However, to be able to specify invalid
// GOFF files, we treat all records the same way.
struct RecordBase {
enum RecordBaseKind {
RBK_ModuleHeader,
RBK_RelocationDirectory,
RBK_Symbol,
RBK_Text,
RBK_DeferredLength,
RBK_EndOfModule
enum class Kind {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you just use the RecordType enum from llvm/BinaryFormat/GOFF.h here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those are currently the same, but I do not think that it is good to couple the record type with the class hierarchy discriminator. E.g. in C++ I could model a relocation entry as a record, and treat it as a sub-record of the relocation directory. I currently do not do this, but coupling the discriminator to the record type would be the design decision that I will never do that, which seems a bit limiting to me.

ModuleHeader,
RelocationDirectory,
Symbol,
Text,
DeferredLength,
EndOfModule
};

private:
const RecordBaseKind Kind;
const Kind RecordKind;

protected:
RecordBase(RecordBaseKind Kind) : Kind(Kind) {}
RecordBase(Kind RecordKind) : RecordKind(RecordKind) {}

public:
RecordBaseKind getKind() const { return Kind; }
Kind getKind() const { return RecordKind; }
};
using RecordPtr = std::unique_ptr<RecordBase>;

struct ModuleHeader : public RecordBase {
ModuleHeader() : RecordBase(RBK_ModuleHeader) {}
ModuleHeader() : RecordBase(Kind::ModuleHeader) {}

uint32_t ArchitectureLevel;
uint16_t PropertiesLength;
std::optional<yaml::BinaryRef> Properties;

static bool classof(const RecordBase *S) {
return S->getKind() == RBK_ModuleHeader;
return S->getKind() == Kind::ModuleHeader;
}
};

struct EndOfModule : public RecordBase {
EndOfModule() : RecordBase(RBK_EndOfModule) {}
EndOfModule() : RecordBase(Kind::EndOfModule) {}

GOFF_ENDFLAGS Flags;
GOFF_AMODE AMODE;
Expand All @@ -77,7 +77,7 @@ struct EndOfModule : public RecordBase {
StringRef EntryName;

static bool classof(const RecordBase *S) {
return S->getKind() == RBK_EndOfModule;
return S->getKind() == Kind::EndOfModule;
}
};

Expand Down
83 changes: 44 additions & 39 deletions llvm/lib/ObjectYAML/GOFFEmitter.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,7 @@ ZerosImpl zeros(const size_t NumBytes) { return ZerosImpl{NumBytes}; }
// The implementation aims at simplicity, not speed.
class GOFFOStream {
public:
explicit GOFFOStream(raw_ostream &OS)
: OS(OS), CurrentType(GOFF::RecordType(-1)) {}
explicit GOFFOStream(raw_ostream &OS) : OS(OS), CurrentType() {}

GOFFOStream &operator<<(StringRef Str) {
write(Str);
Expand All @@ -95,35 +94,37 @@ class GOFFOStream {
};

void GOFFOStream::writeRecordPrefix(uint8_t Flags) {
// See https://www.ibm.com/docs/en/zos/3.1.0?topic=conventions-record-prefix.
uint8_t TypeAndFlags = Flags | (CurrentType << 4);
OS << binaryBe(static_cast<unsigned char>(GOFF::PTVPrefix))
<< binaryBe(static_cast<unsigned char>(TypeAndFlags))
<< binaryBe(static_cast<unsigned char>(0));
OS << binaryBe(uint8_t(GOFF::PTVPrefix)) // The prefix value.
<< binaryBe(uint8_t(TypeAndFlags)) // The record type and the flags.
<< binaryBe(uint8_t(0)); // The version.
}

void GOFFOStream::write(StringRef Str) {
// The flags are determined by the flags of the prvious record, and by the
// remaining size of data.
uint8_t Flags = 0;
size_t Ptr = 0;
// size of the remaining data.
size_t Pos = 0;
size_t Size = Str.size();
while (Size >= GOFF::RecordContentLength) {
if (Flags) {
bool Continuation = false;
while (Size > 0) {
uint8_t Flags = 0;
if (Continuation)
Flags |= Rec_Continuation;
if (Size == GOFF::RecordContentLength)
Flags &= ~Rec_Continued;
} else
Flags |= (Size == GOFF::RecordContentLength) ? 0 : Rec_Continued;
writeRecordPrefix(Flags);
OS.write(&Str.data()[Ptr], GOFF::RecordContentLength);
Size -= GOFF::RecordContentLength;
Ptr += GOFF::RecordContentLength;
}
if (Size) {
Flags &= ~Rec_Continued;
if (Size > GOFF::RecordContentLength) {
Flags |= Rec_Continued;
Continuation = true;
}
writeRecordPrefix(Flags);
OS.write(&Str.data()[Ptr], Size);
OS.write_zeros(GOFF::RecordContentLength - Size);
if (Size < GOFF::RecordContentLength) {
OS.write(&Str.data()[Pos], Size);
OS.write_zeros(GOFF::RecordContentLength - Size);
Size = 0;
} else {
OS.write(&Str.data()[Pos], GOFF::RecordContentLength);
Size -= GOFF::RecordContentLength;
}
Pos += GOFF::RecordContentLength;
}
}

Expand All @@ -145,8 +146,8 @@ class LogicalRecord : public raw_svector_ostream {
};

class GOFFState {
void writeHeader(GOFFYAML::ModuleHeader &ModHdr);
void writeEnd(GOFFYAML::EndOfModule &EndMod);
void writeHeader(const GOFFYAML::ModuleHeader &ModHdr);
void writeEnd(const GOFFYAML::EndOfModule &EndMod);

void reportError(const Twine &Msg) {
ErrHandler(Msg);
Expand All @@ -170,7 +171,9 @@ class GOFFState {
bool HasError;
};

void GOFFState::writeHeader(GOFFYAML::ModuleHeader &ModHdr) {
void GOFFState::writeHeader(const GOFFYAML::ModuleHeader &ModHdr) {
// See
// https://www.ibm.com/docs/en/zos/3.1.0?topic=formats-module-header-record.
GW.newRecord(GOFF::RT_HDR);
LogicalRecord LR(GW);
LR << zeros(45) // Reserved.
Expand All @@ -181,11 +184,13 @@ void GOFFState::writeHeader(GOFFYAML::ModuleHeader &ModHdr) {
LR << *ModHdr.Properties; // Module properties.
}

void GOFFState::writeEnd(GOFFYAML::EndOfModule &EndMod) {
void GOFFState::writeEnd(const GOFFYAML::EndOfModule &EndMod) {
// See https://www.ibm.com/docs/en/zos/3.1.0?topic=formats-end-module-record.
SmallString<16> EntryName;
if (std::error_code EC =
ConverterEBCDIC::convertToEBCDIC(EndMod.EntryName, EntryName))
reportError("Conversion error on " + EndMod.EntryName);
reportError("conversion to EBCDIC 1047 failed on '" + EndMod.EntryName +
"'");

GW.newRecord(GOFF::RT_END);
LogicalRecord LR(GW);
Expand All @@ -201,20 +206,20 @@ void GOFFState::writeEnd(GOFFYAML::EndOfModule &EndMod) {
}

bool GOFFState::writeObject() {
for (auto &RecPtr : Doc.Records) {
auto *Rec = RecPtr.get();
for (const std::unique_ptr<GOFFYAML::RecordBase> &RecPtr : Doc.Records) {
const GOFFYAML::RecordBase *Rec = RecPtr.get();
switch (Rec->getKind()) {
case GOFFYAML::RecordBase::RBK_ModuleHeader:
writeHeader(*static_cast<GOFFYAML::ModuleHeader *>(Rec));
case GOFFYAML::RecordBase::Kind::ModuleHeader:
writeHeader(*static_cast<const GOFFYAML::ModuleHeader *>(Rec));
break;
case GOFFYAML::RecordBase::RBK_EndOfModule:
writeEnd(*static_cast<GOFFYAML::EndOfModule *>(Rec));
case GOFFYAML::RecordBase::Kind::EndOfModule:
writeEnd(*static_cast<const GOFFYAML::EndOfModule *>(Rec));
break;
case GOFFYAML::RecordBase::RBK_RelocationDirectory:
case GOFFYAML::RecordBase::RBK_Symbol:
case GOFFYAML::RecordBase::RBK_Text:
case GOFFYAML::RecordBase::RBK_DeferredLength:
llvm_unreachable(("Not yet implemented"));
case GOFFYAML::RecordBase::Kind::RelocationDirectory:
case GOFFYAML::RecordBase::Kind::Symbol:
case GOFFYAML::RecordBase::Kind::Text:
case GOFFYAML::RecordBase::Kind::DeferredLength:
llvm_unreachable("not yet implemented");
}
if (HasError)
return false;
Expand Down
18 changes: 9 additions & 9 deletions llvm/lib/ObjectYAML/GOFFYAML.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,26 +71,26 @@ void CustomMappingTraits<GOFFYAML::RecordPtr>::inputOne(
Elem = std::make_unique<GOFFYAML::EndOfModule>(std::move(End));
} else if (Key == "RelocationDirectory" || Key == "Symbol" || Key == "Text" ||
Key == "Length")
IO.setError(Twine("Not yet implemented ").concat(Key));
IO.setError(Twine("not yet implemented ").concat(Key));
else
IO.setError(Twine("Unknown record type name ").concat(Key));
IO.setError(Twine("unknown record type name ").concat(Key));
}

void CustomMappingTraits<GOFFYAML::RecordPtr>::output(
IO &IO, GOFFYAML::RecordPtr &Elem) {
switch (Elem->getKind()) {
case GOFFYAML::RecordBase::RBK_ModuleHeader:
case GOFFYAML::RecordBase::Kind::ModuleHeader:
IO.mapRequired("ModuleHeader",
*static_cast<GOFFYAML::ModuleHeader *>(Elem.get()));
break;
case GOFFYAML::RecordBase::RBK_EndOfModule:
case GOFFYAML::RecordBase::Kind::EndOfModule:
IO.mapRequired("End", *static_cast<GOFFYAML::EndOfModule *>(Elem.get()));
break;
case GOFFYAML::RecordBase::RBK_RelocationDirectory:
case GOFFYAML::RecordBase::RBK_Symbol:
case GOFFYAML::RecordBase::RBK_Text:
case GOFFYAML::RecordBase::RBK_DeferredLength:
llvm_unreachable(("Not yet implemented"));
case GOFFYAML::RecordBase::Kind::RelocationDirectory:
case GOFFYAML::RecordBase::Kind::Symbol:
case GOFFYAML::RecordBase::Kind::Text:
case GOFFYAML::RecordBase::Kind::DeferredLength:
llvm_unreachable("not yet implemented");
}
}

Expand Down
7 changes: 7 additions & 0 deletions llvm/test/tools/yaml2obj/GOFF/end-invalid-name.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# RUN: not yaml2obj %s 2>&1 | FileCheck %s

# CHECK: yaml2obj: error: conversion to EBCDIC 1047 failed on 'Euro symbol €'

--- !GOFF
- End:
EntryName: "Euro symbol €"
6 changes: 3 additions & 3 deletions llvm/test/tools/yaml2obj/GOFF/end-long-name.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
# CHECK1-EMPTY:

## 1 record fully used.
## Flags = 0 => no continuation, not continued.
## Flags = 0 => not a continuation, not continued.
# CHECK2: 03 40 00 00 00 00 00 00 00 00 00 00 00 00 00 00
# CHECK2-NEXT: 00 00 00 00 00 00 00 00 00 00 c6 96 96 c2 81 99
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Name Length field is zero. From the documentation: "This length cannot be zero if a name is present."

Either we're writing invalid GOFF or the documentation needs an update.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or perhaps a comment in the test that says we're purposely ignoring the name length field for this test.

# CHECK2-NEXT: 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40
Expand All @@ -34,7 +34,7 @@
# CHECK2-EMPTY:

## 2 records fully used.
## Flags = 1 => no continuation, but continued.
## Flags = 1 => not a continuation, but continued.
# CHECK3: 03 41 00 00 00 00 00 00 00 00 00 00 00 00 00 00
# CHECK3-NEXT: 00 00 00 00 00 00 00 00 00 00 c6 96 96 c2 81 99
# CHECK3-NEXT: 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40
Expand All @@ -49,7 +49,7 @@
# CHECK3-EMPTY:

## 3rd record used half.
## Flags = 1 => no continuation, but continued.
## Flags = 1 => not a continuation, but continued.
# CHECK4: 03 41 00 00 00 00 00 00 00 00 00 00 00 00 00 00
# CHECK4-NEXT: 00 00 00 00 00 00 00 00 00 00 c6 96 96 c2 81 99
# CHECK4-NEXT: 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40
Expand Down
2 changes: 1 addition & 1 deletion llvm/test/tools/yaml2obj/GOFF/invalid-record.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# RUN: not yaml2obj %s 2>&1 | FileCheck %s

# CHECK: YAML:{{[0-9]*}}:{{[0-9]*}}: error: Unknown record type name Foo
# CHECK: YAML:{{[0-9]*}}:{{[0-9]*}}: error: unknown record type name Foo
# CHECk: yaml2obj: error: failed to parse YAML input: Invalid argument

--- !GOFF
Expand Down
Loading