Skip to content

[GOFF] Add writing text records to yaml2obj #93863

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

redstar
Copy link
Member

@redstar redstar commented May 30, 2024

  • [GOFF] Refactor writing GOFF records
  • [GOFF] Add writing text records to yaml2obj

A GOFF file is basically a sequence of records. Change the YAML mapping for GOFF files to reflect this. As result, creating files for testing, e.g. without a module header, becomes possible.
@llvmbot
Copy link
Member

llvmbot commented May 30, 2024

@llvm/pr-subscribers-backend-systemz

@llvm/pr-subscribers-llvm-binary-utilities

Author: Kai Nacke (redstar)

Changes
  • [GOFF] Refactor writing GOFF records
  • [GOFF] Add writing text records to yaml2obj

Patch is 34.29 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/93863.diff

15 Files Affected:

  • (modified) llvm/include/llvm/BinaryFormat/GOFF.h (+6)
  • (modified) llvm/include/llvm/ObjectYAML/GOFFYAML.h (+91-12)
  • (modified) llvm/lib/ObjectYAML/GOFFEmitter.cpp (+132-153)
  • (modified) llvm/lib/ObjectYAML/GOFFYAML.cpp (+102-18)
  • (removed) llvm/test/tools/yaml2obj/GOFF/GOFF-header-settings.yaml (-26)
  • (removed) llvm/test/tools/yaml2obj/GOFF/GOFF-no-header.yaml (-7)
  • (added) llvm/test/tools/yaml2obj/GOFF/end-amode-const.yaml (+15)
  • (added) llvm/test/tools/yaml2obj/GOFF/end-long-name.yaml (+70)
  • (added) llvm/test/tools/yaml2obj/GOFF/end-only.yaml (+22)
  • (renamed) llvm/test/tools/yaml2obj/GOFF/header-end.yaml (+8-6)
  • (added) llvm/test/tools/yaml2obj/GOFF/header-only.yaml (+18)
  • (added) llvm/test/tools/yaml2obj/GOFF/invalid-record.yaml (+7)
  • (added) llvm/test/tools/yaml2obj/GOFF/text-default.yaml (+15)
  • (added) llvm/test/tools/yaml2obj/GOFF/text-only.yaml (+22)
  • (added) llvm/test/tools/yaml2obj/GOFF/text-style-const.yaml (+15)
diff --git a/llvm/include/llvm/BinaryFormat/GOFF.h b/llvm/include/llvm/BinaryFormat/GOFF.h
index 443bcfc9479a8..9a5637d0abe5f 100644
--- a/llvm/include/llvm/BinaryFormat/GOFF.h
+++ b/llvm/include/llvm/BinaryFormat/GOFF.h
@@ -157,6 +157,12 @@ enum ESDAlignment : uint8_t {
   ESD_ALIGN_4Kpage = 12,
 };
 
+enum TXTRecordStyle : uint8_t {
+  TXT_TS_Byte = 0,
+  TXT_TS_Structured = 1,
+  TXT_TS_Unstructured = 2,
+};
+
 enum ENDEntryPointRequest : uint8_t {
   END_EPR_None = 0,
   END_EPR_EsdidOffset = 1,
diff --git a/llvm/include/llvm/ObjectYAML/GOFFYAML.h b/llvm/include/llvm/ObjectYAML/GOFFYAML.h
index f9bf45e95bd3a..1308422b9d3fa 100644
--- a/llvm/include/llvm/ObjectYAML/GOFFYAML.h
+++ b/llvm/include/llvm/ObjectYAML/GOFFYAML.h
@@ -25,25 +25,104 @@ namespace llvm {
 // to use yaml::IO, we use these structures which are closer to the source.
 namespace GOFFYAML {
 
-struct FileHeader {
-  uint32_t TargetEnvironment = 0;
-  uint32_t TargetOperatingSystem = 0;
-  uint16_t CCSID = 0;
-  StringRef CharacterSetName;
-  StringRef LanguageProductIdentifier;
-  uint32_t ArchitectureLevel = 0;
-  std::optional<uint16_t> InternalCCSID;
-  std::optional<uint8_t> TargetSoftwareEnvironment;
+LLVM_YAML_STRONG_TYPEDEF(uint8_t, GOFF_AMODE)
+LLVM_YAML_STRONG_TYPEDEF(uint8_t, GOFF_TXTRECORDSTYLE)
+LLVM_YAML_STRONG_TYPEDEF(uint8_t, GOFF_ENDFLAGS)
+
+// The GOFF format uses different kinds of logical records. The format imposes
+// some rules on those records (e.g. the module header must come first, no
+// forward references to records, etc.). However, to be able to specify invalid
+// GOFF files, we treat all records the same way.
+struct RecordBase {
+  enum RecordBaseKind {
+    RBK_ModuleHeader,
+    RBK_RelocationDirectory,
+    RBK_Symbol,
+    RBK_Text,
+    RBK_DeferredLength,
+    RBK_EndOfModule
+  };
+
+private:
+  const RecordBaseKind Kind;
+
+protected:
+  RecordBase(RecordBaseKind Kind) : Kind(Kind) {}
+
+public:
+  RecordBaseKind getKind() const { return Kind; }
+};
+using RecordPtr = std::unique_ptr<RecordBase>;
+
+struct ModuleHeader : public RecordBase {
+  ModuleHeader() : RecordBase(RBK_ModuleHeader) {}
+
+  uint32_t ArchitectureLevel;
+  uint16_t PropertiesLength;
+  std::optional<yaml::BinaryRef> Properties;
+
+  static bool classof(const RecordBase *S) {
+    return S->getKind() == RBK_ModuleHeader;
+  }
+};
+
+struct Text : public RecordBase {
+  Text() : RecordBase(RBK_Text) {}
+
+  GOFF_TXTRECORDSTYLE Style;
+  uint32_t ESDID;
+  uint32_t Offset;
+  uint32_t TrueLength;
+  uint16_t Encoding;
+  uint16_t DataLength;
+  std::optional<yaml::BinaryRef> Data;
+
+  static bool classof(const RecordBase *S) { return S->getKind() == RBK_Text; }
+};
+
+struct EndOfModule : public RecordBase {
+  EndOfModule() : RecordBase(RBK_EndOfModule) {}
+
+  GOFF_ENDFLAGS Flags;
+  GOFF_AMODE AMODE;
+  uint32_t RecordCount;
+  uint32_t ESDID;
+  uint32_t Offset;
+  uint16_t NameLength;
+  StringRef EntryName;
+
+  static bool classof(const RecordBase *S) {
+    return S->getKind() == RBK_EndOfModule;
+  }
 };
 
 struct Object {
-  FileHeader Header;
-  Object();
+  // A GOFF file is a sequence of records.
+  std::vector<RecordPtr> Records;
 };
 } // end namespace GOFFYAML
 } // end namespace llvm
 
-LLVM_YAML_DECLARE_MAPPING_TRAITS(GOFFYAML::FileHeader)
+LLVM_YAML_DECLARE_ENUM_TRAITS(GOFFYAML::GOFF_AMODE)
+LLVM_YAML_DECLARE_ENUM_TRAITS(GOFFYAML::GOFF_TXTRECORDSTYLE)
+LLVM_YAML_DECLARE_ENUM_TRAITS(GOFFYAML::GOFF_ENDFLAGS)
+
+LLVM_YAML_IS_SEQUENCE_VECTOR(GOFFYAML::RecordPtr)
+LLVM_YAML_DECLARE_MAPPING_TRAITS(GOFFYAML::RecordPtr)
+
+LLVM_YAML_DECLARE_MAPPING_TRAITS(GOFFYAML::ModuleHeader)
+LLVM_YAML_DECLARE_MAPPING_TRAITS(GOFFYAML::Text)
+LLVM_YAML_DECLARE_MAPPING_TRAITS(GOFFYAML::EndOfModule)
 LLVM_YAML_DECLARE_MAPPING_TRAITS(GOFFYAML::Object)
 
+namespace llvm {
+namespace yaml {
+
+template <> struct CustomMappingTraits<GOFFYAML::RecordPtr> {
+  static void inputOne(IO &IO, StringRef Key, GOFFYAML::RecordPtr &Elem);
+  static void output(IO &IO, GOFFYAML::RecordPtr &Elem);
+};
+
+} // namespace yaml
+} // namespace llvm
 #endif // LLVM_OBJECTYAML_GOFFYAML_H
diff --git a/llvm/lib/ObjectYAML/GOFFEmitter.cpp b/llvm/lib/ObjectYAML/GOFFEmitter.cpp
index 345904407e1d2..e8fa37dbbd1d8 100644
--- a/llvm/lib/ObjectYAML/GOFFEmitter.cpp
+++ b/llvm/lib/ObjectYAML/GOFFEmitter.cpp
@@ -11,11 +11,12 @@
 ///
 //===----------------------------------------------------------------------===//
 
-#include "llvm/ADT/IndexedMap.h"
-#include "llvm/ObjectYAML/ObjectYAML.h"
+#include "llvm/BinaryFormat/GOFF.h"
+#include "llvm/ObjectYAML/GOFFYAML.h"
 #include "llvm/ObjectYAML/yaml2obj.h"
 #include "llvm/Support/ConvertEBCDIC.h"
 #include "llvm/Support/Endian.h"
+#include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/raw_ostream.h"
 
 using namespace llvm;
@@ -25,10 +26,10 @@ namespace {
 // Common flag values on records.
 enum {
   // Flag: This record is continued.
-  Rec_Continued = 1,
+  Rec_Continued = 1 << 0,
 
   // Flag: This record is a continuation.
-  Rec_Continuation = 1 << (8 - 6 - 1),
+  Rec_Continuation = 1 << 1,
 };
 
 template <typename ValueType> struct BinaryBeImpl {
@@ -62,119 +63,91 @@ ZerosImpl zeros(const size_t NumBytes) { return ZerosImpl{NumBytes}; }
 
 // The GOFFOstream is responsible to write the data into the fixed physical
 // records of the format. A user of this class announces the start of a new
-// logical record and the size of its payload. While writing the payload, the
-// physical records are created for the data. Possible fill bytes at the end of
-// a physical record are written automatically.
-class GOFFOstream : public raw_ostream {
+// logical record, and writes the full logical block. The physical records are
+// created while the content is written to the underlying stream. Possible fill
+// bytes at the end of a physical record are written automatically.
+// The implementation aims at simplicity, not speed.
+class GOFFOStream {
 public:
-  explicit GOFFOstream(raw_ostream &OS)
-      : OS(OS), LogicalRecords(0), RemainingSize(0), NewLogicalRecord(false) {
-    SetBufferSize(GOFF::PayloadLength);
-  }
-
-  ~GOFFOstream() { finalize(); }
+  explicit GOFFOStream(raw_ostream &OS)
+      : OS(OS), CurrentType(GOFF::RecordType(-1)) {}
 
-  void makeNewRecord(GOFF::RecordType Type, size_t Size) {
-    fillRecord();
-    CurrentType = Type;
-    RemainingSize = Size;
-    if (size_t Gap = (RemainingSize % GOFF::PayloadLength))
-      RemainingSize += GOFF::PayloadLength - Gap;
-    NewLogicalRecord = true;
-    ++LogicalRecords;
+  GOFFOStream &operator<<(StringRef Str) {
+    write(Str);
+    return *this;
   }
 
-  void finalize() { fillRecord(); }
-
-  uint32_t logicalRecords() { return LogicalRecords; }
+  void newRecord(GOFF::RecordType Type) { CurrentType = Type; }
 
 private:
   // The underlying raw_ostream.
   raw_ostream &OS;
 
-  // The number of logical records emitted so far.
-  uint32_t LogicalRecords;
-
-  // The remaining size of this logical record, including fill bytes.
-  size_t RemainingSize;
-
   // The type of the current (logical) record.
   GOFF::RecordType CurrentType;
 
-  // Signals start of new record.
-  bool NewLogicalRecord;
-
-  // Return the number of bytes left to write until next physical record.
-  // Please note that we maintain the total number of bytes left, not the
-  // written size.
-  size_t bytesToNextPhysicalRecord() {
-    size_t Bytes = RemainingSize % GOFF::PayloadLength;
-    return Bytes ? Bytes : GOFF::PayloadLength;
-  }
-
   // Write the record prefix of a physical record, using the current record
   // type.
-  static void writeRecordPrefix(raw_ostream &OS, GOFF::RecordType Type,
-                                size_t RemainingSize,
-                                uint8_t Flags = Rec_Continuation) {
-    uint8_t TypeAndFlags = Flags | (Type << 4);
-    if (RemainingSize > GOFF::RecordLength)
-      TypeAndFlags |= Rec_Continued;
-    OS << binaryBe(static_cast<unsigned char>(GOFF::PTVPrefix))
-       << binaryBe(static_cast<unsigned char>(TypeAndFlags))
-       << binaryBe(static_cast<unsigned char>(0));
-  }
+  void writeRecordPrefix(uint8_t Flags);
 
-  // Fill the last physical record of a logical record with zero bytes.
-  void fillRecord() {
-    assert((GetNumBytesInBuffer() <= RemainingSize) &&
-           "More bytes in buffer than expected");
-    size_t Remains = RemainingSize - GetNumBytesInBuffer();
-    if (Remains) {
-      assert((Remains < GOFF::RecordLength) &&
-             "Attempting to fill more than one physical record");
-      raw_ostream::write_zeros(Remains);
-    }
-    flush();
-    assert(RemainingSize == 0 && "Not fully flushed");
-    assert(GetNumBytesInBuffer() == 0 && "Buffer not fully empty");
-  }
+  // Write a logical record.
+  void write(StringRef Str);
+};
 
-  // See raw_ostream::write_impl.
-  void write_impl(const char *Ptr, size_t Size) override {
-    assert((RemainingSize >= Size) && "Attempt to write too much data");
-    assert(RemainingSize && "Logical record overflow");
-    if (!(RemainingSize % GOFF::PayloadLength)) {
-      writeRecordPrefix(OS, CurrentType, RemainingSize,
-                        NewLogicalRecord ? 0 : Rec_Continuation);
-      NewLogicalRecord = false;
-    }
-    assert(!NewLogicalRecord &&
-           "New logical record not on physical record boundary");
-
-    size_t Idx = 0;
-    while (Size > 0) {
-      size_t BytesToWrite = bytesToNextPhysicalRecord();
-      if (BytesToWrite > Size)
-        BytesToWrite = Size;
-      OS.write(Ptr + Idx, BytesToWrite);
-      Idx += BytesToWrite;
-      Size -= BytesToWrite;
-      RemainingSize -= BytesToWrite;
-      if (Size) {
-        writeRecordPrefix(OS, CurrentType, RemainingSize);
-      }
-    }
+void GOFFOStream::writeRecordPrefix(uint8_t Flags) {
+  uint8_t TypeAndFlags = Flags | (CurrentType << 4);
+  OS << binaryBe(static_cast<unsigned char>(GOFF::PTVPrefix))
+     << binaryBe(static_cast<unsigned char>(TypeAndFlags))
+     << binaryBe(static_cast<unsigned char>(0));
+}
+
+void GOFFOStream::write(StringRef Str) {
+  // The flags are determined by the flags of the prvious record, and by the
+  // remaining size of data.
+  uint8_t Flags = 0;
+  size_t Ptr = 0;
+  size_t Size = Str.size();
+  while (Size >= GOFF::RecordContentLength) {
+    if (Flags) {
+      Flags |= Rec_Continuation;
+      if (Size == GOFF::RecordContentLength)
+        Flags &= ~Rec_Continued;
+    } else
+      Flags |= (Size == GOFF::RecordContentLength) ? 0 : Rec_Continued;
+    writeRecordPrefix(Flags);
+    OS.write(&Str.data()[Ptr], GOFF::RecordContentLength);
+    Size -= GOFF::RecordContentLength;
+    Ptr += GOFF::RecordContentLength;
   }
+  if (Size) {
+    Flags &= ~Rec_Continued;
+    writeRecordPrefix(Flags);
+    OS.write(&Str.data()[Ptr], Size);
+    OS.write_zeros(GOFF::RecordContentLength - Size);
+  }
+}
+
+// A LogicalRecord buffers the data of a record.
+class LogicalRecord : public raw_svector_ostream {
+  GOFFOStream &OS;
+  SmallVector<char, 0> Buffer;
+
+  void anchor() override {};
 
-  // Return the current position within the stream, not counting the bytes
-  // currently in the buffer.
-  uint64_t current_pos() const override { return OS.tell(); }
+public:
+  LogicalRecord(GOFFOStream &OS) : raw_svector_ostream(Buffer), OS(OS) {}
+  ~LogicalRecord() override { OS << str(); }
+
+  LogicalRecord &operator<<(yaml::BinaryRef B) {
+    B.writeAsBinary(*this);
+    return *this;
+  }
 };
 
 class GOFFState {
-  void writeHeader(GOFFYAML::FileHeader &FileHdr);
-  void writeEnd();
+  void writeHeader(GOFFYAML::ModuleHeader &ModHdr);
+  void writeText(GOFFYAML::Text &Txt);
+  void writeEnd(GOFFYAML::EndOfModule &EndMod);
 
   void reportError(const Twine &Msg) {
     ErrHandler(Msg);
@@ -185,8 +158,6 @@ class GOFFState {
             yaml::ErrorHandler ErrHandler)
       : GW(OS), Doc(Doc), ErrHandler(ErrHandler), HasError(false) {}
 
-  ~GOFFState() { GW.finalize(); }
-
   bool writeObject();
 
 public:
@@ -194,72 +165,80 @@ class GOFFState {
                         yaml::ErrorHandler ErrHandler);
 
 private:
-  GOFFOstream GW;
+  GOFFOStream GW;
   GOFFYAML::Object &Doc;
   yaml::ErrorHandler ErrHandler;
   bool HasError;
 };
 
-void GOFFState::writeHeader(GOFFYAML::FileHeader &FileHdr) {
-  SmallString<16> CCSIDName;
-  if (std::error_code EC =
-          ConverterEBCDIC::convertToEBCDIC(FileHdr.CharacterSetName, CCSIDName))
-    reportError("Conversion error on " + FileHdr.CharacterSetName);
-  if (CCSIDName.size() > 16) {
-    reportError("CharacterSetName too long");
-    CCSIDName.resize(16);
-  }
-  SmallString<16> LangProd;
-  if (std::error_code EC = ConverterEBCDIC::convertToEBCDIC(
-          FileHdr.LanguageProductIdentifier, LangProd))
-    reportError("Conversion error on " + FileHdr.LanguageProductIdentifier);
-  if (LangProd.size() > 16) {
-    reportError("LanguageProductIdentifier too long");
-    LangProd.resize(16);
-  }
+void GOFFState::writeHeader(GOFFYAML::ModuleHeader &ModHdr) {
+  GW.newRecord(GOFF::RT_HDR);
+  LogicalRecord LR(GW);
+  LR << zeros(45)                          // Reserved.
+     << binaryBe(ModHdr.ArchitectureLevel) // The architecture level.
+     << binaryBe(ModHdr.PropertiesLength)  // Length of module properties.
+     << zeros(6);                          // Reserved.
+  if (ModHdr.Properties)
+    LR << *ModHdr.Properties; // Module properties.
+}
 
-  GW.makeNewRecord(GOFF::RT_HDR, GOFF::PayloadLength);
-  GW << binaryBe(FileHdr.TargetEnvironment)     // TargetEnvironment
-     << binaryBe(FileHdr.TargetOperatingSystem) // TargetOperatingSystem
-     << zeros(2)                                // Reserved
-     << binaryBe(FileHdr.CCSID)                 // CCSID
-     << CCSIDName                               // CharacterSetName
-     << zeros(16 - CCSIDName.size())            // Fill bytes
-     << LangProd                                // LanguageProductIdentifier
-     << zeros(16 - LangProd.size())             // Fill bytes
-     << binaryBe(FileHdr.ArchitectureLevel);    // ArchitectureLevel
-  // The module propties are optional. Figure out if we need to write them.
-  uint16_t ModPropLen = 0;
-  if (FileHdr.TargetSoftwareEnvironment)
-    ModPropLen = 3;
-  else if (FileHdr.InternalCCSID)
-    ModPropLen = 2;
-  if (ModPropLen) {
-    GW << binaryBe(ModPropLen) << zeros(6);
-    if (ModPropLen >= 2)
-      GW << binaryBe(FileHdr.InternalCCSID ? *FileHdr.InternalCCSID : 0);
-    if (ModPropLen >= 3)
-      GW << binaryBe(FileHdr.TargetSoftwareEnvironment
-                         ? *FileHdr.TargetSoftwareEnvironment
-                         : 0);
-  }
+void GOFFState::writeText(GOFFYAML::Text &Txt) {
+  GW.newRecord(GOFF::RT_TXT);
+  LogicalRecord LR(GW);
+  LR << binaryBe(uint8_t(Txt.Style)) // Text record style.
+     << binaryBe(
+            Txt.ESDID) // ESDID of the element/part to which this data belongs.
+     << zeros(4)       // Reserved.
+     << binaryBe(Txt.Offset)      // Starting offset from element/part.
+     << binaryBe(Txt.TrueLength)  // True length if encoded.
+     << binaryBe(Txt.Encoding)    // Encoding.
+     << binaryBe(Txt.DataLength); // Total length of data.
+  if (Txt.Data)
+    LR << *Txt.Data; // Data.
+  else
+    LR << zeros(Txt.DataLength);
 }
 
-void GOFFState::writeEnd() {
-  GW.makeNewRecord(GOFF::RT_END, GOFF::PayloadLength);
-  GW << binaryBe(uint8_t(0)) // No entry point
-     << binaryBe(uint8_t(0)) // No AMODE
-     << zeros(3)             // Reserved
-     << binaryBe(GW.logicalRecords());
-  // No entry point yet. Automatically fill remaining space with zero bytes.
-  GW.finalize();
+void GOFFState::writeEnd(GOFFYAML::EndOfModule &EndMod) {
+  SmallString<16> EntryName;
+  if (std::error_code EC =
+          ConverterEBCDIC::convertToEBCDIC(EndMod.EntryName, EntryName))
+    reportError("Conversion error on " + EndMod.EntryName);
+
+  GW.newRecord(GOFF::RT_END);
+  LogicalRecord LR(GW);
+  LR << binaryBe(uint8_t(EndMod.Flags)) // The flags.
+     << binaryBe(uint8_t(EndMod.AMODE)) // The addressing mode.
+     << zeros(3)                        // Reserved.
+     << binaryBe(EndMod.RecordCount)    // The record count.
+     << binaryBe(EndMod.ESDID)          // ESDID of the entry point.
+     << zeros(4)                        // Reserved.
+     << binaryBe(EndMod.Offset)         // Offset of entry point.
+     << binaryBe(EndMod.NameLength)     // Length of external name.
+     << EntryName;                      // Name of the entry point.
 }
 
 bool GOFFState::writeObject() {
-  writeHeader(Doc.Header);
-  if (HasError)
-    return false;
-  writeEnd();
+  for (auto &RecPtr : Doc.Records) {
+    auto *Rec = RecPtr.get();
+    switch (Rec->getKind()) {
+    case GOFFYAML::RecordBase::RBK_ModuleHeader:
+      writeHeader(*static_cast<GOFFYAML::ModuleHeader *>(Rec));
+      break;
+    case GOFFYAML::RecordBase::RBK_Text:
+      writeText(*static_cast<GOFFYAML::Text *>(Rec));
+      break;
+    case GOFFYAML::RecordBase::RBK_EndOfModule:
+      writeEnd(*static_cast<GOFFYAML::EndOfModule *>(Rec));
+      break;
+    case GOFFYAML::RecordBase::RBK_RelocationDirectory:
+    case GOFFYAML::RecordBase::RBK_Symbol:
+    case GOFFYAML::RecordBase::RBK_DeferredLength:
+      llvm_unreachable(("Not yet implemented"));
+    }
+    if (HasError)
+      return false;
+  }
   return true;
 }
 
diff --git a/llvm/lib/ObjectYAML/GOFFYAML.cpp b/llvm/lib/ObjectYAML/GOFFYAML.cpp
index ae857980a521b..d3c8b02bcb6a3 100644
--- a/llvm/lib/ObjectYAML/GOFFYAML.cpp
+++ b/llvm/lib/ObjectYAML/GOFFYAML.cpp
@@ -12,34 +12,118 @@
 
 #include "llvm/ObjectYAML/GOFFYAML.h"
 #include "llvm/BinaryFormat/GOFF.h"
-#include <string.h>
 
 namespace llvm {
-namespace GOFFYAML {
 
-Object::Object() {}
+namespace yaml {
 
-} // namespace GOFFYAML
+void ScalarEnumerationTraits<GOFFYAML::GOFF_AMODE>::enumeration(
+    IO &IO, GOFFYAML::GOFF_AMODE &Value) {
+#define ECase(X) IO.enumCase(Value, #X, GOFF::ESD_##X)
+  ECase(AMODE_None);
+  ECase(AMODE_24);
+  ECase(AMODE_31);
+  ECase(AMODE_ANY);
+  ECase(AMODE_64);
+  ECase(AMODE_MIN);
+#undef ECase
+  IO.enumFallback<Hex8>(Value);
+}
 
-namespace yaml {
+void ScalarEnumerationTraits<GOFFYAML::GOFF_TXTRECORDSTYLE>::enumeration(
+    IO &IO, GOFFYAML::GOFF_TXTRECORDSTYLE &Value) {
+#define ECase(X) IO.enumCase(Value, #X, GOFF::TXT_##X)
+  ECase(TS_Byte);
+  ECase(TS_Structured);
+  ECase(TS_Unstructured);
+#undef ECase
+  IO.enumFallback<Hex8>(Value);
+}
+
+void ScalarEnumerationTraits<GOFFYAML::GOFF_ENDFLAGS>::enumeration(
+    IO &IO, GOFFYAML::GOFF_ENDFLAGS &Value) {
+#define ECase(X) IO.enumCase(Value, #X, unsigned(GOFF::END_##X) << 6)
+  ECase(EPR_None);
+  ECase(EPR_EsdidOffset);
+  ECase(EPR_ExternalName);
+  ECase(EPR_Reserved);
+#undef ECase
+  IO.enumFallback<Hex8>(Value);
+}
+
+void MappingTraits<GOFFYAML::ModuleHeader>::mapping(
+    IO &IO, GOFFYAML::ModuleHeader &ModHdr) {
+  IO.mapOptional("ArchitectureLevel", ModHdr.ArchitectureLevel, 0);
+  IO.mapOptional("PropertiesLength", ModHdr.PropertiesLength, 0);
+  IO.mapOptional("Properties", ModHdr.Properties);
+}
+
+void MappingTraits<GOFFYAML::Text>::mapping(IO &IO, GOFFYAML::Text &Txt) {
+  IO.mapOptional("TextStyle", Txt.Style, GOFF::TXT_TS_Byte);
+  IO.mapOptional("ESDID", Txt.ESDID, 0);
+  IO.mapOptional("Offset", Txt.Offset, 0);
+  IO.mapOptional("TrueLength", Txt.TrueLength, 0);
+  IO.mapOptional("TextEncoding", Txt.Encoding, 0);
+  IO.mapOptional("DataLength", Txt.DataLength, 0);
+  IO.mapOptional("Data", Txt.Data);
+}
+
+void MappingTraits<GOFFYAML::EndOfModule>::mapping(IO &IO,
+                                                   GOFFYAML::EndOfModule &End) {
+  IO.mapOptional("Flags", End.Flags, 0);
+  IO.mapOptional("AMODE", End.AMODE, 0);
+  IO.mapOptional("RecordCount", End.RecordCount, 0);
+  IO.mapOptional("ESDID", End.ESDID, 0);
+  IO.mapOptional("Offset", End.Offset, 0);
+  IO.mapOptional("NameLength", End.NameLength, 0);
+  IO.mapOptional("EntryName", End.EntryName);
+}
+
+void CustomMappingTraits<GOFFYAML::RecordPtr>::inputOne(
+    IO &IO, StringRef Key, GOFFYAML::RecordPtr &Elem) {
+  if (Key == "ModuleHeader") {
+    GOFFYAML::ModuleHeader ModHdr;
+    IO.mapRequired("ModuleHeader", ModHdr);
+    Elem = std::make_unique<GOFFYAML::ModuleHeader>(std::move(ModHdr));
+  } else if (Key == "Text") {
+    GOFFYAML::Text Txt;
+    IO.mapRequired("Text", Txt);
+    Elem = std::make_unique<GOFFYAML::Text>(std::move(Txt));
+  } else if (Key ...
[truncated]

@llvmbot
Copy link
Member

llvmbot commented May 30, 2024

@llvm/pr-subscribers-objectyaml

Author: Kai Nacke (redstar)

Changes
  • [GOFF] Refactor writing GOFF records
  • [GOFF] Add writing text records to yaml2obj

Patch is 34.29 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/93863.diff

15 Files Affected:

  • (modified) llvm/include/llvm/BinaryFormat/GOFF.h (+6)
  • (modified) llvm/include/llvm/ObjectYAML/GOFFYAML.h (+91-12)
  • (modified) llvm/lib/ObjectYAML/GOFFEmitter.cpp (+132-153)
  • (modified) llvm/lib/ObjectYAML/GOFFYAML.cpp (+102-18)
  • (removed) llvm/test/tools/yaml2obj/GOFF/GOFF-header-settings.yaml (-26)
  • (removed) llvm/test/tools/yaml2obj/GOFF/GOFF-no-header.yaml (-7)
  • (added) llvm/test/tools/yaml2obj/GOFF/end-amode-const.yaml (+15)
  • (added) llvm/test/tools/yaml2obj/GOFF/end-long-name.yaml (+70)
  • (added) llvm/test/tools/yaml2obj/GOFF/end-only.yaml (+22)
  • (renamed) llvm/test/tools/yaml2obj/GOFF/header-end.yaml (+8-6)
  • (added) llvm/test/tools/yaml2obj/GOFF/header-only.yaml (+18)
  • (added) llvm/test/tools/yaml2obj/GOFF/invalid-record.yaml (+7)
  • (added) llvm/test/tools/yaml2obj/GOFF/text-default.yaml (+15)
  • (added) llvm/test/tools/yaml2obj/GOFF/text-only.yaml (+22)
  • (added) llvm/test/tools/yaml2obj/GOFF/text-style-const.yaml (+15)
diff --git a/llvm/include/llvm/BinaryFormat/GOFF.h b/llvm/include/llvm/BinaryFormat/GOFF.h
index 443bcfc9479a8..9a5637d0abe5f 100644
--- a/llvm/include/llvm/BinaryFormat/GOFF.h
+++ b/llvm/include/llvm/BinaryFormat/GOFF.h
@@ -157,6 +157,12 @@ enum ESDAlignment : uint8_t {
   ESD_ALIGN_4Kpage = 12,
 };
 
+enum TXTRecordStyle : uint8_t {
+  TXT_TS_Byte = 0,
+  TXT_TS_Structured = 1,
+  TXT_TS_Unstructured = 2,
+};
+
 enum ENDEntryPointRequest : uint8_t {
   END_EPR_None = 0,
   END_EPR_EsdidOffset = 1,
diff --git a/llvm/include/llvm/ObjectYAML/GOFFYAML.h b/llvm/include/llvm/ObjectYAML/GOFFYAML.h
index f9bf45e95bd3a..1308422b9d3fa 100644
--- a/llvm/include/llvm/ObjectYAML/GOFFYAML.h
+++ b/llvm/include/llvm/ObjectYAML/GOFFYAML.h
@@ -25,25 +25,104 @@ namespace llvm {
 // to use yaml::IO, we use these structures which are closer to the source.
 namespace GOFFYAML {
 
-struct FileHeader {
-  uint32_t TargetEnvironment = 0;
-  uint32_t TargetOperatingSystem = 0;
-  uint16_t CCSID = 0;
-  StringRef CharacterSetName;
-  StringRef LanguageProductIdentifier;
-  uint32_t ArchitectureLevel = 0;
-  std::optional<uint16_t> InternalCCSID;
-  std::optional<uint8_t> TargetSoftwareEnvironment;
+LLVM_YAML_STRONG_TYPEDEF(uint8_t, GOFF_AMODE)
+LLVM_YAML_STRONG_TYPEDEF(uint8_t, GOFF_TXTRECORDSTYLE)
+LLVM_YAML_STRONG_TYPEDEF(uint8_t, GOFF_ENDFLAGS)
+
+// The GOFF format uses different kinds of logical records. The format imposes
+// some rules on those records (e.g. the module header must come first, no
+// forward references to records, etc.). However, to be able to specify invalid
+// GOFF files, we treat all records the same way.
+struct RecordBase {
+  enum RecordBaseKind {
+    RBK_ModuleHeader,
+    RBK_RelocationDirectory,
+    RBK_Symbol,
+    RBK_Text,
+    RBK_DeferredLength,
+    RBK_EndOfModule
+  };
+
+private:
+  const RecordBaseKind Kind;
+
+protected:
+  RecordBase(RecordBaseKind Kind) : Kind(Kind) {}
+
+public:
+  RecordBaseKind getKind() const { return Kind; }
+};
+using RecordPtr = std::unique_ptr<RecordBase>;
+
+struct ModuleHeader : public RecordBase {
+  ModuleHeader() : RecordBase(RBK_ModuleHeader) {}
+
+  uint32_t ArchitectureLevel;
+  uint16_t PropertiesLength;
+  std::optional<yaml::BinaryRef> Properties;
+
+  static bool classof(const RecordBase *S) {
+    return S->getKind() == RBK_ModuleHeader;
+  }
+};
+
+struct Text : public RecordBase {
+  Text() : RecordBase(RBK_Text) {}
+
+  GOFF_TXTRECORDSTYLE Style;
+  uint32_t ESDID;
+  uint32_t Offset;
+  uint32_t TrueLength;
+  uint16_t Encoding;
+  uint16_t DataLength;
+  std::optional<yaml::BinaryRef> Data;
+
+  static bool classof(const RecordBase *S) { return S->getKind() == RBK_Text; }
+};
+
+struct EndOfModule : public RecordBase {
+  EndOfModule() : RecordBase(RBK_EndOfModule) {}
+
+  GOFF_ENDFLAGS Flags;
+  GOFF_AMODE AMODE;
+  uint32_t RecordCount;
+  uint32_t ESDID;
+  uint32_t Offset;
+  uint16_t NameLength;
+  StringRef EntryName;
+
+  static bool classof(const RecordBase *S) {
+    return S->getKind() == RBK_EndOfModule;
+  }
 };
 
 struct Object {
-  FileHeader Header;
-  Object();
+  // A GOFF file is a sequence of records.
+  std::vector<RecordPtr> Records;
 };
 } // end namespace GOFFYAML
 } // end namespace llvm
 
-LLVM_YAML_DECLARE_MAPPING_TRAITS(GOFFYAML::FileHeader)
+LLVM_YAML_DECLARE_ENUM_TRAITS(GOFFYAML::GOFF_AMODE)
+LLVM_YAML_DECLARE_ENUM_TRAITS(GOFFYAML::GOFF_TXTRECORDSTYLE)
+LLVM_YAML_DECLARE_ENUM_TRAITS(GOFFYAML::GOFF_ENDFLAGS)
+
+LLVM_YAML_IS_SEQUENCE_VECTOR(GOFFYAML::RecordPtr)
+LLVM_YAML_DECLARE_MAPPING_TRAITS(GOFFYAML::RecordPtr)
+
+LLVM_YAML_DECLARE_MAPPING_TRAITS(GOFFYAML::ModuleHeader)
+LLVM_YAML_DECLARE_MAPPING_TRAITS(GOFFYAML::Text)
+LLVM_YAML_DECLARE_MAPPING_TRAITS(GOFFYAML::EndOfModule)
 LLVM_YAML_DECLARE_MAPPING_TRAITS(GOFFYAML::Object)
 
+namespace llvm {
+namespace yaml {
+
+template <> struct CustomMappingTraits<GOFFYAML::RecordPtr> {
+  static void inputOne(IO &IO, StringRef Key, GOFFYAML::RecordPtr &Elem);
+  static void output(IO &IO, GOFFYAML::RecordPtr &Elem);
+};
+
+} // namespace yaml
+} // namespace llvm
 #endif // LLVM_OBJECTYAML_GOFFYAML_H
diff --git a/llvm/lib/ObjectYAML/GOFFEmitter.cpp b/llvm/lib/ObjectYAML/GOFFEmitter.cpp
index 345904407e1d2..e8fa37dbbd1d8 100644
--- a/llvm/lib/ObjectYAML/GOFFEmitter.cpp
+++ b/llvm/lib/ObjectYAML/GOFFEmitter.cpp
@@ -11,11 +11,12 @@
 ///
 //===----------------------------------------------------------------------===//
 
-#include "llvm/ADT/IndexedMap.h"
-#include "llvm/ObjectYAML/ObjectYAML.h"
+#include "llvm/BinaryFormat/GOFF.h"
+#include "llvm/ObjectYAML/GOFFYAML.h"
 #include "llvm/ObjectYAML/yaml2obj.h"
 #include "llvm/Support/ConvertEBCDIC.h"
 #include "llvm/Support/Endian.h"
+#include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/raw_ostream.h"
 
 using namespace llvm;
@@ -25,10 +26,10 @@ namespace {
 // Common flag values on records.
 enum {
   // Flag: This record is continued.
-  Rec_Continued = 1,
+  Rec_Continued = 1 << 0,
 
   // Flag: This record is a continuation.
-  Rec_Continuation = 1 << (8 - 6 - 1),
+  Rec_Continuation = 1 << 1,
 };
 
 template <typename ValueType> struct BinaryBeImpl {
@@ -62,119 +63,91 @@ ZerosImpl zeros(const size_t NumBytes) { return ZerosImpl{NumBytes}; }
 
 // The GOFFOstream is responsible to write the data into the fixed physical
 // records of the format. A user of this class announces the start of a new
-// logical record and the size of its payload. While writing the payload, the
-// physical records are created for the data. Possible fill bytes at the end of
-// a physical record are written automatically.
-class GOFFOstream : public raw_ostream {
+// logical record, and writes the full logical block. The physical records are
+// created while the content is written to the underlying stream. Possible fill
+// bytes at the end of a physical record are written automatically.
+// The implementation aims at simplicity, not speed.
+class GOFFOStream {
 public:
-  explicit GOFFOstream(raw_ostream &OS)
-      : OS(OS), LogicalRecords(0), RemainingSize(0), NewLogicalRecord(false) {
-    SetBufferSize(GOFF::PayloadLength);
-  }
-
-  ~GOFFOstream() { finalize(); }
+  explicit GOFFOStream(raw_ostream &OS)
+      : OS(OS), CurrentType(GOFF::RecordType(-1)) {}
 
-  void makeNewRecord(GOFF::RecordType Type, size_t Size) {
-    fillRecord();
-    CurrentType = Type;
-    RemainingSize = Size;
-    if (size_t Gap = (RemainingSize % GOFF::PayloadLength))
-      RemainingSize += GOFF::PayloadLength - Gap;
-    NewLogicalRecord = true;
-    ++LogicalRecords;
+  GOFFOStream &operator<<(StringRef Str) {
+    write(Str);
+    return *this;
   }
 
-  void finalize() { fillRecord(); }
-
-  uint32_t logicalRecords() { return LogicalRecords; }
+  void newRecord(GOFF::RecordType Type) { CurrentType = Type; }
 
 private:
   // The underlying raw_ostream.
   raw_ostream &OS;
 
-  // The number of logical records emitted so far.
-  uint32_t LogicalRecords;
-
-  // The remaining size of this logical record, including fill bytes.
-  size_t RemainingSize;
-
   // The type of the current (logical) record.
   GOFF::RecordType CurrentType;
 
-  // Signals start of new record.
-  bool NewLogicalRecord;
-
-  // Return the number of bytes left to write until next physical record.
-  // Please note that we maintain the total number of bytes left, not the
-  // written size.
-  size_t bytesToNextPhysicalRecord() {
-    size_t Bytes = RemainingSize % GOFF::PayloadLength;
-    return Bytes ? Bytes : GOFF::PayloadLength;
-  }
-
   // Write the record prefix of a physical record, using the current record
   // type.
-  static void writeRecordPrefix(raw_ostream &OS, GOFF::RecordType Type,
-                                size_t RemainingSize,
-                                uint8_t Flags = Rec_Continuation) {
-    uint8_t TypeAndFlags = Flags | (Type << 4);
-    if (RemainingSize > GOFF::RecordLength)
-      TypeAndFlags |= Rec_Continued;
-    OS << binaryBe(static_cast<unsigned char>(GOFF::PTVPrefix))
-       << binaryBe(static_cast<unsigned char>(TypeAndFlags))
-       << binaryBe(static_cast<unsigned char>(0));
-  }
+  void writeRecordPrefix(uint8_t Flags);
 
-  // Fill the last physical record of a logical record with zero bytes.
-  void fillRecord() {
-    assert((GetNumBytesInBuffer() <= RemainingSize) &&
-           "More bytes in buffer than expected");
-    size_t Remains = RemainingSize - GetNumBytesInBuffer();
-    if (Remains) {
-      assert((Remains < GOFF::RecordLength) &&
-             "Attempting to fill more than one physical record");
-      raw_ostream::write_zeros(Remains);
-    }
-    flush();
-    assert(RemainingSize == 0 && "Not fully flushed");
-    assert(GetNumBytesInBuffer() == 0 && "Buffer not fully empty");
-  }
+  // Write a logical record.
+  void write(StringRef Str);
+};
 
-  // See raw_ostream::write_impl.
-  void write_impl(const char *Ptr, size_t Size) override {
-    assert((RemainingSize >= Size) && "Attempt to write too much data");
-    assert(RemainingSize && "Logical record overflow");
-    if (!(RemainingSize % GOFF::PayloadLength)) {
-      writeRecordPrefix(OS, CurrentType, RemainingSize,
-                        NewLogicalRecord ? 0 : Rec_Continuation);
-      NewLogicalRecord = false;
-    }
-    assert(!NewLogicalRecord &&
-           "New logical record not on physical record boundary");
-
-    size_t Idx = 0;
-    while (Size > 0) {
-      size_t BytesToWrite = bytesToNextPhysicalRecord();
-      if (BytesToWrite > Size)
-        BytesToWrite = Size;
-      OS.write(Ptr + Idx, BytesToWrite);
-      Idx += BytesToWrite;
-      Size -= BytesToWrite;
-      RemainingSize -= BytesToWrite;
-      if (Size) {
-        writeRecordPrefix(OS, CurrentType, RemainingSize);
-      }
-    }
+void GOFFOStream::writeRecordPrefix(uint8_t Flags) {
+  uint8_t TypeAndFlags = Flags | (CurrentType << 4);
+  OS << binaryBe(static_cast<unsigned char>(GOFF::PTVPrefix))
+     << binaryBe(static_cast<unsigned char>(TypeAndFlags))
+     << binaryBe(static_cast<unsigned char>(0));
+}
+
+void GOFFOStream::write(StringRef Str) {
+  // The flags are determined by the flags of the prvious record, and by the
+  // remaining size of data.
+  uint8_t Flags = 0;
+  size_t Ptr = 0;
+  size_t Size = Str.size();
+  while (Size >= GOFF::RecordContentLength) {
+    if (Flags) {
+      Flags |= Rec_Continuation;
+      if (Size == GOFF::RecordContentLength)
+        Flags &= ~Rec_Continued;
+    } else
+      Flags |= (Size == GOFF::RecordContentLength) ? 0 : Rec_Continued;
+    writeRecordPrefix(Flags);
+    OS.write(&Str.data()[Ptr], GOFF::RecordContentLength);
+    Size -= GOFF::RecordContentLength;
+    Ptr += GOFF::RecordContentLength;
   }
+  if (Size) {
+    Flags &= ~Rec_Continued;
+    writeRecordPrefix(Flags);
+    OS.write(&Str.data()[Ptr], Size);
+    OS.write_zeros(GOFF::RecordContentLength - Size);
+  }
+}
+
+// A LogicalRecord buffers the data of a record.
+class LogicalRecord : public raw_svector_ostream {
+  GOFFOStream &OS;
+  SmallVector<char, 0> Buffer;
+
+  void anchor() override {};
 
-  // Return the current position within the stream, not counting the bytes
-  // currently in the buffer.
-  uint64_t current_pos() const override { return OS.tell(); }
+public:
+  LogicalRecord(GOFFOStream &OS) : raw_svector_ostream(Buffer), OS(OS) {}
+  ~LogicalRecord() override { OS << str(); }
+
+  LogicalRecord &operator<<(yaml::BinaryRef B) {
+    B.writeAsBinary(*this);
+    return *this;
+  }
 };
 
 class GOFFState {
-  void writeHeader(GOFFYAML::FileHeader &FileHdr);
-  void writeEnd();
+  void writeHeader(GOFFYAML::ModuleHeader &ModHdr);
+  void writeText(GOFFYAML::Text &Txt);
+  void writeEnd(GOFFYAML::EndOfModule &EndMod);
 
   void reportError(const Twine &Msg) {
     ErrHandler(Msg);
@@ -185,8 +158,6 @@ class GOFFState {
             yaml::ErrorHandler ErrHandler)
       : GW(OS), Doc(Doc), ErrHandler(ErrHandler), HasError(false) {}
 
-  ~GOFFState() { GW.finalize(); }
-
   bool writeObject();
 
 public:
@@ -194,72 +165,80 @@ class GOFFState {
                         yaml::ErrorHandler ErrHandler);
 
 private:
-  GOFFOstream GW;
+  GOFFOStream GW;
   GOFFYAML::Object &Doc;
   yaml::ErrorHandler ErrHandler;
   bool HasError;
 };
 
-void GOFFState::writeHeader(GOFFYAML::FileHeader &FileHdr) {
-  SmallString<16> CCSIDName;
-  if (std::error_code EC =
-          ConverterEBCDIC::convertToEBCDIC(FileHdr.CharacterSetName, CCSIDName))
-    reportError("Conversion error on " + FileHdr.CharacterSetName);
-  if (CCSIDName.size() > 16) {
-    reportError("CharacterSetName too long");
-    CCSIDName.resize(16);
-  }
-  SmallString<16> LangProd;
-  if (std::error_code EC = ConverterEBCDIC::convertToEBCDIC(
-          FileHdr.LanguageProductIdentifier, LangProd))
-    reportError("Conversion error on " + FileHdr.LanguageProductIdentifier);
-  if (LangProd.size() > 16) {
-    reportError("LanguageProductIdentifier too long");
-    LangProd.resize(16);
-  }
+void GOFFState::writeHeader(GOFFYAML::ModuleHeader &ModHdr) {
+  GW.newRecord(GOFF::RT_HDR);
+  LogicalRecord LR(GW);
+  LR << zeros(45)                          // Reserved.
+     << binaryBe(ModHdr.ArchitectureLevel) // The architecture level.
+     << binaryBe(ModHdr.PropertiesLength)  // Length of module properties.
+     << zeros(6);                          // Reserved.
+  if (ModHdr.Properties)
+    LR << *ModHdr.Properties; // Module properties.
+}
 
-  GW.makeNewRecord(GOFF::RT_HDR, GOFF::PayloadLength);
-  GW << binaryBe(FileHdr.TargetEnvironment)     // TargetEnvironment
-     << binaryBe(FileHdr.TargetOperatingSystem) // TargetOperatingSystem
-     << zeros(2)                                // Reserved
-     << binaryBe(FileHdr.CCSID)                 // CCSID
-     << CCSIDName                               // CharacterSetName
-     << zeros(16 - CCSIDName.size())            // Fill bytes
-     << LangProd                                // LanguageProductIdentifier
-     << zeros(16 - LangProd.size())             // Fill bytes
-     << binaryBe(FileHdr.ArchitectureLevel);    // ArchitectureLevel
-  // The module propties are optional. Figure out if we need to write them.
-  uint16_t ModPropLen = 0;
-  if (FileHdr.TargetSoftwareEnvironment)
-    ModPropLen = 3;
-  else if (FileHdr.InternalCCSID)
-    ModPropLen = 2;
-  if (ModPropLen) {
-    GW << binaryBe(ModPropLen) << zeros(6);
-    if (ModPropLen >= 2)
-      GW << binaryBe(FileHdr.InternalCCSID ? *FileHdr.InternalCCSID : 0);
-    if (ModPropLen >= 3)
-      GW << binaryBe(FileHdr.TargetSoftwareEnvironment
-                         ? *FileHdr.TargetSoftwareEnvironment
-                         : 0);
-  }
+void GOFFState::writeText(GOFFYAML::Text &Txt) {
+  GW.newRecord(GOFF::RT_TXT);
+  LogicalRecord LR(GW);
+  LR << binaryBe(uint8_t(Txt.Style)) // Text record style.
+     << binaryBe(
+            Txt.ESDID) // ESDID of the element/part to which this data belongs.
+     << zeros(4)       // Reserved.
+     << binaryBe(Txt.Offset)      // Starting offset from element/part.
+     << binaryBe(Txt.TrueLength)  // True length if encoded.
+     << binaryBe(Txt.Encoding)    // Encoding.
+     << binaryBe(Txt.DataLength); // Total length of data.
+  if (Txt.Data)
+    LR << *Txt.Data; // Data.
+  else
+    LR << zeros(Txt.DataLength);
 }
 
-void GOFFState::writeEnd() {
-  GW.makeNewRecord(GOFF::RT_END, GOFF::PayloadLength);
-  GW << binaryBe(uint8_t(0)) // No entry point
-     << binaryBe(uint8_t(0)) // No AMODE
-     << zeros(3)             // Reserved
-     << binaryBe(GW.logicalRecords());
-  // No entry point yet. Automatically fill remaining space with zero bytes.
-  GW.finalize();
+void GOFFState::writeEnd(GOFFYAML::EndOfModule &EndMod) {
+  SmallString<16> EntryName;
+  if (std::error_code EC =
+          ConverterEBCDIC::convertToEBCDIC(EndMod.EntryName, EntryName))
+    reportError("Conversion error on " + EndMod.EntryName);
+
+  GW.newRecord(GOFF::RT_END);
+  LogicalRecord LR(GW);
+  LR << binaryBe(uint8_t(EndMod.Flags)) // The flags.
+     << binaryBe(uint8_t(EndMod.AMODE)) // The addressing mode.
+     << zeros(3)                        // Reserved.
+     << binaryBe(EndMod.RecordCount)    // The record count.
+     << binaryBe(EndMod.ESDID)          // ESDID of the entry point.
+     << zeros(4)                        // Reserved.
+     << binaryBe(EndMod.Offset)         // Offset of entry point.
+     << binaryBe(EndMod.NameLength)     // Length of external name.
+     << EntryName;                      // Name of the entry point.
 }
 
 bool GOFFState::writeObject() {
-  writeHeader(Doc.Header);
-  if (HasError)
-    return false;
-  writeEnd();
+  for (auto &RecPtr : Doc.Records) {
+    auto *Rec = RecPtr.get();
+    switch (Rec->getKind()) {
+    case GOFFYAML::RecordBase::RBK_ModuleHeader:
+      writeHeader(*static_cast<GOFFYAML::ModuleHeader *>(Rec));
+      break;
+    case GOFFYAML::RecordBase::RBK_Text:
+      writeText(*static_cast<GOFFYAML::Text *>(Rec));
+      break;
+    case GOFFYAML::RecordBase::RBK_EndOfModule:
+      writeEnd(*static_cast<GOFFYAML::EndOfModule *>(Rec));
+      break;
+    case GOFFYAML::RecordBase::RBK_RelocationDirectory:
+    case GOFFYAML::RecordBase::RBK_Symbol:
+    case GOFFYAML::RecordBase::RBK_DeferredLength:
+      llvm_unreachable(("Not yet implemented"));
+    }
+    if (HasError)
+      return false;
+  }
   return true;
 }
 
diff --git a/llvm/lib/ObjectYAML/GOFFYAML.cpp b/llvm/lib/ObjectYAML/GOFFYAML.cpp
index ae857980a521b..d3c8b02bcb6a3 100644
--- a/llvm/lib/ObjectYAML/GOFFYAML.cpp
+++ b/llvm/lib/ObjectYAML/GOFFYAML.cpp
@@ -12,34 +12,118 @@
 
 #include "llvm/ObjectYAML/GOFFYAML.h"
 #include "llvm/BinaryFormat/GOFF.h"
-#include <string.h>
 
 namespace llvm {
-namespace GOFFYAML {
 
-Object::Object() {}
+namespace yaml {
 
-} // namespace GOFFYAML
+void ScalarEnumerationTraits<GOFFYAML::GOFF_AMODE>::enumeration(
+    IO &IO, GOFFYAML::GOFF_AMODE &Value) {
+#define ECase(X) IO.enumCase(Value, #X, GOFF::ESD_##X)
+  ECase(AMODE_None);
+  ECase(AMODE_24);
+  ECase(AMODE_31);
+  ECase(AMODE_ANY);
+  ECase(AMODE_64);
+  ECase(AMODE_MIN);
+#undef ECase
+  IO.enumFallback<Hex8>(Value);
+}
 
-namespace yaml {
+void ScalarEnumerationTraits<GOFFYAML::GOFF_TXTRECORDSTYLE>::enumeration(
+    IO &IO, GOFFYAML::GOFF_TXTRECORDSTYLE &Value) {
+#define ECase(X) IO.enumCase(Value, #X, GOFF::TXT_##X)
+  ECase(TS_Byte);
+  ECase(TS_Structured);
+  ECase(TS_Unstructured);
+#undef ECase
+  IO.enumFallback<Hex8>(Value);
+}
+
+void ScalarEnumerationTraits<GOFFYAML::GOFF_ENDFLAGS>::enumeration(
+    IO &IO, GOFFYAML::GOFF_ENDFLAGS &Value) {
+#define ECase(X) IO.enumCase(Value, #X, unsigned(GOFF::END_##X) << 6)
+  ECase(EPR_None);
+  ECase(EPR_EsdidOffset);
+  ECase(EPR_ExternalName);
+  ECase(EPR_Reserved);
+#undef ECase
+  IO.enumFallback<Hex8>(Value);
+}
+
+void MappingTraits<GOFFYAML::ModuleHeader>::mapping(
+    IO &IO, GOFFYAML::ModuleHeader &ModHdr) {
+  IO.mapOptional("ArchitectureLevel", ModHdr.ArchitectureLevel, 0);
+  IO.mapOptional("PropertiesLength", ModHdr.PropertiesLength, 0);
+  IO.mapOptional("Properties", ModHdr.Properties);
+}
+
+void MappingTraits<GOFFYAML::Text>::mapping(IO &IO, GOFFYAML::Text &Txt) {
+  IO.mapOptional("TextStyle", Txt.Style, GOFF::TXT_TS_Byte);
+  IO.mapOptional("ESDID", Txt.ESDID, 0);
+  IO.mapOptional("Offset", Txt.Offset, 0);
+  IO.mapOptional("TrueLength", Txt.TrueLength, 0);
+  IO.mapOptional("TextEncoding", Txt.Encoding, 0);
+  IO.mapOptional("DataLength", Txt.DataLength, 0);
+  IO.mapOptional("Data", Txt.Data);
+}
+
+void MappingTraits<GOFFYAML::EndOfModule>::mapping(IO &IO,
+                                                   GOFFYAML::EndOfModule &End) {
+  IO.mapOptional("Flags", End.Flags, 0);
+  IO.mapOptional("AMODE", End.AMODE, 0);
+  IO.mapOptional("RecordCount", End.RecordCount, 0);
+  IO.mapOptional("ESDID", End.ESDID, 0);
+  IO.mapOptional("Offset", End.Offset, 0);
+  IO.mapOptional("NameLength", End.NameLength, 0);
+  IO.mapOptional("EntryName", End.EntryName);
+}
+
+void CustomMappingTraits<GOFFYAML::RecordPtr>::inputOne(
+    IO &IO, StringRef Key, GOFFYAML::RecordPtr &Elem) {
+  if (Key == "ModuleHeader") {
+    GOFFYAML::ModuleHeader ModHdr;
+    IO.mapRequired("ModuleHeader", ModHdr);
+    Elem = std::make_unique<GOFFYAML::ModuleHeader>(std::move(ModHdr));
+  } else if (Key == "Text") {
+    GOFFYAML::Text Txt;
+    IO.mapRequired("Text", Txt);
+    Elem = std::make_unique<GOFFYAML::Text>(std::move(Txt));
+  } else if (Key ...
[truncated]

@redstar
Copy link
Member Author

redstar commented May 30, 2024

Hm. I wanted to stack this PR on #93855, but looks like I can't change the target branch.

@redstar redstar force-pushed the knacke/goff-yaml2obj-2 branch from 239deea to 7689de9 Compare June 7, 2024 17:30
Add support for writing text records to yaml2obj.
@redstar redstar force-pushed the knacke/goff-yaml2obj-2 branch from 7689de9 to 48df2fc Compare June 7, 2024 17:47
@jh7370
Copy link
Collaborator

jh7370 commented Jun 10, 2024

Hm. I wanted to stack this PR on #93855, but looks like I can't change the target branch.

You can't stack PRs from forks on each other, because then the PR wouldn't be part of the llvm repo. You might want to look into using spr for your PRs in this case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants