[llvm][yaml2obj] Modify section header overriding timing #130942
Conversation
Signed-off-by: Ruoyu Qiu <[email protected]>
|
@llvm/pr-subscribers-llvm-binary-utilities @llvm/pr-subscribers-objectyaml Author: Ruoyu Qiu (cabbaken) Changesyaml2obj should determine the program header offset (and other properties) based on the intended values rather than the final sh_offset of the section header. This change adjusts the timing of when the section header is overridden to ensure that the program headers are set correctly. More details here. Full diff: https://github.com/llvm/llvm-project/pull/130942.diff 1 Files Affected:
diff --git a/llvm/lib/ObjectYAML/ELFEmitter.cpp b/llvm/lib/ObjectYAML/ELFEmitter.cpp
index 9ae76a71ede5e..48d5d58beea86 100644
--- a/llvm/lib/ObjectYAML/ELFEmitter.cpp
+++ b/llvm/lib/ObjectYAML/ELFEmitter.cpp
@@ -206,6 +206,9 @@ template <class ELFT> class ELFState {
NameToIdxMap DynSymN2I;
ELFYAML::Object &Doc;
+ std::vector<std::pair<unsigned, ELFYAML::Section>>
+ SectionHeadersOverrideHelper;
+
StringSet<> ExcludedSectionHeaders;
uint64_t LocationCounter = 0;
@@ -226,6 +229,7 @@ template <class ELFT> class ELFState {
StringRef SecName, ELFYAML::Section *YAMLSec);
void initSectionHeaders(std::vector<Elf_Shdr> &SHeaders,
ContiguousBlobAccumulator &CBA);
+ void overrideSectionHeaders(std::vector<Elf_Shdr> &SHeaders);
void initSymtabSectionHeader(Elf_Shdr &SHeader, SymtabType STType,
ContiguousBlobAccumulator &CBA,
ELFYAML::Section *YAMLSec);
@@ -845,7 +849,7 @@ void ELFState<ELFT>::initSectionHeaders(std::vector<Elf_Shdr> &SHeaders,
}
LocationCounter += SHeader.sh_size;
- overrideFields<ELFT>(Sec, SHeader);
+ SectionHeadersOverrideHelper.push_back({SN2I.get(Sec->Name), *Sec});
continue;
}
@@ -899,12 +903,17 @@ void ELFState<ELFT>::initSectionHeaders(std::vector<Elf_Shdr> &SHeaders,
}
LocationCounter += SHeader.sh_size;
-
- // Override section fields if requested.
- overrideFields<ELFT>(Sec, SHeader);
+ SectionHeadersOverrideHelper.push_back({SN2I.get(Sec->Name), *Sec});
}
}
+template <class ELFT>
+void ELFState<ELFT>::overrideSectionHeaders(std::vector<Elf_Shdr> &SHeaders) {
+ for (std::pair<unsigned, ELFYAML::Section> &IndexAndSec :
+ SectionHeadersOverrideHelper)
+ overrideFields<ELFT>(&IndexAndSec.second, SHeaders[IndexAndSec.first]);
+}
+
template <class ELFT>
void ELFState<ELFT>::assignSectionAddress(Elf_Shdr &SHeader,
ELFYAML::Section *YAMLSec) {
@@ -2090,6 +2099,9 @@ bool ELFState<ELFT>::writeELF(raw_ostream &OS, ELFYAML::Object &Doc,
// Now we can decide segment offsets.
State.setProgramHeaderLayout(PHeaders, SHeaders);
+ // Override section fields if requested.
+ State.overrideSectionHeaders(SHeaders);
+
bool ReachedLimit = CBA.getOffset() > MaxSize;
if (Error E = CBA.takeLimitError()) {
// We report a custom error message instead below.
|
|
We can see the test of But the code here(FileSize and MemSize) this behavior is correct. Both the last two section of the ProgramHeader is And I also found another two failed tests
@jh7370 Do you have any ideas about this? Should these tests be correct? |
There was a problem hiding this comment.
Unfortunately, apart from maybe a couple of quick responses over the next couple of hours, I've run out of time for this today and am off work for the next couple of weeks, so it'll be ~3 weeks before I can continue looking at this. Hopefully @MaskRay might be able to review this PR further. I'm happy for it to go in (once the issues have been addressed) if he is.
We can see the test of
tools/yaml2obj/ELF/program-header-size-offset.yamlfailed because of the unmatchedFileSizeandMemSizeof the last PorgramHeader after delaying theoverrideFields().But the code here(FileSize and MemSize) this behavior is correct. Both the last two section of the ProgramHeader is
SHT_NOBITS, which won't occupy physical space in a file. Does this indicate that the current test has a mistake?
I think the test is just using ShOffset when it should be using Offset instead. I'm not 100% certain about this though and would like @MaskRay to take a look if he gets a chance. Does changing that help fix any issues?
(Additionally, the code only checks for the last
SHT_NOBITShere, rather than all of them.)
It's too long ago to remember whether this was by design or not. I'd avoid changing it unless you can see there's an obvious hole in the test coverage of the actual code. @MaskRay may have a different opinion and I'm happy to defer to that.
And I also found another two failed tests
* [`tools/llvm-readobj/ELF/malformed-pt-dynamic.test`](https://github.com/llvm/llvm-project/blob/main/llvm/test/tools/llvm-readobj/ELF/malformed-pt-dynamic.test#L43): Failed because it assumes `p_offset` is determined by `sh_offset`.
I think the fix here is to avoid this assumption somehow. The test itself is fine, it's just the change in yaml2obj behaviour means we need a different way of forcing the p_offset value. I believe you can fix this test by removing the ShOffset field and replacing it with an Offset field in the program header description.
* [`tools/obj2yaml/ELF/program-headers.yaml`](https://github.com/llvm/llvm-project/blob/main/llvm/test/tools/obj2yaml/ELF/program-headers.yaml#L437): Failed due to a mismatch between the overridden `sh_offset` and the corresponding `p_offset`.
The line number reference here didn't make sense to me for the description you've mentioned. Are you referring to the bar/zed sections at
and the check at line 533? If so, I'm not sure what the fix should be here. ChangingShOffset to Offset in the zed section description might work, but I'm not sure if it will or not (it might result in a different error). If that doesn't work, you might be able to use the SectionHeaderTable field in the YAML to reorder the section headers: put the zed section first in the Sections: list in the YAML, without any specific offset (but a non-zero size), followed by bar, but then in the SectionHeaderTable listing, specify the sections in the opposite order. You might need to break the individual test case into two parts, with two different YAMLs, each for one of the warning messages. Aside: this test case is in the wrong directory, as it's purely a yaml2obj test, not an obj2yaml one. I'm not sure it's worth moving though.
llvm/lib/ObjectYAML/ELFEmitter.cpp
Outdated
| @@ -845,7 +849,7 @@ void ELFState<ELFT>::initSectionHeaders(std::vector<Elf_Shdr> &SHeaders, | |||
| } | |||
|
|
|||
| LocationCounter += SHeader.sh_size; | |||
| overrideFields<ELFT>(Sec, SHeader); | |||
| SectionHeadersOverrideHelper.push_back({SN2I.get(Sec->Name), *Sec}); | |||
There was a problem hiding this comment.
I'm struggling to follow why you can't just store Sec and SHeader in the vector (SHeader as a pointer, obviously)?
llvm/lib/ObjectYAML/ELFEmitter.cpp
Outdated
| void ELFState<ELFT>::overrideSectionHeaders(std::vector<Elf_Shdr> &SHeaders) { | ||
| for (std::pair<unsigned, ELFYAML::Section> &IndexAndSec : | ||
| SectionHeadersOverrideHelper) | ||
| overrideFields<ELFT>(&IndexAndSec.second, SHeaders[IndexAndSec.first]); |
There was a problem hiding this comment.
I think it would just be simpler to update the overrideFields function to take the vector of pairs and iterate it over it inside the function, rather than have this additional function. If you want to rename it to overrideSectionHeaders for additional clarity, I'm not opposed to that.
There was a problem hiding this comment.
initImplicitHeader() will also use overrideFields(), so I defined a new function to call overrideFields().
llvm/lib/ObjectYAML/ELFEmitter.cpp
Outdated
| @@ -2090,6 +2099,9 @@ bool ELFState<ELFT>::writeELF(raw_ostream &OS, ELFYAML::Object &Doc, | |||
| // Now we can decide segment offsets. | |||
| State.setProgramHeaderLayout(PHeaders, SHeaders); | |||
|
|
|||
| // Override section fields if requested. | |||
There was a problem hiding this comment.
| // Override section fields if requested. | |
| // Override section fields, if requested. This needs to happen after program header layout happens, because otherwise the layout will use the new values. |
(and reflow to column width rules of course)
…emove unneed test Signed-off-by: Ruoyu Qiu <[email protected]>
Yes, this fixed this test.
I tried to fix this tests, but I got in malformed-pt-dynamic.test with --- a/llvm/test/tools/llvm-readobj/ELF/malformed-pt-dynamic.test
+++ b/llvm/test/tools/llvm-readobj/ELF/malformed-pt-dynamic.test
@@ -133,7 +133,6 @@ Sections:
Type: SHT_DYNAMIC
Address: 0x1000
Offset: 0x1000
- ShOffset: [[OFFSET=<none>]]
Entries:
- Tag: DT_NULL
Value: 0
@@ -142,5 +141,6 @@ Sections:
ProgramHeaders:
- Type: PT_DYNAMIC
FileSize: [[FILESIZE=<none>]]
+ Offset: [[OFFSET=<none>]]
FirstSec: .dynamic
LastSec: .dynamicThis occurs because there is a check when the program header offset is explicitly defined. It seems that
And about # NOBITS-NEXT: FirstSec: .bss
# NOBITS-NEXT: LastSec: .bssand it will continue to fail here: # UNSORTED-NEXT: error: sections in the program header with index 3 are not sorted by their file offset |
Signed-off-by: Ruoyu Qiu <[email protected]>
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
Signed-off-by: Ruoyu Qiu <[email protected]>
yaml2obj should determine the program header offset (and other properties) based on the intended values rather than the final sh_offset of the section header.
This change adjusts the timing of when the section header is overridden to ensure that the program headers are set correctly.
More details here.