Skip to content

Commit 891af9b

Browse files
egorovcharenkometa-codesync[bot]
authored andcommitted
Skip filter_strip_marker for symlinks during shipit mirror
Summary: The `oss-mcrouter-darwin-getdeps` (T262442347) and `oss-mcrouter-windows-getdeps` sandcastle jobs have been failing continuously since D93801016 landed (2026-02-27) with: ``` FileNotFoundError: [Errno 2] No such file or directory: '/data/sandcastle/temp/skycastle/fbcode_builder_getdeps/shipit/mcrouter/mcrouter/scripts/order_centos-7.2/15_fbthrift' ``` Root cause: mcrouter's `scripts/order_centos-7.2/` directory contains relative symlinks like `15_fbthrift -> ../recipes/fbthrift.sh`. During `ShipitPathMap.mirror()`, `os.walk` visits `order_centos-7.2/` before `recipes/` (alphabetical order). For each file `copy_if_different()` reproduces symlinks via `os.symlink(target, dest)` — which can be dangling because the target hasn't been mirrored yet. The new `filter_strip_marker()` call added by D93801016 then tries to `open(dest_name)`, follows the dangling symlink, and crashes. Fix: skip `filter_strip_marker` for symlinks. The actual target file gets its markers stripped when `os.walk` visits it directly, so behavior on real files is unchanged. This unblocks external builds of mcrouter on macOS and Windows, which have been broken for ~2 months. No production impact — internal mcrouter/uCache builds use Buck2 and never go through this shipit/getdeps path. Two earlier attempts addressed the Windows variant of the same bug but did not land: D98509103 (abandoned) and D98518095 (unpublished, same approach as this diff). This diff supersedes both and ties to the Darwin task. #minion-generated-fix Reviewed By: r-barnes Differential Revision: D102330897 fbshipit-source-id: 72ca83526b66d088c4094235d8e8727414330d15
1 parent 465dd47 commit 891af9b

1 file changed

Lines changed: 41 additions & 1 deletion

File tree

build/fbcode_builder/getdeps/test/strip_marker_test.py

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
import tempfile
1111
import unittest
1212

13-
from ..fetcher import filter_strip_marker
13+
from ..fetcher import filter_strip_marker, ShipitPathMap
1414
from ..manifest import ManifestParser
1515

1616

@@ -157,3 +157,43 @@ def test_binary_file_skipped(self) -> None:
157157
self.assertEqual(f.read(), binary_content)
158158
finally:
159159
os.unlink(path)
160+
161+
162+
class ShipitMirrorSymlinkTest(unittest.TestCase):
163+
"""Regression test for mcrouter OSS getdeps build (T262442347).
164+
165+
mcrouter's scripts/order_centos-7.2/15_fbthrift is a relative symlink to
166+
../recipes/fbthrift.sh. os.walk visits order_centos-7.2/ before recipes/
167+
alphabetically, so when mirror() copied the symlink first, filter_strip_marker
168+
followed the dangling link and crashed with FileNotFoundError. mirror() must
169+
skip filter_strip_marker for symlinks; the actual target file gets filtered
170+
when os.walk reaches it on its own.
171+
"""
172+
173+
def test_mirror_skips_filter_for_dangling_symlinks(self) -> None:
174+
with tempfile.TemporaryDirectory() as src_root, tempfile.TemporaryDirectory() as dest_root:
175+
project_dir = os.path.join(src_root, "proj")
176+
order_dir = os.path.join(project_dir, "scripts", "order_centos-7.2")
177+
recipes_dir = os.path.join(project_dir, "scripts", "recipes")
178+
os.makedirs(order_dir)
179+
os.makedirs(recipes_dir)
180+
181+
target_file = os.path.join(recipes_dir, "fbthrift.sh")
182+
with open(target_file, "w") as f:
183+
f.write("#!/bin/bash\necho hello\n")
184+
185+
symlink_path = os.path.join(order_dir, "15_fbthrift")
186+
os.symlink("../recipes/fbthrift.sh", symlink_path)
187+
188+
mapping = ShipitPathMap()
189+
mapping.add_mapping("proj", "proj")
190+
mapping.mirror(src_root, dest_root)
191+
192+
mirrored_symlink = os.path.join(
193+
dest_root, "proj", "scripts", "order_centos-7.2", "15_fbthrift"
194+
)
195+
mirrored_target = os.path.join(
196+
dest_root, "proj", "scripts", "recipes", "fbthrift.sh"
197+
)
198+
self.assertTrue(os.path.islink(mirrored_symlink))
199+
self.assertTrue(os.path.isfile(mirrored_target))

0 commit comments

Comments
 (0)