Skip to content

Commit 4357d39

Browse files
Alramechfacebook-github-bot
authored andcommitted
Implement usage of root inside changesSinceV2
Summary: # Context We are introducing EdenFS notifications to support scalable and ergonomic file system notifications for EdenFS mounts. Watchman has a concept known as a relative root https://www.internalfb.com/intern/staticdocs/watchman/docs/file-query#relative-roots This diff introduces the concept of a root to changesSinceV2. A root is a path starting from the existing mount, from which all paths are computed against and acts as an overall include root, eg. all returned paths will start with the root, after which include/exclude roots will be computed, and paths outside the root will not be returned. # This Diff Implements root checking. Use the following flowchart for computing included/excluded roots. ```lang=mermaid flowchart TD A[Has Root?] -->|No| B(No Change) A -->|Yes| C(Has included roots?) C-->|Yes| D(Root is prepended to included roots and excluded roots) C-->|No| E(Root becomes included root, and prependended to excluded roots, if any) ``` Use the following flowchart for determining how to handle outputs ```lang=mermaid flowchart TD A[Has Root?] -->|No| B(No Change) A -->|Yes| C(Does change contain only one path) C-->|Yes| D(Root is stripped from output) C-->|No| E(Are both paths inside the root?) E-->|Yes| F(Strip root from both paths, return replace/rename) E-->|No| G(Is the change from outside the root into the root) G-->|Yes| H(Does the path already exist inside the root) G-->|No| I(Convert the change into Removed) H-->|Yes| J(Convert the change into Modified, from Replaced) H-->|No| K(Convert the change into Added, From Renamed) ``` Note that a directory rename is converted from a Large Change(DirectoryRenamed) into a Small Change(Directory Added/removed/modified) # Technical Details This diff adds some additional allocation overhead. This is to make the code more readable. If it becomes a performance issue, this can be optimized down the line. The root matching is implemented by augmenting the includedRoots entries, as demonstrated in the flowchart above. The input included/excluded roots are appended to the root value. # Discussion Points Reviewed By: jdelliot Differential Revision: D72603242 fbshipit-source-id: 60d3b54ffd0ad5472856b13bf3c8920f1801ac01
1 parent d80377a commit 4357d39

File tree

5 files changed

+496
-71
lines changed

5 files changed

+496
-71
lines changed

eden/fs/service/EdenServiceHandler.cpp

Lines changed: 149 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -557,6 +557,14 @@ RelativePath relpathFromUserPath(StringPiece userPath) {
557557
}
558558
}
559559

560+
RelativePathPiece relpathPieceFromUserPath(StringPiece userPath) {
561+
if (userPath.empty() || userPath == ".") {
562+
return RelativePathPiece{};
563+
} else {
564+
return RelativePathPiece{userPath};
565+
}
566+
}
567+
560568
facebook::eden::InodePtr inodeFromUserPath(
561569
facebook::eden::EdenMount& mount,
562570
StringPiece rootRelativePath,
@@ -2232,20 +2240,37 @@ buildIncludedAndExcludedRoots(
22322240
bool includeVCSRoots,
22332241
const std::vector<RelativePath>& vcsDirectories,
22342242
const std::vector<PathString>& includedRoots,
2235-
const std::vector<PathString>& excludedRoots) {
2236-
std::vector<RelativePath> outIncludedRoots(includedRoots.size());
2237-
std::transform(
2238-
includedRoots.begin(),
2239-
includedRoots.end(),
2240-
outIncludedRoots.begin(),
2241-
[](PathString root) { return RelativePath{root}; });
2243+
const std::vector<PathString>& excludedRoots,
2244+
const RelativePathPiece& root) {
2245+
// This uses/returns RelativePath instead of RelativePathPiece due to the
2246+
// value constructed with the root + include/excludeRoot going out of
2247+
// scope
2248+
std::vector<RelativePath> outIncludedRoots;
2249+
outIncludedRoots.reserve(includedRoots.size());
2250+
// If there are includedRoots, append them to the root if there
2251+
// is one, otherwise just fill out the vector with the values
2252+
if (includedRoots.size() > 0) {
2253+
std::transform(
2254+
includedRoots.begin(),
2255+
includedRoots.end(),
2256+
std::back_inserter(outIncludedRoots),
2257+
[root](const PathString& includedRoot) {
2258+
return root + relpathPieceFromUserPath(includedRoot);
2259+
});
2260+
} else if (!root.empty()) {
2261+
// If there are no includedRoots and there is a root, use
2262+
// it as an includedRoot
2263+
outIncludedRoots.emplace_back(root);
2264+
}
22422265

22432266
std::vector<RelativePath> outExcludedRoots(excludedRoots.size());
22442267
std::transform(
22452268
excludedRoots.begin(),
22462269
excludedRoots.end(),
22472270
outExcludedRoots.begin(),
2248-
[](PathString root) { return RelativePath{root}; });
2271+
[root](const PathString& excludedRoot) {
2272+
return root + relpathPieceFromUserPath(excludedRoot);
2273+
});
22492274

22502275
if (includeVCSRoots) {
22512276
outIncludedRoots.insert(
@@ -2329,13 +2354,17 @@ void EdenServiceHandler::sync_changesSinceV2(
23292354
std::unique_ptr<ChangesSinceV2Params> params) {
23302355
auto mountHandle = lookupMount(params->mountPoint());
23312356
const auto& fromPosition = *params->fromPosition_ref();
2332-
auto root = params->root_ref().has_value() ? params->root_ref().value() : "";
2357+
RelativePathPiece root = params->root_ref().has_value()
2358+
? RelativePathPiece{params->root_ref().value()}
2359+
: RelativePathPiece{};
2360+
23332361
auto includedRoots = params->includedRoots_ref().has_value()
23342362
? params->includedRoots_ref().value()
23352363
: std::vector<PathString>{};
23362364
auto excludedRoots = params->excludedRoots_ref().has_value()
23372365
? params->excludedRoots_ref().value()
23382366
: std::vector<PathString>{};
2367+
23392368
auto includedSuffixes = params->includedSuffixes_ref().has_value()
23402369
? params->includedSuffixes_ref().value()
23412370
: std::vector<std::string>{};
@@ -2372,12 +2401,11 @@ void EdenServiceHandler::sync_changesSinceV2(
23722401
if (!root.empty()) {
23732402
auto& mount = mountHandle.getEdenMount();
23742403
auto& mountPath = mount.getPath();
2375-
auto repoPath = RelativePathPiece{root};
23762404
bool rootExists =
23772405
waitForPendingWrites(mount, *params->sync())
23782406
.thenValue(
2379-
[&mount, repoPath, fetchContext = fetchContext.copy()](auto&&) {
2380-
return mount.getVirtualInode(repoPath, fetchContext)
2407+
[&mount, root, fetchContext = fetchContext.copy()](auto&&) {
2408+
return mount.getVirtualInode(root, fetchContext)
23812409
.thenTry([](folly::Try<VirtualInode> tree) mutable {
23822410
if (tree.hasException()) {
23832411
// Root does not exist, or something else went wrong
@@ -2418,13 +2446,14 @@ void EdenServiceHandler::sync_changesSinceV2(
24182446
// TODO: move to helper
24192447
auto config = server_->getServerState()->getEdenConfig();
24202448
auto maxNumberOfChanges = config->notifyMaxNumberOfChanges.getValue();
2421-
auto includedAndExcludeRoots = buildIncludedAndExcludedRoots(
2449+
auto includedAndExcludedRoots = buildIncludedAndExcludedRoots(
24222450
params->includeVCSRoots().has_value()
24232451
? params->includeVCSRoots().value()
24242452
: false,
24252453
config->vcsDirectories.getValue(),
24262454
includedRoots,
2427-
excludedRoots);
2455+
excludedRoots,
2456+
root);
24282457
auto includedAndExcludedSuffixes = std::make_pair(
24292458
std::move(includedSuffixes), std::move(excludedSuffixes));
24302459

@@ -2440,8 +2469,8 @@ void EdenServiceHandler::sync_changesSinceV2(
24402469
// Changes can effect either path1 or both paths
24412470
// Determine if path1 pass the filters and default path2 to not
24422471
bool includePath1 = isPathIncluded(
2443-
includedAndExcludeRoots.first,
2444-
includedAndExcludeRoots.second,
2472+
includedAndExcludedRoots.first,
2473+
includedAndExcludedRoots.second,
24452474
includedAndExcludedSuffixes.first,
24462475
includedAndExcludedSuffixes.second,
24472476
current.path1);
@@ -2456,8 +2485,8 @@ void EdenServiceHandler::sync_changesSinceV2(
24562485
includePath2 = includePath1
24572486
? false
24582487
: isPathIncluded(
2459-
includedAndExcludeRoots.first,
2460-
includedAndExcludeRoots.second,
2488+
includedAndExcludedRoots.first,
2489+
includedAndExcludedRoots.second,
24612490
includedAndExcludedSuffixes.first,
24622491
includedAndExcludedSuffixes.second,
24632492
current.path2);
@@ -2467,55 +2496,132 @@ void EdenServiceHandler::sync_changesSinceV2(
24672496
// validate the infoN states would be a lot simpler if we
24682497
// removed this infoN state and replaced them with a simple
24692498
// enum
2470-
if (info.existedBefore) {
2471-
// Replaced
2472-
Replaced replaced;
2473-
replaced.from() = current.path1.asString();
2474-
replaced.to() = current.path2.asString();
2475-
replaced.fileType() = static_cast<Dtype>(current.type);
2476-
smallChange.replaced_ref() = std::move(replaced);
2477-
change.smallChange_ref() = std::move(smallChange);
2478-
} else {
2479-
// Renamed
2480-
if (current.type == dtype_t::Dir) {
2481-
DirectoryRenamed directoryRenamed;
2482-
directoryRenamed.from() = current.path1.asString();
2483-
directoryRenamed.to() = current.path2.asString();
2484-
largeChange.directoryRenamed_ref() =
2485-
std::move(directoryRenamed);
2486-
change.largeChange_ref() = std::move(largeChange);
2499+
2500+
// Constructs a Piece from the RelativePath
2501+
RelativePathPiece pathString1 = current.path1;
2502+
RelativePathPiece pathString2 = current.path2;
2503+
2504+
// If root is empty, returns true if pathString is not also empty
2505+
bool path1InRoot = pathString1.isSubDirOf(root);
2506+
bool path2InRoot = pathString2.isSubDirOf(root);
2507+
2508+
if (!root.empty()) {
2509+
// Filters include the path that matches the filter, but we want
2510+
// to exclude it from a root
2511+
// This is to match watchman's behavior regarding
2512+
// relative roots.
2513+
if (pathString1 == root || pathString2 == root) {
2514+
// Return value ignored here
2515+
return true;
2516+
}
2517+
// Trim the root + the separator
2518+
if (path1InRoot) {
2519+
pathString1 = pathString1.substr(root.view().size() + 1);
2520+
}
2521+
if (path2InRoot) {
2522+
pathString2 = pathString2.substr(root.view().size() + 1);
2523+
}
2524+
}
2525+
2526+
if (path1InRoot && path2InRoot) {
2527+
if (info.existedBefore) {
2528+
// Replaced
2529+
Replaced replaced;
2530+
replaced.from() = pathString1.asString();
2531+
replaced.to() = pathString2.asString();
2532+
replaced.fileType() = static_cast<Dtype>(current.type);
2533+
smallChange.replaced_ref() = std::move(replaced);
2534+
change.smallChange_ref() = std::move(smallChange);
24872535
} else {
2488-
Renamed renamed;
2489-
renamed.from() = current.path1.asString();
2490-
renamed.to() = current.path2.asString();
2491-
renamed.fileType() = static_cast<Dtype>(current.type);
2492-
smallChange.renamed_ref() = std::move(renamed);
2536+
// Renamed
2537+
if (current.type == dtype_t::Dir) {
2538+
DirectoryRenamed directoryRenamed;
2539+
directoryRenamed.from() = pathString1.asString();
2540+
directoryRenamed.to() = pathString2.asString();
2541+
largeChange.directoryRenamed_ref() =
2542+
std::move(directoryRenamed);
2543+
change.largeChange_ref() = std::move(largeChange);
2544+
} else {
2545+
Renamed renamed;
2546+
renamed.from() = pathString1.asString();
2547+
renamed.to() = pathString2.asString();
2548+
renamed.fileType() = static_cast<Dtype>(current.type);
2549+
smallChange.renamed_ref() = std::move(renamed);
2550+
change.smallChange_ref() = std::move(smallChange);
2551+
}
2552+
}
2553+
} else {
2554+
if (path1InRoot) {
2555+
// File/Directory was renamed or replaced to a path outside of
2556+
// root. Report change as removed.
2557+
Removed removed;
2558+
removed.path() = pathString1.asString();
2559+
removed.fileType() = static_cast<Dtype>(current.type);
2560+
smallChange.removed_ref() = std::move(removed);
24932561
change.smallChange_ref() = std::move(smallChange);
2562+
} else {
2563+
// File/Directory was renamed or replaced to a path inside of
2564+
// root. Report change as added (if renamed) or modified (if
2565+
// replaced).
2566+
if (info.existedBefore) {
2567+
// Modified
2568+
Modified modified;
2569+
modified.path() = pathString2.asString();
2570+
modified.fileType() = static_cast<Dtype>(current.type);
2571+
smallChange.modified_ref() = std::move(modified);
2572+
change.smallChange_ref() = std::move(smallChange);
2573+
} else {
2574+
Added added;
2575+
added.path() = pathString2.asString();
2576+
added.fileType() = static_cast<Dtype>(current.type);
2577+
smallChange.added_ref() = std::move(added);
2578+
change.smallChange_ref() = std::move(smallChange);
2579+
}
24942580
}
24952581
}
24962582
}
24972583
}
24982584
// All single file changes have path1 pass the filters
24992585
else if (includePath1) {
25002586
const auto& info = current.info1;
2587+
2588+
// Filters include the path that matches the filter, but we want
2589+
// to exclude it from a root
2590+
// This is to match watchman's behavior regarding
2591+
// relative roots.
2592+
if (!root.empty() && current.path1 == root) {
2593+
// Return value ignored here
2594+
return true;
2595+
}
2596+
2597+
// If a root is specified, it should be present due to being added
2598+
// to includedRoots. Strip it and the first '/' out
2599+
2600+
// Need to explicitly allocate storage in this scope for
2601+
// mac/windows
2602+
RelativePathPiece pathString = current.path1;
2603+
2604+
if (!root.empty()) {
2605+
pathString = pathString.substr(root.view().size());
2606+
}
25012607
if (!info.existedBefore) {
25022608
// Added
25032609
Added added;
2504-
added.path() = current.path1.asString();
2610+
added.path() = pathString.asString();
25052611
added.fileType() = static_cast<Dtype>(current.type);
25062612
smallChange.added_ref() = std::move(added);
25072613
change.smallChange_ref() = std::move(smallChange);
25082614
} else if (!info.existedAfter) {
25092615
// Removed
25102616
Removed removed;
2511-
removed.path() = current.path1.asString();
2617+
removed.path() = pathString.asString();
25122618
removed.fileType() = static_cast<Dtype>(current.type);
25132619
smallChange.removed_ref() = std::move(removed);
25142620
change.smallChange_ref() = std::move(smallChange);
25152621
} else {
25162622
// Modified
25172623
Modified modified;
2518-
modified.path() = current.path1.asString();
2624+
modified.path() = pathString.asString();
25192625
modified.fileType() = static_cast<Dtype>(current.type);
25202626
smallChange.modified_ref() = std::move(modified);
25212627
change.smallChange_ref() = std::move(smallChange);

0 commit comments

Comments
 (0)