Skip to content

Commit 7a46fc1

Browse files
ricklavoiemeta-codesync[bot]
authored andcommitted
Rewrite AnalysisScheduler with trace-based scheduling algorithm for HHBBC
Summary: The previous AnalysisScheduler required N analysis rounds to propagate information through a dependency chain of length N. This rewrite introduces a trace-based scheduling algorithm that processes entire dependency chains together, allowing information to propagate through the entire chain in a single round, dramatically reducing total analysis time. TRACE-BASED SCHEDULING (index.cpp:23683-25388, index.h:2389-2707): The new scheduler builds "traces" - sets of transitive dependencies that should be analyzed together. If A depends on B depends on C depends on D, then A's trace includes [A,B,C,D]. All entities in the trace are processed in the same job, collapsing what would be 4 sequential rounds into 1. The scheduler operates by: - Determining entity eligibility (which entities might change this round) - Building traces by following dependency chains up to configurable size/depth - Packing traces into size-bounded buckets using greedy bin-packing with overlap optimization (multiple traces can share bundles) - Classifying bundles as reportBundles (analyze and report), noReportBundles (analyze speculatively but don't report), or pureDepBundles (information only) Entities are tracked at fine granularity but scheduled at bundle granularity since entities are stored in bundles for distributed execution. DATA STRUCTURE CHANGES (index.h:2236-2387): Restructured AnalysisInput and AnalysisOutput to work with bundle-based scheduling. Added comprehensive metadata for dependency tracking, eligibility determination, and bundle classification. Inputs now carry reportBundles and noReportBundles with metadata describing which specific entities to process and their dependencies. INTEGRATION CHANGES (whole-program.cpp:432-935): Adapted whole-program analysis to work with the new scheduler: - Unified AnalyzeConstantsJob and regular analysis into templated AnalyzeJob<Mode> that works with bundles - Converted from per-entity job inputs to bundle-based inputs - Added final_pass() function (whole-program.cpp:837-897) to run the final analysis pass with no traces (immediate dependencies only) - Added emit_units() function (whole-program.cpp:900-935) to emit optimized unit emitters from final pass results This completes the main work of distributed hhbbc! The full functionality is gated by a compile-time only option. The default is the old behavior of doing analyze constants distributed, then doing the rest locally. The reason it is not on by default is that AnalysisIndex still needs to implement a few optimizations to be at parity with the old way. This avoids shipping regressions and will be short-lived. Reviewed By: mdko Differential Revision: D89634732 fbshipit-source-id: 6eceb97fcda760605422dda3c4a5ea9508425872
1 parent c673925 commit 7a46fc1

13 files changed

Lines changed: 4991 additions & 3958 deletions

hphp/hhbbc/debug.cpp

Lines changed: 14 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -317,18 +317,6 @@ void state_after(const char* when, const ParsedUnit& parsed) {
317317

318318
//////////////////////////////////////////////////////////////////////
319319

320-
namespace {
321-
322-
template <typename Clock>
323-
std::string ts(typename Clock::time_point t) {
324-
char snow[64];
325-
auto tm = Clock::to_time_t(t);
326-
ctime_r(&tm, snow);
327-
// Eliminate trailing newline from ctime_r.
328-
snow[24] = '\0';
329-
return snow;
330-
}
331-
332320
std::string format_bytes(size_t bytes) {
333321
auto s = folly::prettyPrint(
334322
bytes,
@@ -349,6 +337,20 @@ std::string format_duration(std::chrono::microseconds usecs) {
349337
return s;
350338
}
351339

340+
//////////////////////////////////////////////////////////////////////
341+
342+
namespace {
343+
344+
template <typename Clock>
345+
std::string ts(typename Clock::time_point t) {
346+
char snow[64];
347+
auto tm = Clock::to_time_t(t);
348+
ctime_r(&tm, snow);
349+
// Eliminate trailing newline from ctime_r.
350+
snow[24] = '\0';
351+
return snow;
352+
}
353+
352354
std::string client_stats(const extern_worker::Client::Stats& stats) {
353355
auto const pct = [] (size_t a, size_t b) -> std::string {
354356
if (!b) return "--";

0 commit comments

Comments
 (0)