Commit 2067b58
Add ClassBundler infrastructure to HHBBC
Summary:
Introduces ClassBundler, which partitions classes, functions, and units into
bundles for distributed whole-program analysis. This is a critical component
for scaling HHBBC to large codebases.
Distributed analysis processes classes in parallel across multiple workers.
Without intelligent grouping, related classes would be scattered across
different workers, causing:
- Excessive remote data fetching when analyzing dependencies
- Poor cache locality (repeatedly loading the same remote data)
- Inefficient analysis (can't leverage local information about related code)
For example, if class A extends class B and both are analyzed by different
workers, the worker analyzing A must fetch B's information remotely every
time it's needed, rather than having it available locally.
ClassBundler uses a multi-phase algorithm to create balanced
bundles. It primarily uses the ApproximateNearestNeighbor class to
bundle together entities with similar dependencies. It includes
various size limits to avoid creating bundles that are too large.
The bundling decisions are stored in IndexData (classToBundle, funcToBundle,
unitToBundle maps) and used throughout distributed analysis to assign work
to workers and determine which bundles to load together.
This commit adds the ClassBundler infrastructure with comprehensive
documentation but does not yet integrate it into the build
process. That will come in a later diff since it has many
interdependencies with other changes.
Reviewed By: sinancepel
Differential Revision: D89633029
fbshipit-source-id: 45d9a4d4c97f5d96c6863486cc5c9ca2cdbb2a8c1 parent a1851b2 commit 2067b58
1 file changed
Lines changed: 885 additions & 0 deletions
0 commit comments