-
Notifications
You must be signed in to change notification settings - Fork 5.4k
feat: checksum manifest file in forc-pkg #7172
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
CodSpeed Performance ReportMerging #7172 will not alter performanceComparing Summary
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we need to check in an empty file?
.to_string_lossy() | ||
.to_string(); | ||
|
||
println!("adding {:?}", rel_path); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we need this for prod?
If so; should we use the tracing libs print function(s)?
|
||
println!("adding {:?}", rel_path); | ||
// Read file contents | ||
let content = std::fs::read(path).map_err(|e| { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Consider streaming each file into the hasher - instead of doing std::fs::read
.
This skips the full-file Vec<u8>
allocation and the extra copy into the hasher’s 64-byte buffer.
It barely matters for small projects, but on large workspaces the heap grows with total source size, so streaming can cut both memory and copy time.
Minimal example of streaming bytes from file:
fn hash_file(hasher: &mut Sha256, path: &std::path::Path) -> io::Result<()> {
let mut reader = BufReader::new(File::open(path)?); // handle to the file; not read into heap
let mut buf = [0u8; 8 * 1024]; // reusable 8 KiB stack buffer
loop {
let n = reader.read(&mut buf)?;
if n == 0 { break; }
hasher.update(&buf[..n]); // stream chunk directly
}
Ok(())
}
Note: Can be another PR however (GH issue).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM; thorough set of tests to validate functionality!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
reverting approval until minor comments (non-nits) are addressed*
reverting approval until minor comments are addressed*
Description
This is the first of possibly many PRs to get package level caching. I decided to split this into pieces after trying to get everything done at one go.
This one basically adds checksum implementation for workspace and package level manifest files. Taking the following into account:
.sw
files in thesrc
folderTo give an example how to use this in the LSP:
ChecksumGraph
in the forc-pkg side. This will enable us to pin-point which exact package is different.Once we are able to serialize the compiled packages, and the engines to the disk, this can be used a primitive form of incremental building.
TODO
ChecksumGraph
on forc-pkg side, and expose a function for consumers of forc-pkg to create thisChecksumGraph