[WIP, RFC] Use relative paths for `FILE` #1861

jakirkham · 2025-03-14T23:42:56Z

Description

Explore using GCC's -fmacro-prefix-map to remap __FILE__ from absolute paths to source directory relative paths.

Related to issue: rapidsai/build-planning#159

Checklist

I am familiar with the Contributing Guidelines.
New or existing tests cover these changes.
The documentation is up to date with these changes.

copy-pr-bot · 2025-03-14T23:42:59Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

jakirkham · 2025-03-14T23:43:40Z

/ok to test

jakirkham · 2025-03-15T00:19:43Z

The builds completed. Though CI is marked as errored. Not seeing a specific error in the logs. The only thing that sticks out are these warnings:

│ │ ⚠ warning Warnings:
 │ │ ⚠ warning Overdepending against cuda-cudart
 │ │ ⚠ warning Overdepending against __glibc

Are these warnings being treated as errors?

jakirkham · 2025-03-15T00:21:27Z

Ah maybe it is this error during the upload step where the tree command is not found?

/__w/_temp/51babaa4-efec-4e3b-b633-23f9d878e32a.sh: line 2: tree: command not found

Did something change in our GHA workflow? Or CI images?

jakirkham · 2025-03-15T00:23:12Z

Looks like this CI error will be fixed by this GHA workflow update: rapidsai/shared-workflows#299

Edit: We need to wait for the nightly job on branch-25.04 to rerurn. Then we can restart here

Edit 2: This is now cleared up. Turns out a new GHA run (instead of restarting) picks up the shared workflow fix

jakirkham · 2025-03-15T01:11:41Z

/ok to test

jakirkham

Added a few comments below providing more details about the changes

jakirkham · 2025-03-15T01:30:47Z

conda/recipes/rmm/recipe.yaml

+      # Remove `-fdebug-prefix-map` line from CFLAGS and CXXFLAGS so the
+      # incrementing version number in the compile line doesn't break the
+      # cache
+      set -x
+      export CFLAGS=$(echo $CFLAGS | sed -E 's@\-fdebug\-prefix\-map[^ ]*@@g')
+      export CXXFLAGS=$(echo $CXXFLAGS | sed -E 's@\-fdebug\-prefix\-map[^ ]*@@g')
+      set +x


Have copied the stripping of the existing -fdebug-prefix-map over from librmm:

rmm/conda/recipes/librmm/recipe.yaml

Lines 21 to 28 in f87d7a6

content: |

# Remove `-fdebug-prefix-map` line from CFLAGS and CXXFLAGS so the

# incrementing version number in the compile line doesn't break the

# cache

set -x

export CFLAGS=$(echo $CFLAGS | sed -E 's@\-fdebug\-prefix\-map[^ ]*@@g')

export CXXFLAGS=$(echo $CXXFLAGS | sed -E 's@\-fdebug\-prefix\-map[^ ]*@@g')

set +x

We would want this in both builds for the same reasons

jakirkham · 2025-03-15T01:30:58Z

conda/recipes/rmm/recipe.yaml

+      # Use relative paths for `__FILE__` by passing `-fmacro-prefix-map`
+      # https://gcc.gnu.org/onlinedocs/gcc-13.3.0/gcc/Preprocessor-Options.html#index-fmacro-prefix-map
+      set -x
+      export CFLAGS="${CFLAGS} -fmacro-prefix-map=$(pwd)/=''"
+      export CXXFLAGS="${CXXFLAGS} -fmacro-prefix-map=$(pwd)/=''"
+      set +x


As $(pwd) is the top-level directory of this repo, passing the -fmacro-prefix-map option simply removes the $(pwd) portion of the path in all __FILE__ expansions. Thus leaving us with paths relative to the top of the repo

A couple things we may want to consider:

Moving this into our CMake logic for all builds

Applying this behavior to more prefix mappings. -ffile-prefix-map would do this for all prefix-mapping cases GCC handles

Beyond fixing the __FILE__ expansion this gets us closer to reproducible builds. Also it should behave more robustly with caching (though we should test to confirm)

I like this approach. I would consider merging this as-is, and doing (1) or (2) as follow-up work.

I would also support this approach here. I think this is a good idea. I suggest that we try and put this into rapids-cmake so that we get this for free in all our libraries. The main challenge I see with that is identifying the appropriate root path to use. I agree with Bradley that I'm OK with doing that as a follow-up if you want to get this PR merged sooner. I can open a suitable rapids-cmake issue.

Opened rapidsai/rapids-cmake#798

Tried to describe what this would look like in the context of rapids-cmake: rapidsai/rapids-cmake#798 (comment)

Though am sure it can be improved further. Please feel free to suggest there

jakirkham · 2025-03-15T02:18:59Z

Put together an example here: conda/conda-build#1674 (comment)

bdice · 2025-03-17T16:14:00Z

This is really nice. I'm 100% supportive of this approach!

robertmaynard · 2025-03-20T20:20:49Z

conda/recipes/rmm/recipe.yaml

+      # cache
+      set -x
+      export CFLAGS=$(echo $CFLAGS | sed -E 's@\-fdebug\-prefix\-map[^ ]*@@g')
+      export CXXFLAGS=$(echo $CXXFLAGS | sed -E 's@\-fdebug\-prefix\-map[^ ]*@@g')


We should also export this in CUDAFLAGS I expect.

Agreed. Tried to capture a better recommendation here: rapidsai/rapids-cmake#798 (comment)

Please feel free to suggest further improvements

jakirkham · 2025-04-09T03:55:44Z

Closing in favor of addressing this in RAPIDS-CMake via PR: rapidsai/rapids-cmake#817

jakirkham added 2 commits March 14, 2025 16:37

Strip -fdebug-prefix-map from compiler flags

f990e97

Map __FILE__ to a relative path

96c5811

github-project-automation bot added this to RMM Project Board Mar 14, 2025

github-actions bot added the conda label Mar 14, 2025

jakirkham added non-breaking Non-breaking change improvement Improvement / enhancement to an existing function conda and removed conda labels Mar 14, 2025

jakirkham mentioned this pull request Mar 14, 2025

Replace uses of __FILE__ with __FILE_NAME__ in macros rapidsai/build-planning#159

Open

Fix script/content format

e2f1daf

jakirkham commented Mar 15, 2025

View reviewed changes

Merge branch 'branch-25.04' into use_rel_pth_prefix

4f67028

warrickhe mentioned this pull request Mar 17, 2025

Add Keep Option Parameter to Distinct rapidsai/cudf#18237

Merged

3 tasks

bdice assigned jakirkham Mar 18, 2025

vyasr mentioned this pull request Mar 18, 2025

[FEA] Set -ffile_prefix_map when in a conda-build rapidsai/rapids-cmake#798

Closed

robertmaynard requested changes Mar 20, 2025

View reviewed changes

github-project-automation bot moved this to Review in RMM Project Board Mar 20, 2025

jakirkham closed this Apr 9, 2025

github-project-automation bot moved this from Review to Done in RMM Project Board Apr 9, 2025

jakirkham deleted the use_rel_pth_prefix branch April 9, 2025 03:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP, RFC] Use relative paths for `FILE` #1861

[WIP, RFC] Use relative paths for `FILE` #1861

jakirkham commented Mar 14, 2025

copy-pr-bot bot commented Mar 14, 2025

jakirkham commented Mar 14, 2025

jakirkham commented Mar 15, 2025 •

edited

Loading

jakirkham commented Mar 15, 2025

jakirkham commented Mar 15, 2025 •

edited

Loading

jakirkham commented Mar 15, 2025

jakirkham left a comment

jakirkham Mar 15, 2025

jakirkham Mar 15, 2025

bdice Mar 17, 2025 •

edited

Loading

vyasr Mar 18, 2025

vyasr Mar 18, 2025

jakirkham Mar 28, 2025

jakirkham commented Mar 15, 2025

bdice commented Mar 17, 2025

robertmaynard Mar 20, 2025

jakirkham Mar 28, 2025

jakirkham commented Apr 9, 2025

	content: \|
	# Remove `-fdebug-prefix-map` line from CFLAGS and CXXFLAGS so the
	# incrementing version number in the compile line doesn't break the
	# cache
	set -x
	export CFLAGS=$(echo $CFLAGS \| sed -E 's@\-fdebug\-prefix\-map[^ ]*@@g')
	export CXXFLAGS=$(echo $CXXFLAGS \| sed -E 's@\-fdebug\-prefix\-map[^ ]*@@g')
	set +x

[WIP, RFC] Use relative paths for __FILE__ #1861

[WIP, RFC] Use relative paths for __FILE__ #1861

Conversation

jakirkham commented Mar 14, 2025

Description

Checklist

copy-pr-bot bot commented Mar 14, 2025

jakirkham commented Mar 14, 2025

jakirkham commented Mar 15, 2025 • edited Loading

jakirkham commented Mar 15, 2025

jakirkham commented Mar 15, 2025 • edited Loading

jakirkham commented Mar 15, 2025

jakirkham left a comment

Choose a reason for hiding this comment

jakirkham Mar 15, 2025

Choose a reason for hiding this comment

jakirkham Mar 15, 2025

Choose a reason for hiding this comment

bdice Mar 17, 2025 • edited Loading

Choose a reason for hiding this comment

vyasr Mar 18, 2025

Choose a reason for hiding this comment

vyasr Mar 18, 2025

Choose a reason for hiding this comment

jakirkham Mar 28, 2025

Choose a reason for hiding this comment

jakirkham commented Mar 15, 2025

bdice commented Mar 17, 2025

robertmaynard Mar 20, 2025

Choose a reason for hiding this comment

jakirkham Mar 28, 2025

Choose a reason for hiding this comment

jakirkham commented Apr 9, 2025

[WIP, RFC] Use relative paths for `FILE` #1861

[WIP, RFC] Use relative paths for `FILE` #1861

jakirkham commented Mar 15, 2025 •

edited

Loading

jakirkham commented Mar 15, 2025 •

edited

Loading

bdice Mar 17, 2025 •

edited

Loading