Skip to content

Commit

Permalink
Use consistent prefix map when building deterministic system libraries (
Browse files Browse the repository at this point in the history
emscripten-core#23222)

When `deterministic_paths` is set, we are currently using
`-ffile-prefix-map` to produce the same path in data and debug info. In
the case of absolute paths, their emscripten path is replaced with a
fake path `/emsdk/emscripten`, and in the case of relative paths, all
path relative to the emscripten directory is removed, so
`../../system/lib/somefile.c` becomes `system/lib/somefiles.c`.
https://github.com/emscripten-core/emscripten/blob/f66b5d706e174d9e5cc6122c06ea29dcd2735cd0/tools/system_libs.py#L472-L477
https://github.com/emscripten-core/emscripten/blob/f66b5d706e174d9e5cc6122c06ea29dcd2735cd0/tools/system_libs.py#L495-L501

So this does not make relative paths and absolute paths the same, which
can lead to different builds depending on whether the command line uses
absolute paths vs. relative ones. Currently we use relative paths when
`EMCC_BATCH_BUILD` is set. And Ninja builds cannot use relative paths.

This is also what was suggested in
emscripten-core#23195 (comment)
while discussins `__FILE__` problem in emscripten-core#23195.

This does not change any code size tests because no code size tests
happens to contain `__FILE__` at the moment. (One will be added in
emscripten-core#22994)
  • Loading branch information
aheejin authored Dec 19, 2024
1 parent 0186a3f commit 971d4f6
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 7 deletions.
2 changes: 1 addition & 1 deletion test/test_other.py
Original file line number Diff line number Diff line change
Expand Up @@ -10409,7 +10409,7 @@ def test_dwarf_system_lib(self):
[LLVM_DWARFDUMP, libc, '-debug-info', '-debug-line', '--recurse-depth=0'],
stdout=PIPE).stdout
# Check that the embedded location of the source file is correct.
self.assertIn('DW_AT_name\t("system/lib/emmalloc.c")', dwdump)
self.assertIn('DW_AT_name\t("/emsdk/emscripten/system/lib/emmalloc.c")', dwdump)
self.assertIn('DW_AT_comp_dir\t("/emsdk/emscripten")', dwdump)

@parameterized({
Expand Down
16 changes: 10 additions & 6 deletions tools/system_libs.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,10 @@
# link time.
USE_NINJA = int(os.environ.get('EMCC_USE_NINJA', '0'))

# A (fake) deterministic emscripten path to use in __FILE__ macro and debug info
# to produce reproducible builds across platforms.
DETERMINISITIC_PREFIX = '/emsdk/emscripten'


def files_in_path(path, filenames):
srcdir = utils.path_from_root(path)
Expand Down Expand Up @@ -472,9 +476,9 @@ def generate_ninja(self, build_dir, libname):
if self.deterministic_paths:
source_dir = utils.path_from_root()
relative_source_dir = os.path.relpath(source_dir, build_dir)
cflags += [f'-ffile-prefix-map={source_dir}=/emsdk/emscripten',
f'-ffile-prefix-map={relative_source_dir}/=',
'-fdebug-compilation-dir=/emsdk/emscripten']
cflags += [f'-ffile-prefix-map={source_dir}={DETERMINISITIC_PREFIX}',
f'-ffile-prefix-map={relative_source_dir}={DETERMINISITIC_PREFIX}',
f'-fdebug-compilation-dir={DETERMINISITIC_PREFIX}']
asflags = get_base_cflags(preprocess=False)
input_files = self.get_files()
ninja_file = os.path.join(build_dir, 'build.ninja')
Expand All @@ -496,9 +500,9 @@ def build_objects(self, build_dir):
source_dir = utils.path_from_root()
if batch_inputs:
relative_source_dir = os.path.relpath(source_dir, build_dir)
cflags += [f'-ffile-prefix-map={relative_source_dir}/=']
cflags += [f'-ffile-prefix-map={source_dir}=/emsdk/emscripten',
'-fdebug-compilation-dir=/emsdk/emscripten']
cflags += [f'-ffile-prefix-map={relative_source_dir}={DETERMINISITIC_PREFIX}']
cflags += [f'-ffile-prefix-map={source_dir}={DETERMINISITIC_PREFIX}',
f'-fdebug-compilation-dir={DETERMINISITIC_PREFIX}']
case_insensitive = is_case_insensitive(build_dir)
for src in self.get_files():
ext = shared.suffix(src)
Expand Down

0 comments on commit 971d4f6

Please sign in to comment.