Skip to content

Commit

Permalink
Always use -fmacro-prefix-map
Browse files Browse the repository at this point in the history
When `deterministic_paths` is set, we are currently using
`-ffile-prefix-map` to produce the same path in data and debug info. In
the case of absolute paths, their emscripten path is replaced with a
fake path `/emsdk/emscripten`, and in the case of relative paths, all
path relative to the emscripten directory is removed, so
`../../system/lib/somefile.c` becomes `system/lib/somefiles.c`.
https://github.com/emscripten-core/emscripten/blob/f66b5d706e174d9e5cc6122c06ea29dcd2735cd0/tools/system_libs.py#L472-L477
https://github.com/emscripten-core/emscripten/blob/f66b5d706e174d9e5cc6122c06ea29dcd2735cd0/tools/system_libs.py#L495-L501

But this does not make relative paths and absolute paths the same, which
can be a problem when data generated by `__FILE__` macro is included in
one of code size tests. This problem is discussed in emscripten-core#23195.

This PR makes `__FILE__` macro produce the same data in all cases by
using the fake path `/emsdk/emscripten` as its base, so that it wouldn't
change any results for code size tests. This is done by
`-fmacro-prefix-map`. This differs from the current behavior because we
don't handle relative and absolute paths differently.

For the debug info, when `deterministic_paths` is set, this uses a fake
path `/emsdk/emscripten` as a base emscripten path. When
`deterministic_paths` is not set, this uses real local absolute paths in
the debug info. This allows local developers to see their real paths in
the debug info while continuing to use the same (fake) path
`/emsdk/emscripten` we have used so far for the release binaries. Users
can set their debug base path to whatever path they like, but given that
we have used `/emsdk/emscripten` in release binaries for a while, it is
possible that some users have set their configuration with this
directory, so it would be better not to break them by changing it. This
is done by `-ffile-prefix-map` as we have done so far, which is an alias
for both `-fdebug-prefix-map` and `-fmacro-prefix-map`.

This is basically implementing what's suggested in
emscripten-core#23195 (comment)
and
emscripten-core#23195 (comment)

This also turns `deterministic_paths` on for the Ninja path in
embuilder for consistency with the non-Ninja path.

Fixes #23915.
  • Loading branch information
aheejin committed Dec 18, 2024
1 parent f66b5d7 commit 42986e0
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 14 deletions.
2 changes: 1 addition & 1 deletion embuilder.py
Original file line number Diff line number Diff line change
Expand Up @@ -291,7 +291,7 @@ def main():
library.erase()
if do_build:
if USE_NINJA:
library.generate()
library.generate(deterministic_paths=True)
else:
library.build(deterministic_paths=True)
elif what == 'sysroot':
Expand Down
33 changes: 20 additions & 13 deletions tools/system_libs.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@
# link time.
USE_NINJA = int(os.environ.get('EMCC_USE_NINJA', '0'))

FAKE_EMSCRIPTEN_PATH = '/emsdk/emscripten'


def files_in_path(path, filenames):
srcdir = utils.path_from_root(path)
Expand Down Expand Up @@ -427,8 +429,8 @@ def build(self, deterministic_paths=False):
self.deterministic_paths = deterministic_paths
return cache.get(self.get_path(), self.do_build, force=USE_NINJA == 2, quiet=USE_NINJA)

def generate(self):
self.deterministic_paths = False
def generate(self, deterministic_paths=False):
self.deterministic_paths = deterministic_paths
return cache.get(self.get_path(), self.do_generate, force=USE_NINJA == 2, quiet=USE_NINJA,
deferred=True)

Expand Down Expand Up @@ -469,12 +471,15 @@ def generate_ninja(self, build_dir, libname):
utils.safe_ensure_dirs(build_dir)

cflags = self.get_cflags()
source_dir = utils.path_from_root()
relative_source_dir = os.path.relpath(source_dir, build_dir)
if self.deterministic_paths:
source_dir = utils.path_from_root()
relative_source_dir = os.path.relpath(source_dir, build_dir)
cflags += [f'-ffile-prefix-map={source_dir}=/emsdk/emscripten',
f'-ffile-prefix-map={relative_source_dir}/=',
'-fdebug-compilation-dir=/emsdk/emscripten']
cflags += [f'-ffile-prefix-map={source_dir}={FAKE_EMSCRIPTEN_PATH}',
f'-ffile-prefix-map={relative_source_dir}={FAKE_EMSCRIPTEN_PATH}',
'-fdebug-compilation-dir={FAKE_EMSCRIPTEN_PATH}']
else:
cflags += [f'-fmacro-prefix-map={source_dir}={FAKE_EMSCRIPTEN_PATH}',
f'-fmacro-prefix-map={relative_source_dir}={FAKE_EMSCRIPTEN_PATH}']
asflags = get_base_cflags(preprocess=False)
input_files = self.get_files()
ninja_file = os.path.join(build_dir, 'build.ninja')
Expand All @@ -492,13 +497,15 @@ def build_objects(self, build_dir):
commands = []
objects = set()
cflags = self.get_cflags()
source_dir = utils.path_from_root()
relative_source_dir = os.path.relpath(source_dir, build_dir)
if self.deterministic_paths:
source_dir = utils.path_from_root()
if batch_inputs:
relative_source_dir = os.path.relpath(source_dir, build_dir)
cflags += [f'-ffile-prefix-map={relative_source_dir}/=']
cflags += [f'-ffile-prefix-map={source_dir}=/emsdk/emscripten',
'-fdebug-compilation-dir=/emsdk/emscripten']
cflags += [f'-ffile-prefix-map={relative_source_dir}={FAKE_EMSCRIPTEN_PATH}']
cflags += [f'-ffile-prefix-map={source_dir}={FAKE_EMSCRIPTEN_PATH}',
'-fdebug-compilation-dir={FAKE_EMSCRIPTEN_PATH}']
else:
cflags += [f'-fmacro-prefix-map={relative_source_dir}={FAKE_EMSCRIPTEN_PATH}']
cflags += [f'-fmacro-prefix-map={source_dir}={FAKE_EMSCRIPTEN_PATH}']
case_insensitive = is_case_insensitive(build_dir)
for src in self.get_files():
ext = shared.suffix(src)
Expand Down

0 comments on commit 42986e0

Please sign in to comment.