Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add analogue to _concat for writable directories #4380

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions CHANGES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,20 @@ RELEASE VERSION/DATE TO BE FILLED IN LATER
- Use of NotImplemented instead of NotImplementedError for special methods
of _ListVariable class

From Mats Wichmann:
- Introduce a new _concat_dir function which can be used for
building cli strings when you need two sets of prefix/suffix
strings. _concat_dir transforms a list so the first item is wrapped
with the first pair of prefix/suffix args and the remaining
items with the second pair. The use case is when passing
a directory-to-write-to - those typically need to be unique,
but RDirs may expand to multiples, and we want to accomodate
also searching those directories in case you have repositories
setup - those could have already built files in them which could
be searched. Example: Fortran module directory, D interface file
directory. Introduces a new construction variable of the same name,
just to keep consistent with how existing _concat works.

RELEASE 4.6.0 - Sun, 19 Nov 2023 17:22:20 -0700

From Max Bachmann:
Expand Down Expand Up @@ -232,6 +246,11 @@ RELEASE 4.6.0 - Sun, 19 Nov 2023 17:22:20 -0700
GNU linker, the ':' in '-l:libfoo.a'). Fixes Github issue #3951.
- Update PCH builder docs with some usage notes.

From Jonathon Reinhart:
- Fix another instance of `int main()` in CheckLib() causing failures
when using -Wstrict-prototypes.


RELEASE 4.5.2 - Sun, 21 Mar 2023 14:08:29 -0700

From Michał Górny:
Expand Down
7 changes: 7 additions & 0 deletions RELEASE.txt
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,13 @@ FIXES

- Fix of the --debug=sconscript option to return exist statements when using return
statement with stop flag enabled
- When expanding $FORTRANMODDIR and $DI_FILE_DIR (the latter new in this
release), incorrect results could be obtained when Repository() has been
called, and/or when a variant directory is used with duplicate=False.
In some cases, this could cause the compiler to abort with an error.
The expansion is now handled so the path relative to the build directory
is passed to the compiler as the directory to write into, and the
additional directories are enabled as search directories.

IMPROVEMENTS
------------
Expand Down
62 changes: 62 additions & 0 deletions SCons/Defaults.py
Original file line number Diff line number Diff line change
Expand Up @@ -417,6 +417,67 @@ def _concat(prefix, items_iter, suffix, env, f=lambda x: x, target=None, source=
# pylint: enable-msg=too-many-arguments


# pylint: disable-msg=too-many-arguments
def _concat_dir(
prefix: str,
items,
suffix: str,
extra_prefix: str,
extra_suffix: str,
env,
f: Callable[[list], list] = lambda x: x,
target=None,
source=None,
affect_signature: bool = True,
) -> list:
"""Transform *items* into a list for command-line usage.

Like :func:`_concat`, but wraps an expanded list differently.
If *f* returns multiple items, the first item is wrapped by *prefix*
and *suffix*, while the rest are wrapped with *extra_prefix*
and *extra_suffix*. Used for special cases like Fortran module
directories, where only a single directory can be supplied as the
output destination, but repository locations and the source directory
in a variantdir setup should be added as search locations - which
takes a different syntax from different construction variables.

Args:
prefix: text to prepend to first element
items: string or iterable to transform
suffix: text to append to first element
extra_prefix: text to prepend to additional elements
extra_suffix: text to append to additional elements
env: construction environment for interpolation
f: optional function to perform a transformation on the list.
The default is a stub which performs no transformation.
target: optional target for interpolation into *items* elements
source: optional source for interpolation into *items* elements
affect_signature: optional flag: if false, surround contents with
markers indicating not to use the contents when calculating the
build signature. The default is ``True``.
"""
if not items:
return items

l = f(SCons.PathList.PathList(items).subst_path(env, target, source))
if l is not None:
items = l

if not affect_signature:
value = ['$(']
else:
value = []
value += _concat_ixes(prefix, items[:1], suffix, env)
if len(items) > 1:
value += _concat_ixes(extra_prefix, items[1:], extra_suffix, env)

if not affect_signature:
value += ["$)"]

return value
# pylint: enable-msg=too-many-arguments


def _concat_ixes(prefix, items_iter, suffix, env):
"""
Creates a new list from 'items_iter' by concatenating the 'prefix' and
Expand Down Expand Up @@ -718,6 +779,7 @@ def __lib_either_version_flag(env, version_var1, version_var2, flags_var):
'ENV': {},
'IDLSUFFIXES': SCons.Tool.IDLSuffixes,
'_concat': _concat,
'_concat_dir': _concat_dir,
'_defines': _defines,
'_stripixes': _stripixes,
'_LIBFLAGS': '${_concat(LIBLINKPREFIX, LIBS, LIBLINKSUFFIX, __env__)}',
Expand Down
110 changes: 110 additions & 0 deletions SCons/Defaults.xml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,116 @@ See its __doc__ string for a discussion of the format.
</summary>
</cvar>

<cvar name ="_concat_dir">
<summary>
<para>
A reference to a function which transforms
a value by concatenating passed prefix and suffix strings
(after variable substitution and optional additional processing),
to prepare option strings either to be passed to the external build program
or to help calculate the &buildsig; - the function will be called
for both purposes in the course of a build.
This allows values such as a module directory to be
to be stored in a portable manner,
rather than including non-portable compiler syntax.
Similar to &cv-link-_concat;, but for the special case where the
build program is being instructed to write to a directory.
For example, &cv-link-FORTRANMODDIR;
holds the name of the directory where the compiler will write
generated Fortran module files. Under some circumstances,
the processing by the specified function
can produce several names from the original variable
(if &f-link-Repository; and/or &f-link-VariantDir; are in use),
but the directory to write to must be unambiguous.
The &cv-_concat_dir; function wraps any additional directory
name strings in a different prefix/suffix pair,
which are passed in additional arguments to the function.
</para>

<para>
The default value for &cv-_concat_dir; is an internal &SCons; function.
It is possible to supply a custom &cv-_concat_dir; function,
but care must be taken not not disrupt the behavior
of existing &consvars; written to use the default function,
so replacing the default is discouraged.
</para>

<para>
A function used for &cv-_concat_dir; must accept six required arguments:
<parameter>prefix</parameter>,
<parameter>items</parameter>,
<parameter>suffix</parameter>,
<parameter>extra_prefix</parameter>,
<parameter>extra_suffix</parameter> and
<parameter>env</parameter>,
as well as four optional arguments:
<parameter>f</parameter>,
<parameter>target</parameter>,
<parameter>source</parameter> and
<parameter>affect_signature</parameter>.
<parameter>items</parameter>
is the element to proces -
often it will be the name of a &consvar;.
<parameter>prefix</parameter> and
<parameter>suffix</parameter>
are the strings to add to the front and back of the
first item of the list generated by calling the function
<parameter>f</parameter>.
<parameter>extra_prefix</parameter> and
<parameter>extra_suffix</parameter>
are the strings to add to the front and back of the remaining items.
<parameter>env</parameter>
is the &consenv; to interpolate &consvars; from;
the &consvars; will appear as local variables at evaluation time
and so are referenced without prefixing with a <literal>$</literal>.
The &cv-_concat_dir; function must return the substituted elements
as a string or list, either of which can be empty - it must not fail if
<parameter>items</parameter> has no value or has an empty value.
The optional arguments
<parameter>target</parameter> and
<parameter>source</parameter>
can be used if the target and/or source files
need to be interpolated into
<parameter>items</parameter>;
the special variables <literal>TARGET</literal>
and <literal>SOURCE</literal> are often used.
After source/target interpolation,
the function <parameter>f</parameter> is called.
It accepts a single list argument, and returns
a list with the necessary modifications applied.
<parameter>f</parameter> must be called before
the prefix/suffix additions, as it may change the
contents of the list.
&cv-link-RDirs; is often used as the function -
it may add additional directory paths if a repository
and/or variant directory is in use.
If the default internal implementation is used for &cv-_concat_dir;,
the default for <parameter>f</parameter> is a
pass-through which does not modify the item.
The optional <parameter>affect_signature</parameter>
indicates whether the contents should be used for
&buildsig; calculation. If it evaluates false,
any non-empty return should be wrapped with
<literal>$(</literal> and <literal>$)</literal>
to tell &SCons; to exclude it from the calculation.
If the default internal implementation is used for &cv-_concat;,
<parameter>affect_signature</parameter> defaults to
<constant>True</constant>.
</para>

<para>
Example use:
</para>

<example_commands>
env['_FORTRANMODFLAG'] = (
"$( ${_concat_dir(FORTRANMODDIRPREFIX, FORTRANMODDIR, FORTRANMODDIRSUFFIX, "
"INCFORTRANPREFIX, INCFORTRANSUFFIX, __env__, RDirs, TARGET, SOURCE)} $)"
)
</example_commands>
</summary>
</cvar>

<cvar name="CONFIGUREDIR">
<summary>
<para>
Expand Down
38 changes: 38 additions & 0 deletions SCons/EnvironmentTests.py
Original file line number Diff line number Diff line change
Expand Up @@ -1567,6 +1567,44 @@ def test__stripixes(self) -> None:
self.assertEqual('preasuf prebsuf prezzxxx-csuf', x)


def test_concat_dir(self) -> None:
"""Test _concat_dir()"""
e1 = self.TestEnvironment(
PRE='PP',
SUF='SS',
XPRE='XP',
XSUF='XS',
STR='a b',
LIST=['a', 'b'],
LONGLIST=['a', 'b', 'c', 'd'],
)
s = e1.subst
with self.subTest():
x = s("${_concat_dir('', '', '', '', '', __env__)}")
self.assertEqual(x, '')
with self.subTest():
x = s("${_concat_dir('', [], '', '', '', __env__)}")
self.assertEqual(x, '')
with self.subTest():
x = s("${_concat_dir(PRE, '', SUF, XPRE, XSUF, __env__)}")
self.assertEqual(x, '')
with self.subTest():
x = s("${_concat_dir(PRE, STR, SUF, XPRE, XSUF, __env__)}")
self.assertEqual(x, 'PPa bSS')
with self.subTest():
x = s("${_concat_dir(PRE, LIST, SUF, XPRE, XSUF, __env__)}")
self.assertEqual(x, 'PPaSS XPbXS')
with self.subTest():
x = s(
"${_concat_dir(PRE, LIST, SUF, XPRE, XSUF, __env__, affect_signature=False)}",
raw=True,
)
self.assertEqual(x, '$( PPaSS XPbXS $)')
with self.subTest():
x = s("${_concat_dir(PRE, LONGLIST, SUF, XPRE, XSUF, __env__)}")
self.assertEqual(x, 'PPaSS XPbXS XPcXS XPdXS')


def test_gvars(self) -> None:
"""Test the Environment gvars() method"""
env = self.TestEnvironment(XXX = 'x', YYY = 'y', ZZZ = 'z')
Expand Down
2 changes: 1 addition & 1 deletion SCons/Tool/FortranCommon.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ def add_fortran_to_env(env) -> None:
env['FORTRANMODDIR'] = '' # where the compiler should place .mod files
env['FORTRANMODDIRPREFIX'] = '' # some prefix to $FORTRANMODDIR - similar to $INCPREFIX
env['FORTRANMODDIRSUFFIX'] = '' # some suffix to $FORTRANMODDIR - similar to $INCSUFFIX
env['_FORTRANMODFLAG'] = '$( ${_concat(FORTRANMODDIRPREFIX, FORTRANMODDIR, FORTRANMODDIRSUFFIX, __env__, RDirs, TARGET, SOURCE)} $)'
env['_FORTRANMODFLAG'] = '$( ${_concat_dir(FORTRANMODDIRPREFIX, FORTRANMODDIR, FORTRANMODDIRSUFFIX, INCFORTRANPREFIX, INCFORTRANSUFFIX, __env__, RDirs, TARGET, SOURCE)} $)'

def add_f77_to_env(env) -> None:
"""Add Builders and construction variables for f77 dialect."""
Expand Down
4 changes: 1 addition & 3 deletions SCons/Tool/dmd.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,9 +95,7 @@ def generate(env) -> None:
env['_DINCFLAGS'] = '${_concat(DINCPREFIX, DPATH, DINCSUFFIX, __env__, RDirs, TARGET, SOURCE)}'
env['_DVERFLAGS'] = '${_concat(DVERPREFIX, DVERSIONS, DVERSUFFIX, __env__)}'
env['_DDEBUGFLAGS'] = '${_concat(DDEBUGPREFIX, DDEBUG, DDEBUGSUFFIX, __env__)}'

env['_DI_FLAGS'] = "${DI_FILE_DIR and DI_FILE_DIR_PREFIX+DI_FILE_DIR+DI_FILE_DIR_SUFFFIX}"

env['_DI_FLAGS'] = '$( ${_concat_dir(DI_FILE_DIR_PREFIX, DI_FILE_DIR, DI_FILE_DIR_SUFFFIX, DINCPREFIX, DINCSUFFIX, __env__, RDirs, TARGET, SOURCE)} $)'
env['_DFLAGS'] = '${_concat(DFLAGPREFIX, DFLAGS, DFLAGSUFFIX, __env__)}'

env['SHDC'] = '$DC'
Expand Down
2 changes: 1 addition & 1 deletion SCons/Tool/gdc.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ def generate(env) -> None:
env['_DINCFLAGS'] = '${_concat(DINCPREFIX, DPATH, DINCSUFFIX, __env__, RDirs, TARGET, SOURCE)}'
env['_DVERFLAGS'] = '${_concat(DVERPREFIX, DVERSIONS, DVERSUFFIX, __env__)}'
env['_DDEBUGFLAGS'] = '${_concat(DDEBUGPREFIX, DDEBUG, DDEBUGSUFFIX, __env__)}'
env['_DI_FLAGS'] = "${DI_FILE_DIR and DI_FILE_DIR_PREFIX+DI_FILE_DIR+DI_FILE_DIR_SUFFFIX}"
env['_DI_FLAGS'] = '$( ${_concat_dir(DI_FILE_DIR_PREFIX, DI_FILE_DIR, DI_FILE_DIR_SUFFFIX, DINCPREFIX, DINCSUFFIX, __env__, RDirs, TARGET, SOURCE)} $)'
env['_DFLAGS'] = '${_concat(DFLAGPREFIX, DFLAGS, DFLAGSUFFIX, __env__)}'

env['SHDC'] = '$DC'
Expand Down
4 changes: 1 addition & 3 deletions SCons/Tool/ldc.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,9 +71,7 @@ def generate(env) -> None:
env['_DINCFLAGS'] = '${_concat(DINCPREFIX, DPATH, DINCSUFFIX, __env__, RDirs, TARGET, SOURCE)}'
env['_DVERFLAGS'] = '${_concat(DVERPREFIX, DVERSIONS, DVERSUFFIX, __env__)}'
env['_DDEBUGFLAGS'] = '${_concat(DDEBUGPREFIX, DDEBUG, DDEBUGSUFFIX, __env__)}'

env['_DI_FLAGS'] = "${DI_FILE_DIR and DI_FILE_DIR_PREFIX+DI_FILE_DIR+DI_FILE_DIR_SUFFFIX}"

env['_DI_FLAGS'] = '$( ${_concat_dir(DI_FILE_DIR_PREFIX, DI_FILE_DIR, DI_FILE_DIR_SUFFFIX, DINCPREFIX, DINCSUFFIX, __env__, RDirs, TARGET, SOURCE)} $)'
env['_DFLAGS'] = '${_concat(DFLAGPREFIX, DFLAGS, DFLAGSUFFIX, __env__)}'

env['SHDC'] = '$DC'
Expand Down
Loading