-
Notifications
You must be signed in to change notification settings - Fork 7.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Call native operator #13068
Comments
I like the idea and find it useful, but think it would be far more useful (I'm thinking specifically in the case of a DSL) if I can get a single string expansion. So if I've built a cmd.exe compatible string in my PS, I have a way to string verbatim to cmd.exe. Perhaps
That would potentially also let me specify my interpreter:
in terms of the name/operator, is "&!" in use? That has similarity to shbang( "#!" ). |
The shell can be specified by simply specifying it as the command:
The $mycommand = "/bin/sh ls"
Invoke-Expression "&! $mycommand" However, in cases where you want to mix PowerShell with the native command, it's probably best to use the current syntax and just be aware of what needs to be escaped properly as I would consider that advanced usage. |
I can't believe you just suggested iex. :-) |
@essentialexch there are a few times where it's appropriate :) Note that in the example, expectation is that the user properly validated the contents of |
Great writeup and a solution for problems like the one you described are definitely needed. |
I agree
So this is the best way in case of conditional and whatnot for me |
@bergmeister I originally thought of |
@JustinGrote, there's no reason to not continue to use that. the |
Oh man, I remember having discussions with Bruce and Jason about this a decade ago. My gut feeling is that inventing another operator seems unnecessary here. I know that some people on this thread have not heard about # run ls in wsl, return results to powershell, then pipe to wsl again, to grep.
& wsl ls | wsl grep -i "foo"
# the entire pipeline right of wsl will run in wsl.
& --% wsl ls | grep -i "foo" Why hasn't anyone suggested this? It's logically consistent with usage of |
@oising well for one, that command doesn't work because you have to preface it with the command you want to run, are you suggesting adding the functionality? This kinda does what is expected:
|
@JustinGrote Yes, I know it doesn't work now :) I am suggesting that instead of I've expanded my example to show the differences. |
@oising I'd be cool with that over a new operator, though that is a lot of obtuse typing for "new powershell user who knows bash" which is what I assume this would be meant for. |
Not at all. It's for the PowerShell user that doesn't understand the complex quoting rules between PowerShell/cmd.exe/bash. As the issue title says "call native", for calling native executables. |
True, or for those of us who'd rather not have to think about them at all. |
Yeah I'm with @oising. This concept exists, it's just woefully insufficient. If we're going to implement something completely new we're better off deprecating/removing the old syntax. I feel oftentimes these ideas are voiced, not enough weight is given to their actual target demographic. If users are already struggling to find the existing methods, and finding the existing methods lacking when they are found, it's a call to a) improve documentation, and b) improve the actual functionality of the existing operators. Instead we get a weird option c) which is supposed to somehow address the past issues whilst introducing yet another operator which is relying on documentation which is already not accessible enough for users to find it naturally. I agree that error messages and/or the suggestions system should be used to help introduce these concepts. I don't agree that we need a third? fourth? fifth? (I've literally lost count, someone help me out here) way to invoke commands. The existing methods are insufficient -- that means we should improve them, not leave them behind like cobwebs to confuse users further when they start digging into them. The first question I tend to get when people realise there's 5 ways of doing something is "why are there 2-3 ways that literally do the same thing but don't work as well", and my answer is as it always has been -- the PS team is way too focused on making sure everything from the past decade++ still works, when trying to add/improve functionality. Here we see it again, we have existing but insufficient implementations that need revision and improvement, and the solution proposed is to further muddy the pool by adding another potentially incomplete implementation. We should be finishing what we start, not starting another implementation all over again, IMO. |
I believe if we are going to be using & for more than a few Unique use cases we should start thinking about making this considered a verb or look at standardization across the language. The last thing I would want is confusion about what & is meant to do. In its entirety. |
I'd consider 3 scenarios:
I wonder where @mklement0 comments? :-) |
At start of reading this thread I thought; great idea! But after reading comments it started me thinking. I have always found the & command 'not PS standards worthy" and something reminding me of 'PERL days' (ugh). Introducing another non-standard PS command (not being noun-verb) will not help. Certainly not those less savvy at PS. I did not know about the --% parameter either. I use the same principle as @JustinGrote as a solution. Cutting/pasting commands into PS shell has never been my main 'thing'. I vote for exploring improving existing commands. And - indeed - improving some documentation on --% and & usage @iSazonov the use case @SteveL-MSFT is proposing is the 'cut paste' scenario. Partial expansion would make things much more difficult I think (on the AST side of things) |
Immediately though of “#!” and then “!#”. Liking the “&!”. |
We can probably look at adding token accelerators to function calls in the tokenizer. This would allow for the use of the following. Iex -l I feel adding more unique tokens at the parsing level makes the barrier to entry go up and discoverability for this feature goes down. Get-Help & for example doesn't show up with anything. And we have to look for it in about_Operators. However there are unique cases for the call_operator that doesn't get documented. I believe whatever is decided on it needs to fully take into account how discoverable this "feature" is, from a new user perspective and keep in mind that newer users will try to use this on Powershell 5 not without knowing its new to pwsh core if the discoverability is too low. |
I'd love us to change the existing behaviour, but changing such a fundamental API in PowerShell would just break so many things. We could possibly consider migrating over with a configuration. Breaking things more directly might once have been possible before PS 6 went GA, but honestly, even then it would have been a serious threat to backward compatibility (not unlike Python's In terms of passing arguments to subprocesses, I think both synchronous invocation and For synchronous invocation with cmdline arg passing, I see four kinds of argument passing possible:
I think the first behaviour should be slowly phased out by introducing the second as an experimental feature and then swapping them. The second and third behaviours could have their own sigils like |
If the request is only to address copy-paste scenario like "Any text content after this operator will call into the "default shell" of the OS to execute." why do not say this explicitly by
|
I absolutely don't see how this would resolve #1995: See #1995 (comment) Also, the problem with WSL is a problem of the behavior of Oh, and isn't this almost a duplicate of #12975 ? Because I can essentially mostly repeat, what I mentioned there: @bitcrazedThe problem is that the lack of an ability to delimit a portion of a command-line to be passed varbatim to the receiving command/script is something that trips users up all the time. The only thing regarding "portion of a command-line to be passed varbatim to the receiving command/script", that would be improved with this operator would be meaning (2.) above, and as mentioned, I think developing in that direction is bad. If all you want to do is a shortcut do copy&paste existing command lines, why don't you add a cmdlet (e.g.
If it must be one line, then add one-line-here-string syntax, but please don't implement a special operator, that can't be properly found and just complicates everything. And please don't abandon #1995 for this! |
Indeed. We're talking about a cross-platform (in syntax only) mechanism for passing a command-line string to the platform-native shell. In a nutshell: Such a narrow use case does not call for an operator, but for a command - the previously discussed
In fact using the existing string literals, such as - if needed - a here-string is the appropriate thing to do - both conceptually and syntactically - no need to invent new syntax. After all, you're passing a string to an external program, which then interprets it according to its rules. Using strings is also the right choice for being able to use string interpolation, so as to be able to embed PowerShell variables and expressions in the command line being passed. The above solves all conceptual problems that would arise from shoehorning a different shell's syntax into PowerShell's syntax without proper delimiters, via an operator, as proposed here, which would invite perennial confusion and headaches. I do understand the desire to make this use case syntactically easier; the choices are:
|
Whatever decision is made about the operator, I think there should be a mode to enable globally (in my $profile for example), to make all the commands have this new operator that fixes the broken behavior implicitly included before them. This allows using PowerShell like Bash without the quoting issue. With the new mode enabled, I should be able to do this as I can in Bash: ❯ node -e 'console.log("good")'
good
❯ node -e "console.log('good')"
good
❯ $str="good"
❯ node -e "console.log('$str')"
good |
@aminya, what you're describing is an unrelated, general problem with how PowerShell passes arguments to external programs, discussed at length in #1995 (comment); the plan is to initially provide a fix in the form of an experimental feature ( |
Thanks. Also, it would be good to remove the need for esc operator when we want to interpolate the flags. Notice the need for esc $someFlag="--some-flag"
$esc = '--%'
script/build --first-flag $esc $someFlag In bash this becomes script/build --first-flag $someFlag |
@aminya, there's no reason to use However, I suggest keeping this issue focused on the original intent: offering a convenient way to execute commands written for the platform-native shell. |
This situation is so unfortunate for me, I need to fastly process the output for Please provide us with better ways to hook new I think if there was a simpler way to hook Meanwhile, I have a solution, just use my
|
You could look https://github.com/PowerShell/Crescendo. My alternative proposal partially is in #13428.
|
Wow, that's exactly what I needed, thank you.
|
I'm not so surprised by that, I kind of hate command line integrations between programs, seriously, calling a program by the command line and passing arguments as an array of strings that need to be parsed... But I agree with you that a tool like Crescendo for parsing metadata and making all command line tools have a standardized interface is absolutely needed, it should be a core module and very solid with the most used software, like Perhaps this Unix idea of everything being a text was not a good idea at all, powershell was supposed to solve that, it kind of does if you only use cmdlets... |
That idea has been superseded by the desktop bus. |
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you. |
2 similar comments
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you. |
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you. |
This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes. |
Problem Statement
Currently, there are cases where cutting and pasting a native command line fails to run as expected in PowerShell. This may be due to incorrect parsing of quotes meant to be passed to the native command or use of PowerShell syntax that is not meant to be interpreted as PowerShell. PowerShell has a
--%
special argument when used with native commands treats the rest of the arguments as literals passed to the native command, but has several issues:|
,&&
, and||
take precedence, so:wsl --% ls | less
would executewsl ls
and pipe the results toless
running in PowerShell rather thanless
running in wsl--%
towards the beginningProposed technical implementation details
Proposal is to introduce a new
--%
(Call Native) operator.Any text content after this operator will call into the "default shell" of the OS to execute. On Windows, this would be cmd.exe and on Unix-based systems this would be /bin/sh. This resolves the globbing issue on Unix-based systems, and also allow
%variable%
expansion on Windows. Unlike--%
switch, this also means that|
,&&
, and||
are treated as part of the native command line.This means that these two are functionally the same:
where
$foo
and$PWD
is evaluated by the shell within WSL. Note that in the first example, you would have to know to escape&&
to have it execute within WSL instead of within PowerShell.To pipe output from such execution back into PowerShell, the user is required to store the results into a variable first:
Note that unlike the current
&
call operator, you cannot use any PowerShell syntax, so:would not resolve
$commandline
as a variable first by PowerShell, but instead pass$commandline
to the default shell to process unresolved.The cut & paste problem is solved by simply pasting after
--%
is typed.The above example for wsl would look like:
where the intent is to have that whole line execute within the WSL Linux instance.
Discoverability
Users already familiar with
--%
as a switch may easily transition to using this new operator where it makes sense. For new users,--%
is unique so that search engines find it easily related to PowerShell.Alternate Considerations
&!
and&n
were proposed as the sigil, but there was push back because&!
is a valid operator in some languages making a web search for documentation more difficult. There was also a concern whether visual similar to&
call operator would be confusing to users.There is question about supporting line-continuation when using this new operator. I would suggest that we do not support it initially.
A cmdlet solution instead of an operator solution was proposed. We believe this doesn't solve the "cut & paste" problem as now you need to know to put the pipeline in single quotes and/or escape special characters. We do believe a cmdlet as in
Invoke-NativeCommand
(noun to be determined) would be useful as an additional option instead of replacing the need for an operator.Related issues
This should also solve these issues:
#12491
#1761
The text was updated successfully, but these errors were encountered: