Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with example in #11573 #11578

Closed
3 tasks done
sethvs opened this issue Dec 11, 2024 · 0 comments · Fixed by #11595
Closed
3 tasks done

Issue with example in #11573 #11578

sethvs opened this issue Dec 11, 2024 · 0 comments · Fixed by #11595

Comments

@sethvs
Copy link
Contributor

sethvs commented Dec 11, 2024

Prerequisites

  • Existing Issue: Search the existing issues for this repository. If there is an issue that fits your needs do not file a new one. Subscribe, react, or comment on that issue instead.
  • Descriptive Title: Write the title for this issue as a short synopsis. If possible, provide context. For example, "Typo in Get-Foo cmdlet" instead of "Typo."
  • Verify Version: If there is a mismatch between documentation and the behavior on your system, ensure that the version you are using is the same as the documentation. Check this box if they match or the issue you are reporting is not version specific.

Links

#11573

Summary

I believe that this example
https://github.com/sdwheeler/PowerShell-Docs/blob/b00ec932fc27ee5d1f6f512d19a3c60a46f537d8/reference/7.5/Microsoft.PowerShell.Core/About/about_Type_Conversion.md?plain=1#L403-L418
is not fully correct.

Result of the [Int64]::MaxValue + 1L is of type [Double].

PS C:\> ([Int64]::MaxValue + 1L).GetType().FullName
System.Double

So, when we convert it to [Decimal] it is rounded to 15 significant digits.

As in the Remarks section of the Convert.ToDecimal Method:
The Decimal value returned by this method contains a maximum of 15 significant digits. If the value parameter contains more than 15 significant digits, it is rounded using rounding to nearest. The following example illustrates how the Convert.ToDecimal(Double) method uses rounding to nearest to return a Decimal value with 15 significant digits.

So, either we add L to 1 or not, the result will be the same:

PS C:\> ([Int64]::MaxValue + 1L).GetType().FullName
System.Double

PS C:\> [decimal]([Int64]::MaxValue + 1L)
9223372036854780000
PS C:\> ([Int64]::MaxValue + 1).GetType().FullName
System.Double

PS C:\> [decimal]([Int64]::MaxValue + 1)
9223372036854780000

On the other hand, if we leave the result as [Double], adding L to 1 will also not change the result:

PS C:\> '{0:N0}' -f ([Int64]::MaxValue + 1L)
9,223,372,036,854,775,808

PS C:\> '{0:N0}' -f ([Int64]::MaxValue + 1)
9,223,372,036,854,775,808

Details

No response

Suggested Fix

No response

@sethvs sethvs added issue-doc-bug Issue - error in documentation needs-triage Waiting - Needs triage labels Dec 11, 2024
@sdwheeler sdwheeler removed issue-doc-bug Issue - error in documentation needs-triage Waiting - Needs triage labels Dec 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants