-
Notifications
You must be signed in to change notification settings - Fork 504
Best Practices
Matthew Casperson edited this page Nov 18, 2024
·
8 revisions
This page is a collection of common snippets and practices to apply to step templates.
The properties defined against a step template have limited ability to define validation rules, so the scripts should validate these parameters before running any other logic.
$global:AzureAppConfigRetrievalMethod = $OctopusParameters["AzFunction.SetAppSettings.FromAzAppConfig.RetrievalMethod"]
if ([string]::IsNullOrWhiteSpace($global:AzureAppConfigRetrievalMethod)) {
throw "Required parameter AzFunction.SetAppSettings.FromAzAppConfig.RetrievalMethod not specified"
}
The script should validate that any required CLI tools are available:
function Test-ForAzCLI() {
$oldPreference = $ErrorActionPreference
$ErrorActionPreference = "Stop"
try {
return Get-Command "az"
}
catch {
return $false
}
finally {
$ErrorActionPreference = $oldPreference
}
}
Octopus prints all text stream to stderr as an error. This is arguably not the correct implementation, but isn't likely to be changed. This function allows a command to be run capturing all the output as a string:
$global:stdErr = [System.Text.StringBuilder]::new()
$global:myprocessrunning = $true
Function Invoke-CustomCommand
{
Param (
$commandPath,
$commandArguments,
$workingDir = (Get-Location),
$path = @()
)
$global:stdErr.Clear()
$global:myprocessrunning = $true
$path += $env:PATH
$newPath = $path -join [IO.Path]::PathSeparator
$pinfo = New-Object System.Diagnostics.ProcessStartInfo
$pinfo.FileName = $commandPath
$pinfo.WorkingDirectory = $workingDir
$pinfo.RedirectStandardError = $true
$pinfo.RedirectStandardOutput = $true
$pinfo.StandardOutputEncoding = [System.Text.Encoding]::UTF8
$pinfo.StandardErrorEncoding = [System.Text.Encoding]::UTF8
$pinfo.UseShellExecute = $false
$pinfo.Arguments = $commandArguments
$pinfo.EnvironmentVariables["PATH"] = $newPath
$p = New-Object System.Diagnostics.Process
# https://learn.microsoft.com/en-us/dotnet/api/system.diagnostics.process.standardoutput
# Reading from one stream must be async
# We read the error stream, because events can be handled out of order,
# and it is better to have this happen with debug output
Register-ObjectEvent -InputObject $p -EventName "ErrorDataReceived" -Action {
$global:stdErr.AppendLine($EventArgs.Data)
} | Out-Null
# We must wait for the Exited event rather than WaitForExit()
# because WaitForExit() can result in events being missed
# https://stackoverflow.com/questions/13113624/captured-output-of-command-run-by-powershell-is-sometimes-incomplete
Register-ObjectEvent -InputObject $p -EventName "Exited" -action {
$global:myprocessrunning = $false
} | Out-Null
$p.StartInfo = $pinfo
$p.Start() | Out-Null
$p.BeginErrorReadLine()
# Wait 10 minutes before forcibly killing the process
$processTimeout = 1000 * 60 * 10
while (($global:myprocessrunning -eq $true) -and ($processTimeout -gt 0))
{
# We must use lots of shorts sleeps rather than a single long one otherwise events are not processed
$processTimeout -= 50
Start-Sleep -m 50
}
$output = $p.StandardOutput.ReadToEnd()
if ($processTimeout -le 0)
{
$p.Kill()
}
$executionResults = [pscustomobject]@{
StdOut = $output
StdErr = $global:stdErr.ToString()
ExitCode = $p.ExitCode
}
return $executionResults
}
function Write-Results
{
[cmdletbinding()]
param (
[Parameter(Mandatory=$True,ValuefromPipeline=$True)]
$results
)
if (![String]::IsNullOrWhiteSpace($results.StdOut))
{
Write-Verbose $results.StdOut
}
}
Only later versions of PowerShell exposed variables $IsWindows and $IsLinux. This polyfill makes those variables available for all scripts:
if ($null -eq $IsWindows) {
Write-Host "Determining Operating System..."
$IsWindows = ([System.Environment]::OSVersion.Platform -eq "Win32NT")
$IsLinux = ([System.Environment]::OSVersion.Platform -eq "Unix")
}
Sometimes you want a trimmed string, or null if the string was null, because $null.Trim() will throw an error:
function Format-StringAsNullOrTrimmed {
[cmdletbinding()]
param (
[Parameter(ValuefromPipeline=$True)]
$input
)
if ([string]::IsNullOrWhitespace($input)) {
return $null
}
return $input.Trim()
}
Most scripts treat $null, empty strings, and whitespace strings as falsy. Unless you have a specific need to distinguish between these types of strings, prefer the use of the [string]::IsNullOrWhiteSpace() check over checks like $myString -eq "" or $myString -eq $null.
When a step creates output variables, print them to the log. This makes it easy to copy and paste the correct syntax:
$StepName = $OctopusParameters["Octopus.Step.Name"]
Write-Host "Created output variable: ##{Octopus.Action[$StepName].Output.AppSettingsJson}"
Get-LatestVersionDownloadUrl can be used to find the URL to download the latest release assets from a GitHub repo:
# Gets download url of latest release with an asset
Function Get-LatestVersionDownloadUrl {
# Define parameters
param(
$Repository,
$Version
)
# Define local variables
$releases = "https://api.github.com/repos/$Repository/releases"
# Get latest version
Write-Host "Determining latest release of $Repository ..."
$tags = (Invoke-WebRequest $releases -UseBasicParsing | ConvertFrom-Json)
if ($null -ne $Version) {
# Get specific version
$tags = ($tags | Where-Object { $_.name.EndsWith($Version) })
# Check to see if nothing was returned
if ($null -eq $tags) {
# Not found
Write-Host "No release found matching version $Version, getting highest version using Major.Minor syntax..."
# Get the tags
$tags = (Invoke-WebRequest $releases -UseBasicParsing | ConvertFrom-Json)
# Parse the version number into a version object
$parsedVersion = [System.Version]::Parse($Version)
$partialVersion = "$($parsedVersion.Major).$($parsedVersion.Minor)"
# Filter tags to ones matching only Major.Minor of version specified
$tags = ($tags | Where-Object { $_.name.Contains("$partialVersion.") -and $_.draft -eq $false })
# Grab the latest
if ($null -eq $tags)
{
# decrement minor version
$minorVersion = [int]$parsedVersion.Minor
$minorVersion --
# return the urls
return (Get-LatestVersionDownloadUrl -Repository $Repository -Version "$($parsedVersion.Major).$($minorVersion)")
}
}
}
# Find the latest version with a downloadable asset
foreach ($tag in $tags) {
if ($tag.assets.Count -gt 0) {
return $tag.assets.browser_download_url
}
}
# Return the version
return $null
}
This function fixes the <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)> error on Windows:
def fix_ssl():
# https://stackoverflow.com/a/63877194
if platform.system() == 'Windows':
try:
from urllib.request import HTTPSHandler
context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
context.options |= ssl.OP_NO_SSLv2
context.verify_mode = ssl.CERT_REQUIRED
context.load_verify_locations(certifi.where(), None)
https_handler = HTTPSHandler(context=context, check_hostname=True)
opener = urllib.request.build_opener(https_handler)
except ImportError:
opener = urllib.request.build_opener()
urllib.request.install_opener(opener)# If this script is not being run as part of an Octopus step, return variables from environment variables.
# Periods are replaced with underscores, and the variable name is converted to uppercase
if 'get_octopusvariable' not in globals():
def get_octopusvariable(variable):
return os.environ[re.sub('\\.', '_', variable.upper())]
# If this script is not being run as part of an Octopus step, print directly to std out.
if 'printverbose' not in globals():
def printverbose(msg):
print(msg)def get_octopusvariable_quiet(variable):
"""
Gets an octopus variable, or an empty string if it does not exist.
:param variable: The variable name
:return: The variable value, or an empty string if the variable does not exist
"""
try:
return get_octopusvariable(variable)
except:
return ''def printverbose_noansi(output):
"""
Strip ANSI color codes and print the output as verbose
:param output: The output to print
"""
output_no_ansi = re.sub(r'\x1b\[[0-9;]*m', '', output)
printverbose(output_no_ansi)def execute(args, cwd=None, env=None, print_args=None, print_output=printverbose_noansi):
"""
The execute method provides the ability to execute external processes while capturing and returning the
output to std err and std out and exit code.
"""
process = subprocess.Popen(args,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
cwd=cwd,
env=env)
stdout, stderr = process.communicate()
retcode = process.returncode
if print_args is not None:
print_output(' '.join(args))
if print_output is not None:
print_output(stdout)
print_output(stderr)
return stdout, stderr, retcode