Docker build not supported on windows: module 'os' has no attribute 'geteuid'

[boonew2]

I didn't see not supporting Windows called out anywhere in the readme

Repro steps
install:

pip install cloudformation-cli
pip install git+https://github.com/aws-cloudformation/aws-cloudformation-rpdk-python-plugin.git#egg=cloudformation-cli-python-plugin

init:

cfn init
Initializing new project
What's the name of your resource type?
(Organization::Service::Resource)
>> Windows::Bug::Repro
Select a language for code generation:
[1] python36
[2] python37
(enter an integer):
>> 2
Use docker for platform-independent packaging (Y/n)?
This is highly recommended unless you are experienced
with cross-platform Python packaging.
>> y
Initialized a new project in C:\temp\cfn-resource-repro

cp ..\cloudformation-cli-python-lib-0.0.1.tar.gz .\

cfn submit --dry-run
Starting Docker build. This may take several minutes if the image 'lambci/lambda:build-python3.7' needs to be pulled first.
=== Unhandled exception ===
Please report this issue to the team.
Issue tracker: https://github.com/aws-cloudformation/aws-cloudformation-rpdk/issues
Please include the log file 'rpdk.log'

[2020-06-15T15:52:09Z] DEBUG    - Logging set up successfully
[2020-06-15T15:52:09Z] DEBUG    - Running submit: Namespace(command=<function submit at 0x000001CD92DE19D0>, dry_run=True, endpoint_url=None, region=None, role_arn=None, set_default=False, subparser_name='submit', use_role=True, verbose=0, version=False)
[2020-06-15T15:52:09Z] DEBUG    - Root directory: C:\temp\cfn-resource-repro
[2020-06-15T15:52:09Z] DEBUG    - Loading project file 'C:\temp\cfn-resource-repro\.rpdk-config'
[2020-06-15T15:52:09Z] INFO     - Validating your resource specification...
[2020-06-15T15:52:09Z] DEBUG    - Resolved '#/definitions/InitechDateFormat' to 'file:///C:/temp/cfn-resource-repro/windows-bug-repro.json#/definitions/InitechDateFormat'
[2020-06-15T15:52:09Z] DEBUG    - Parsed 'file:///C:/temp/cfn-resource-repro/windows-bug-repro.json#/definitions/InitechDateFormat' to '(<BASE>, 'definitions', 'InitechDateFormat')'
[2020-06-15T15:52:09Z] DEBUG    - Edge from '(<BASE>, 'properties', 'DueDate')' to '(<BASE>, 'definitions', 'InitechDateFormat')'
[2020-06-15T15:52:09Z] DEBUG    - Resolved '#/definitions/InitechDateFormat' to 'file:///C:/temp/cfn-resource-repro/windows-bug-repro.json#/definitions/InitechDateFormat'
[2020-06-15T15:52:09Z] DEBUG    - Parsed 'file:///C:/temp/cfn-resource-repro/windows-bug-repro.json#/definitions/InitechDateFormat' to '(<BASE>, 'definitions', 'InitechDateFormat')'
[2020-06-15T15:52:09Z] DEBUG    - Edge from '(<BASE>, 'properties', 'ApprovalDate')' to '(<BASE>, 'definitions', 'InitechDateFormat')'
[2020-06-15T15:52:09Z] DEBUG    - Resolved '#/definitions/Memo' to 'file:///C:/temp/cfn-resource-repro/windows-bug-repro.json#/definitions/Memo'
[2020-06-15T15:52:09Z] DEBUG    - Parsed 'file:///C:/temp/cfn-resource-repro/windows-bug-repro.json#/definitions/Memo' to '(<BASE>, 'definitions', 'Memo')'
[2020-06-15T15:52:09Z] DEBUG    - Edge from '(<BASE>, 'properties', 'Memo')' to '(<BASE>, 'definitions', 'Memo')'
[2020-06-15T15:52:09Z] DEBUG    - Resolved '#/definitions/Memo' to 'file:///C:/temp/cfn-resource-repro/windows-bug-repro.json#/definitions/Memo'
[2020-06-15T15:52:09Z] DEBUG    - Parsed 'file:///C:/temp/cfn-resource-repro/windows-bug-repro.json#/definitions/Memo' to '(<BASE>, 'definitions', 'Memo')'
[2020-06-15T15:52:09Z] DEBUG    - Edge from '(<BASE>, 'properties', 'SecondCopyOfMemo')' to '(<BASE>, 'definitions', 'Memo')'
[2020-06-15T15:52:09Z] DEBUG    - Rewriting refs in '<BASE>' (file:///C:/temp/cfn-resource-repro/windows-bug-repro.json)[2020-06-15T15:52:09Z] DEBUG    -   '#/definitions/InitechDateFormat' -> '#/definitions/InitechDateFormat'
[2020-06-15T15:52:09Z] DEBUG    -   '#/definitions/InitechDateFormat' -> '#/definitions/InitechDateFormat'
[2020-06-15T15:52:09Z] DEBUG    -   '#/definitions/Memo' -> '#/definitions/Memo'
[2020-06-15T15:52:09Z] DEBUG    -   '#/definitions/Memo' -> '#/definitions/Memo'
[2020-06-15T15:52:09Z] DEBUG    - overrides.json not found. Not writing to package.
[2020-06-15T15:52:09Z] DEBUG    - Package started
[2020-06-15T15:52:09Z] DEBUG    - 'C:\temp\cfn-resource-repro\build' not found, skipping removal
Traceback (most recent call last):
  File "c:\python38\lib\site-packages\rpdk\python\codegen.py", line 192, in _remove_build_artifacts
    shutil.rmtree(deps_path)
  File "c:\python38\lib\shutil.py", line 737, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\python38\lib\shutil.py", line 596, in _rmtree_unsafe
    onerror(os.scandir, path, sys.exc_info())
  File "c:\python38\lib\shutil.py", line 593, in _rmtree_unsafe
    with os.scandir(path) as scandir_it:
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\temp\\cfn-resource-repro\\build'
[2020-06-15T15:52:09Z] DEBUG    - Dependencies build started from 'C:\temp\cfn-resource-repro'
[2020-06-15T15:52:09Z] DEBUG    - command is 'pip install --no-cache-dir --no-color --disable-pip-version-check --upgrade --find-links /project --requirement /project/requirements.txt --target /project/build'
[2020-06-15T15:52:09Z] WARNING  - Starting Docker build. This may take several minutes if the image 'lambci/lambda:build-python3.7' needs to be pulled first.
[2020-06-15T15:52:09Z] DEBUG    - Trying paths: ['C:\\Users\\user\\.docker\\config.json', 'C:\\Users\\user\\.dockercfg']
[2020-06-15T15:52:09Z] DEBUG    - Found file at path: C:\Users\user\.docker\config.json
[2020-06-15T15:52:09Z] DEBUG    - Found 'auths' section
[2020-06-15T15:52:09Z] DEBUG    - Auth data for xxxxxxxxxxx.dkr.ecr.us-west-1.amazonaws.com is absent. Client might be using a credentials store instead.
[2020-06-15T15:52:09Z] DEBUG    - Found 'credsStore' section
[2020-06-15T15:52:09Z] DEBUG    - Unhandled exception
Traceback (most recent call last):
  File "c:\python38\lib\site-packages\rpdk\core\cli.py", line 98, in main
    args.command(args)
  File "c:\python38\lib\site-packages\rpdk\core\submit.py", line 15, in submit
    project.submit(
  File "c:\python38\lib\site-packages\rpdk\core\project.py", line 336, in submit
    self._plugin.package(self, zip_file)
  File "c:\python38\lib\site-packages\rpdk\python\codegen.py", line 180, in package
    self._build(project.root)
  File "c:\python38\lib\site-packages\rpdk\python\codegen.py", line 199, in _build
    self._docker_build(base_path)
  File "c:\python38\lib\site-packages\rpdk\python\codegen.py", line 256, in _docker_build
    user=f"{os.geteuid()}:{os.getgid()}",
AttributeError: module 'os' has no attribute 'geteuid'

Problem line:

cloudformation-cli-python-plugin/python/rpdk/python/codegen.py

Line 256 in 20f4657

user=f"{os.geteuid()}:{os.getgid()}",

[richardhboyd]

oof, this was my code change.

[richardhboyd]

without that line the build fails on AL2/AL2012 instances.

[richardhboyd]

os.seteuid() and os.geteuid() methods are available only on UNIX platforms and functionality of os.seteuid() method is typically available only to the superuser as only superuser can change user id.

from here

hmmm. I'll need to think of a way around this. I wonder how the SAM CLI does this.

[ammokhov]

interesting but looks like a known issue - apache/mxnet#13759

[mdaehnert]

os.seteuid() and os.geteuid() methods are available only on UNIX platforms and functionality of os.seteuid() method is typically available only to the superuser as only superuser can change user id.

from here

hmmm. I'll need to think of a way around this. I wonder how the SAM CLI does this.

I had a look at sam-cli and found out, that they don't use "user=..." at all:
samcli/local/docker/container.py#L133

[richardhboyd]

if we remove the user setting, the files created end up being owned by 'root' and the current user can't make any changes to them. this doesn't seem to affect SAM because once the files are made, they're sent to S3 and the current user doesn't need to make any changes to them. I think that because SAM nests the files within a directory that the current user owns, the current user can delete the directory without a problem, but CFN CLI doesn't do the same nesting behavior.

[mdaehnert]

Ok. What's your preferred solution on that? As right now, it's locking out PC developers because of it. (Of course: Using a VM is possible, but not straightforward :-) )
Do you plan on changing the directory from temp to something like .build?

[giliam-v]

Is there any update on this?

This issue is making it impossible for me to author hooks on Windows.
If I disable docker build then the cfn submit succeeds, but the hook encounters an import failure during execution:

This happens even with the demo hook from AWS blog:

Unable to import module 'demo_testing_myfirsthook.handlers': cannot import name 'asn1' from 'cryptography.hazmat.bindings._rust' (unknown location)

The only way to make the hook work is to submit from a Linux machine.

[boonew2]

Yeah disappointing to come back two years later and get bit by the exact same thing :(

[Morgomor]

Any updates on the investigation? I really want to test this out and submit my own cfn hooks! With native Windows support, that would make it a lot more convenient for me and others :)

[mmaeng]

@Morgomor I've put in a PR which should resolve the issue #208 . Thanks for your report!

[mmaeng]

With #208 builds on Windows machines and any OS where os.geteuid() should get resolved. Part of the fix was a pip upgrade command, which was needed to get builds running using the Python 3.6 container image.

This update was breaking the build process on non-Windows machines. This upgrade is not needed is you are using Python 3.7 or higher on Windows. We will be deprecating Python 3.6 soon and we recommend upgrading.

If truly required, please

1. clone the this repository to your local workstation
2. restore the lines removed in fix(docker): pip upgrade command removed #218
3. pip install -e . from the local path from step 1

[mmaeng]

The released packages related to this issue
cloudformation-cli - 0.2.26
cloudformation-cli-python-plugin - 2.1.6
cloudformation-cli-python-lib - 2.1.13