Document permissions needed by logging system test #2740

duggelz · 2016-11-15T21:19:09Z

The logging system test produces the following errors. As per @dhermes in #2669 the logging error is because some bucket or other object needs to add [email protected] to an ACL.

======================================================================
ERROR: test_create_sink_bigquery_dataset (logging_.TestLogging)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vmagent/app/google-cloud-python/system_tests/logging_.py", line 400, in test_create_sink_bigquery_dataset
    sink.create()
  File "/home/vmagent/app/google-cloud-python/.tox/system-tests/local/lib/python2.7/site-packages/google/cloud/logging/sink.py", line 119, in create
    self.project, self.name, self.filter_, self.destination)
  File "/home/vmagent/app/google-cloud-python/.tox/system-tests/local/lib/python2.7/site-packages/google/cloud/logging/_gax.py", line 210, in sink_create
    self._gax_api.create_sink(parent, sink_pb, options=options)
  File "/home/vmagent/app/google-cloud-python/.tox/system-tests/local/lib/python2.7/site-packages/google/cloud/gapic/logging/v2/config_service_v2_api.py", line 303, in create_sink
    return self._create_sink(request, options)
  File "/home/vmagent/app/google-cloud-python/.tox/system-tests/local/lib/python2.7/site-packages/google/gax/api_callable.py", line 481, in inner
    return api_caller(api_call, this_settings, request)
  File "/home/vmagent/app/google-cloud-python/.tox/system-tests/local/lib/python2.7/site-packages/google/gax/api_callable.py", line 469, in base_caller
    return api_call(*args)
  File "/home/vmagent/app/google-cloud-python/.tox/system-tests/local/lib/python2.7/site-packages/google/gax/api_callable.py", line 434, in inner
    errors.create_error('RPC failed', cause=exception))
  File "/home/vmagent/app/google-cloud-python/.tox/system-tests/local/lib/python2.7/site-packages/google/gax/api_callable.py", line 430, in inner
    return a_func(*args, **kwargs)
  File "/home/vmagent/app/google-cloud-python/.tox/system-tests/local/lib/python2.7/site-packages/google/gax/api_callable.py", line 64, in inner
    return a_func(*updated_args, **kwargs)
  File "/home/vmagent/app/google-cloud-python/.tox/system-tests/local/lib/python2.7/site-packages/grpc/_channel.py", line 481, in __call__
    return _end_unary_response_blocking(state, False, deadline)
  File "/home/vmagent/app/google-cloud-python/.tox/system-tests/local/lib/python2.7/site-packages/grpc/_channel.py", line 432, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
GaxError: GaxError(RPC failed, caused by <_Rendezvous of RPC that terminated with (StatusCode.PERMISSION_DENIED, The caller does not have permission)>)

======================================================================
ERROR: test_create_sink_pubsub_topic (logging_.TestLogging)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vmagent/app/google-cloud-python/system_tests/logging_.py", line 361, in test_create_sink_pubsub_topic
    policy = topic.get_iam_policy()
  File "/home/vmagent/app/google-cloud-python/.tox/system-tests/local/lib/python2.7/site-packages/google/cloud/pubsub/topic.py", line 340, in get_iam_policy
    resp = api.get_iam_policy(self.full_name)
  File "/home/vmagent/app/google-cloud-python/.tox/system-tests/local/lib/python2.7/site-packages/google/cloud/pubsub/_http.py", line 505, in get_iam_policy
    return conn.api_request(method='GET', path=path)
  File "/home/vmagent/app/google-cloud-python/.tox/system-tests/local/lib/python2.7/site-packages/google/cloud/_http.py", line 354, in api_request
    error_info=method + ' ' + url)
Forbidden: 403 User not authorized to perform this action. (GET https://pubsub.googleapis.com/v1/projects/cloud-python-runtime-qa/topics/logging-test-sink-1479244623664:getIamPolicy)

Command line was tox -e system-tests,system-tests3 -- logging
Test was run inside the gcr.io/google_appengine/python:2016-11-10_06_11 docker image.
Python versions were 2.7.9 and 3.4.2.
Cloud project name for testing was cloud-python-runtime-qa
More context here: https://github.com/GoogleCloudPlatform/python-runtime/blob/master/system_tests/Dockerfile

The text was updated successfully, but these errors were encountered:

tseaver · 2016-11-15T21:36:26Z

@duggelz I can't reproduce that failure here on master.

$ tox -e system-tests -- logging
GLOB sdist-make: /home/tseaver/projects/agendaless/Google/src/google-cloud-python/setup.py
system-tests recreate: /home/tseaver/projects/agendaless/Google/src/google-cloud-python/.tox/system-tests
system-tests installdeps: /home/tseaver/projects/agendaless/Google/src/google-cloud-python/core, /home/tseaver/projects/agendaless/Google/src/google-cloud-python/bigtable, /home/tseaver/projects/agendaless/Google/src/google-cloud-python/storage, /home/tseaver/projects/agendaless/Google/src/google-cloud-python/datastore, /home/tseaver/projects/agendaless/Google/src/google-cloud-python/bigquery, /home/tseaver/projects/agendaless/Google/src/google-cloud-python/pubsub, /home/tseaver/projects/agendaless/Google/src/google-cloud-python/logging, /home/tseaver/projects/agendaless/Google/src/google-cloud-python/dns, /home/tseaver/projects/agendaless/Google/src/google-cloud-python/language, /home/tseaver/projects/agendaless/Google/src/google-cloud-python/error_reporting, /home/tseaver/projects/agendaless/Google/src/google-cloud-python/resource_manager, /home/tseaver/projects/agendaless/Google/src/google-cloud-python/monitoring, /home/tseaver/projects/agendaless/Google/src/google-cloud-python/vision, /home/tseaver/projects/agendaless/Google/src/google-cloud-python/translate, /home/tseaver/projects/agendaless/Google/src/google-cloud-python/speech, /home/tseaver/projects/agendaless/Google/src/google-cloud-python/runtimeconfig, pytest
system-tests inst: /home/tseaver/projects/agendaless/Google/src/google-cloud-python/.tox/dist/google-cloud-0.20.0.zip
system-tests installed: -f file:///home/tseaver/.pip/wheels,enum34==1.1.6,future==0.16.0,futures==3.0.5,gapic-google-cloud-speech-v1beta1==0.11.1,gapic-google-logging-v2==0.10.1,gapic-google-pubsub-v1==0.10.1,google-cloud==0.20.0,google-cloud-bigquery==0.20.0,google-cloud-bigtable==0.20.0,google-cloud-core==0.20.0,google-cloud-datastore==0.20.1,google-cloud-dns==0.20.0,google-cloud-error-reporting==0.20.0,google-cloud-happybase==0.20.0,google-cloud-language==0.20.0,google-cloud-logging==0.20.0,google-cloud-monitoring==0.20.0,google-cloud-pubsub==0.20.0,google-cloud-resource-manager==0.20.0,google-cloud-runtimeconfig==0.20.0,google-cloud-speech==0.20.0.dev0,google-cloud-storage==0.20.0,google-cloud-translate==0.20.0,google-cloud-vision==0.20.0,google-gax==0.14.1,googleapis-common-protos==1.5.0,grpc-google-cloud-speech-v1beta1==0.11.1,grpc-google-iam-v1==0.10.1,grpc-google-logging-v2==0.10.1,grpc-google-pubsub-v1==0.10.1,grpcio==1.0.1,httplib2==0.9.2,oauth2client==3.0.0,ply==3.8,protobuf==3.1.0.post1,py==1.4.31,pyasn1==0.1.9,pyasn1-modules==0.0.8,pytest==3.0.4,rsa==3.4.2,six==1.10.0
system-tests runtests: PYTHONHASHSEED='858897741'
system-tests runtests: commands[0] | python /home/tseaver/projects/agendaless/Google/src/google-cloud-python/system_tests/attempt_system_tests.py logging
test_create_metric (logging_.TestLogging) ... ok
test_create_sink_bigquery_dataset (logging_.TestLogging) ... ok
test_create_sink_pubsub_topic (logging_.TestLogging) ... ok
test_create_sink_storage_bucket (logging_.TestLogging) ... ok
test_list_metrics (logging_.TestLogging) ... ok
test_list_sinks (logging_.TestLogging) ... ok       
test_log_handler_async (logging_.TestLogging) ... _has_entries. Trying again in 1 seconds...
_has_entries. Trying again in 2 seconds...
ok
test_log_handler_sync (logging_.TestLogging) ... _has_entries. Trying again in 1 seconds...
ok
test_log_root_handler (logging_.TestLogging) ... It was the best of times.
_has_entries. Trying again in 1 seconds...
_has_entries. Trying again in 2 seconds...
ok
test_log_struct (logging_.TestLogging) ... _has_entries. Trying again in 1 seconds...
_has_entries. Trying again in 2 seconds...
ok
test_log_struct_w_metadata (logging_.TestLogging) ... _has_entries. Trying again in 1 seconds...
ok
test_log_text (logging_.TestLogging) ... _has_entries. Trying again in 1 seconds...
ok
test_log_text_w_metadata (logging_.TestLogging) ... _has_entries. Trying again in 1 seconds...
_has_entries. Trying again in 2 seconds...
ok
test_reload_metric (logging_.TestLogging) ... ok
test_reload_sink (logging_.TestLogging) ... ok
test_update_metric (logging_.TestLogging) ... ok
test_update_sink (logging_.TestLogging) ... ok

----------------------------------------------------------------------
Ran 17 tests in 51.815s

OK

which doesn't surprise me, as test_create_sink_bigquery_dataset and test_create_sink_pubsub_topic are both careful to set up the correct permissions on their respective dataset / topic. In fact, your second failure is happening inside that setup for pubsub: it suggests that the user doesn't have permission to get / modify the IAM policy for the topic.

duggelz · 2016-11-15T21:44:36Z

Right, the problem is that I don't know what permissions are missing in my cloud-python-runtime-qa project that make this fail for me. Presumably whatever project you're running the system tests against is set up correctly.

daspecster · 2016-11-15T21:55:07Z

@duggelz are both bigquery and logging APIs enabled on console.cloud.google.com?

duggelz · 2016-11-15T21:59:04Z

[REDACTED]

theacodes · 2016-11-15T23:11:19Z

@duggelz I removed your comment as it contained names of private APIs.

dhermes · 2016-11-15T23:13:11Z

@duggelz Ping me on hangouts and we'll try to sort it out? I'd also recommend running with gRPC turned off and pdb activated:

$ tox -e system-tests --recreate --notest
$ source .tox/system-tests/bin/activate
(system-tests) $ GOOGLE_CLOUD_DISABLE_GRPC=true \
> .tox/system-tests/bin/py.test \
> system_tests/logging_.py --pdb

duggelz · 2016-11-15T23:52:59Z

CC @waprin who might know the answer.

More specific error message:

Forbidden: 403 The caller does not have permission (POST https://logging.googleapis.com/v2/projects/cloud-python-runtime-qa/sinks)

I'm trying to translate URL -> Thing I click on in the API library.

waprin · 2016-11-16T00:12:39Z

I believe you need to give either project owner, or "Logs Configuration Writer" role to the service account (or whatever client id you are using to run the test), and then give cloud-logs@google edit access to your BigQuery sink.

https://cloud.google.com/logging/docs/export/configure_export#setting_product_name_short_permissions_for_writing_exported_logs

waprin · 2016-11-16T00:15:13Z

The doc link I provided has the other permissions you need to setup for each of the sinks. If you're still stuck I will run through this all again on my own new project in a bit and see if I run into the same thing.

dhermes · 2016-11-16T00:22:59Z

@waprin We should add a check, maybe in setUpModule, to give a better error message in the system test when this isn't set up correctly.

duggelz · 2016-11-16T00:45:58Z

Thanks, that was very helpful, it is the service account that is the problem. I verified it works fine using my personal credentials.

However, how do I give [email protected] access to buckets when the bucket is created dynamically by the system test with a name like g-c-python-testing-1478122601994?

Just giving the service account and [email protected] both "Owner" and Logs Configuration Writer for the whole project doesn't appear to be sufficient.

duggelz · 2016-11-16T00:49:31Z

I guess I didn't wait long enough, I do have it working now. I'll see if I can prune the permissions back from Owner.

duggelz · 2016-11-16T01:24:51Z

The minimal set of permissions that work seem to be:

service account: Owner permissions to project
[email protected]: Editor permissions to the project

Requiring Owner is not great, but I couldn't find any lesser combination of permissions that worked.

So, the bug is now "please add this information, and information about the required APIs, to the instructions for running the system tests."

duggelz mentioned this issue Nov 15, 2016

Easier system test setup/teardown #2669

Closed

daspecster added the api: logging Issues related to the Cloud Logging API. label Nov 15, 2016

lukesneeringer added the priority: p2 Moderately-important priority. Fix may not be included in next release. label Apr 19, 2017

lukesneeringer mentioned this issue Aug 11, 2017

Accurately document system test permissions. #3795

Merged

lukesneeringer closed this as completed in #3795 Aug 11, 2017

JustinBeckwith assigned lukesneeringer Feb 1, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Document permissions needed by logging system test #2740

Document permissions needed by logging system test #2740

duggelz commented Nov 15, 2016

tseaver commented Nov 15, 2016

duggelz commented Nov 15, 2016

daspecster commented Nov 15, 2016

duggelz commented Nov 15, 2016 •

edited by theacodes

Loading

theacodes commented Nov 15, 2016

dhermes commented Nov 15, 2016

duggelz commented Nov 15, 2016

waprin commented Nov 16, 2016 •

edited

Loading

waprin commented Nov 16, 2016

dhermes commented Nov 16, 2016

duggelz commented Nov 16, 2016

duggelz commented Nov 16, 2016

duggelz commented Nov 16, 2016 •

edited

Loading

Document permissions needed by logging system test #2740

Document permissions needed by logging system test #2740

Comments

duggelz commented Nov 15, 2016

tseaver commented Nov 15, 2016

duggelz commented Nov 15, 2016

daspecster commented Nov 15, 2016

duggelz commented Nov 15, 2016 • edited by theacodes Loading

theacodes commented Nov 15, 2016

dhermes commented Nov 15, 2016

duggelz commented Nov 15, 2016

waprin commented Nov 16, 2016 • edited Loading

waprin commented Nov 16, 2016

dhermes commented Nov 16, 2016

duggelz commented Nov 16, 2016

duggelz commented Nov 16, 2016

duggelz commented Nov 16, 2016 • edited Loading

duggelz commented Nov 15, 2016 •

edited by theacodes

Loading

waprin commented Nov 16, 2016 •

edited

Loading

duggelz commented Nov 16, 2016 •

edited

Loading