-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add keyFilePath, emulatorHost and projectId as configuration parameters #6
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,73 +1,85 @@ | ||
Google Pub/Sub transport implementation for Symfony Messenger | ||
======== | ||
|
||
This bundle provides a simple implementation of Google Pub/Sub transport for Symfony Messenger. | ||
|
||
The bundle requires only `symfony/messenger`, `google/cloud-pubsub` and `symfony/options-resolver` packages. | ||
In contrast with [Enqueue GPS transport](https://github.com/php-enqueue/gps), | ||
it doesn't require [Enqueue](https://github.com/php-enqueue) | ||
and [some bridge](https://github.com/sroze/messenger-enqueue-transport#readme). | ||
It supports ordering messages with `OrderingKeyStamp` and it's not outdated. | ||
|
||
## Installation | ||
|
||
### Step 1: Install the Bundle | ||
|
||
From within container execute the following command to download the latest version of the bundle: | ||
|
||
```console | ||
$ composer require petitpress/gps-messenger-bundle --no-scripts | ||
``` | ||
|
||
### Step 2: Configure environment variables | ||
|
||
Official [Google Cloud PubSub SDK](https://github.com/googleapis/google-cloud-php-pubsub) | ||
requires some globally accessible environment variables. | ||
|
||
You might need to change default Symfony DotEnv instance to use `putenv` | ||
as Google needs to access some variables through `getenv`. To do so, use putenv method in `config/bootstrap.php`: | ||
```php | ||
(new Dotenv())->usePutenv()->... | ||
``` | ||
|
||
List of Google Pub/Sub configurable variables : | ||
```dotenv | ||
# use these for production environemnt: | ||
GOOGLE_APPLICATION_CREDENTIALS='google-pubsub-credentials.json' | ||
GCLOUD_PROJECT='project-id' | ||
|
||
# use these for development environemnt (if you have installed Pub/Sub emulator): | ||
PUBSUB_EMULATOR_HOST=http://localhost:8538 | ||
``` | ||
|
||
### Step 3: Configure Symfony Messenger | ||
```yaml | ||
# config/packages/messenger.yaml | ||
|
||
framework: | ||
messenger: | ||
transports: | ||
gps_transport: | ||
dsn: 'gps://default' | ||
options: | ||
max_messages_pull: 10 # optional (default: 10) | ||
topic: # optional (default name: messages) | ||
name: 'messages' | ||
queue: # optional (default the same as topic.name) | ||
name: 'messages' | ||
``` | ||
or: | ||
```yaml | ||
# config/packages/messenger.yaml | ||
|
||
framework: | ||
messenger: | ||
transports: | ||
gps_transport: | ||
dsn: 'gps://default/messages?max_messages_pull=10' | ||
``` | ||
|
||
### Step 4: Use available stamps if needed | ||
|
||
* `OrderingKeyStamp`: use for keeping messages of the same context in order. | ||
For more information, read an [official documentation](https://cloud.google.com/pubsub/docs/publisher#using_ordering_keys). | ||
Google Pub/Sub transport implementation for Symfony Messenger | ||
======== | ||
|
||
This bundle provides a simple implementation of Google Pub/Sub transport for Symfony Messenger. | ||
|
||
The bundle requires only `symfony/messenger`, `google/cloud-pubsub` and `symfony/options-resolver` packages. | ||
In contrast with [Enqueue GPS transport](https://github.com/php-enqueue/gps), | ||
it doesn't require [Enqueue](https://github.com/php-enqueue) | ||
and [some bridge](https://github.com/sroze/messenger-enqueue-transport#readme). | ||
It supports ordering messages with `OrderingKeyStamp` and it's not outdated. | ||
|
||
## Installation | ||
|
||
### Step 1: Install the Bundle | ||
|
||
From within container execute the following command to download the latest version of the bundle: | ||
|
||
```console | ||
$ composer require petitpress/gps-messenger-bundle --no-scripts | ||
``` | ||
|
||
### Step 2: Configure environment variables | ||
|
||
Official [Google Cloud PubSub SDK](https://github.com/googleapis/google-cloud-php-pubsub) | ||
requires some globally accessible environment variables. | ||
|
||
If you want to provide the PubSub authentication info through environment variables | ||
you might need to change default Symfony DotEnv instance to use `putenv` | ||
as Google needs to access some variables through `getenv`. To do so, use putenv method in `config/bootstrap.php` | ||
(does no longer exist in Symfony 5.3 and above): | ||
```php | ||
(new Dotenv())->usePutenv()->... | ||
``` | ||
|
||
List of Google Pub/Sub configurable variables : | ||
```dotenv | ||
# use these for production environemnt: | ||
GOOGLE_APPLICATION_CREDENTIALS='google-pubsub-credentials.json' | ||
GOOGLE_CLOUD_PROJECT='project-id' | ||
|
||
# use these for development environemnt (if you have installed Pub/Sub emulator): | ||
PUBSUB_EMULATOR_HOST=http://localhost:8538 | ||
``` | ||
|
||
If you want to use the bundle with Symfony Version 5.3 and above you need to configure those variables | ||
inside the `config/packages/messenger.yaml`. | ||
|
||
### Step 3: Configure Symfony Messenger | ||
```yaml | ||
# config/packages/messenger.yaml | ||
|
||
framework: | ||
messenger: | ||
transports: | ||
gps_transport: | ||
dsn: 'gps://default' | ||
options: | ||
max_messages_pull: 10 # optional (default: 10) | ||
topic: # optional (default name: messages) | ||
name: 'messages' | ||
queue: # optional (default the same as topic.name) | ||
name: 'messages' | ||
|
||
# optional (see google-cloud-php-pubsub documentation on GOOGLE_APPLICATION_CREDENTIALS) | ||
keyFilePath: '%env(GOOGLE_APPLICATION_CREDENTIALS)%' | ||
# optional (see google-cloud-php-pubsub documentation on PUBSUB_EMULATOR_HOST) | ||
emulatorHost: '%env(PUBSUB_EMULATOR_HOST)%' | ||
# mandatory (see google-cloud-php-pubsub documentation on GOOGLE_CLOUD_PROJECT) | ||
projectId: '%env(GOOGLE_CLOUD_PROJECT)%' | ||
``` | ||
or: | ||
```yaml | ||
# config/packages/messenger.yaml | ||
|
||
framework: | ||
messenger: | ||
transports: | ||
gps_transport: | ||
dsn: 'gps://default/messages?max_messages_pull=10' | ||
``` | ||
|
||
### Step 4: Use available stamps if needed | ||
|
||
* `OrderingKeyStamp`: use for keeping messages of the same context in order. | ||
For more information, read an [official documentation](https://cloud.google.com/pubsub/docs/publisher#using_ordering_keys). |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
{} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,94 @@ | ||
<?php | ||
|
||
namespace Transport; | ||
|
||
use Google\Cloud\Core\Exception\GoogleException; | ||
use PetitPress\GpsMessengerBundle\Transport\GpsConfigurationInterface; | ||
use PetitPress\GpsMessengerBundle\Transport\GpsConfigurationResolverInterface; | ||
use PetitPress\GpsMessengerBundle\Transport\GpsTransport; | ||
use PetitPress\GpsMessengerBundle\Transport\GpsTransportFactory; | ||
use PHPUnit\Framework\TestCase; | ||
use Prophecy\Argument; | ||
use Prophecy\PhpUnit\ProphecyTrait; | ||
use Prophecy\Prophecy\ObjectProphecy; | ||
use Symfony\Component\Messenger\Transport\Serialization\SerializerInterface; | ||
|
||
class GpsTransportFactoryTest extends TestCase | ||
{ | ||
use ProphecyTrait; | ||
|
||
private ObjectProphecy $serializerProphecy; | ||
private GpsTransportFactory $gpsTransportFactory; | ||
|
||
protected function setUp(): void | ||
{ | ||
parent::setUp(); | ||
|
||
$gpsConfigurtionProphecy = $this->prophesize(GpsConfigurationInterface::class); | ||
$gpsCongigurationResolverProphecy = $this->prophesize(GpsConfigurationResolverInterface::class); | ||
$this->serializerProphecy = $this->prophesize(SerializerInterface::class); | ||
|
||
$gpsCongigurationResolverProphecy->resolve(Argument::any(), Argument::any())->willReturn($gpsConfigurtionProphecy->reveal()); | ||
|
||
$this->gpsTransportFactory = new GpsTransportFactory($gpsCongigurationResolverProphecy->reveal()); | ||
} | ||
|
||
public function testCreateTransportFailsWithoutProjectId() | ||
{ | ||
$dsn = 'gps://'; | ||
$options = []; | ||
|
||
static::assertFalse(getenv(GpsTransportFactory::GOOGLE_CLOUD_PROJECT)); | ||
|
||
$this->expectException(GoogleException::class); | ||
|
||
$this->gpsTransportFactory->createTransport($dsn, $options, $this->serializerProphecy->reveal()); | ||
} | ||
|
||
public function testCreateTransportWithProjectIdFromEnvironmentVar() | ||
{ | ||
$dsn = 'gps://'; | ||
$options = []; | ||
|
||
putenv(GpsTransportFactory::GOOGLE_CLOUD_PROJECT . '=' . 'random'); | ||
|
||
$transport = $this->gpsTransportFactory->createTransport($dsn, $options, $this->serializerProphecy->reveal()); | ||
|
||
static::assertInstanceOf(GpsTransport::class, $transport); | ||
static::assertEquals('random', getenv(GpsTransportFactory::GOOGLE_CLOUD_PROJECT)); | ||
static::assertFalse(getenv(GpsTransportFactory::GOOGLE_APPLICATION_CREDENTIALS)); | ||
static::assertFalse(getenv(GpsTransportFactory::PUBSUB_EMULATOR_HOST)); | ||
} | ||
|
||
public function testCreateTransportWithProjectIdFromEnvironmentVarAndConfiguration() | ||
{ | ||
$dsn = 'gps://'; | ||
$options = ['projectId' => 'specfic']; | ||
|
||
putenv(GpsTransportFactory::GOOGLE_CLOUD_PROJECT . '=' . 'random'); | ||
|
||
$transport = $this->gpsTransportFactory->createTransport($dsn, $options, $this->serializerProphecy->reveal()); | ||
|
||
static::assertInstanceOf(GpsTransport::class, $transport); | ||
static::assertEquals('specfic', getenv(GpsTransportFactory::GOOGLE_CLOUD_PROJECT)); | ||
static::assertFalse(getenv(GpsTransportFactory::GOOGLE_APPLICATION_CREDENTIALS)); | ||
static::assertFalse(getenv(GpsTransportFactory::PUBSUB_EMULATOR_HOST)); | ||
} | ||
|
||
public function testCreateTransportWithEmulator() | ||
{ | ||
$dsn = 'gps://'; | ||
$options = [ | ||
'projectId' => 'random', | ||
'emulatorHost' => 'address://emulator/host', | ||
'keyFilePath' => __DIR__ . '/../Resources/credentials.json' | ||
]; | ||
|
||
$transport = $this->gpsTransportFactory->createTransport($dsn, $options, $this->serializerProphecy->reveal()); | ||
|
||
static::assertInstanceOf(GpsTransport::class, $transport); | ||
static::assertEquals(__DIR__ . '/../Resources/credentials.json', getenv(GpsTransportFactory::GOOGLE_APPLICATION_CREDENTIALS)); | ||
static::assertEquals('random', getenv(GpsTransportFactory::GOOGLE_CLOUD_PROJECT)); | ||
static::assertEquals('address://emulator/host', getenv(GpsTransportFactory::PUBSUB_EMULATOR_HOST)); | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,43 +1,67 @@ | ||
<?php | ||
declare(strict_types=1); | ||
namespace PetitPress\GpsMessengerBundle\Transport; | ||
<?php | ||
|
||
declare(strict_types=1); | ||
|
||
namespace PetitPress\GpsMessengerBundle\Transport; | ||
|
||
use Google\Cloud\PubSub\PubSubClient; | ||
use Symfony\Component\Messenger\Transport\Serialization\SerializerInterface; | ||
use Symfony\Component\Messenger\Transport\TransportFactoryInterface; | ||
use Symfony\Component\Messenger\Transport\TransportInterface; | ||
use Symfony\Component\Messenger\Transport\TransportInterface; | ||
|
||
/** | ||
* @author Ronald Marfoldi <[email protected]> | ||
*/ | ||
*/ | ||
final class GpsTransportFactory implements TransportFactoryInterface | ||
{ | ||
private GpsConfigurationResolverInterface $gpsConfigurationResolver; | ||
|
||
const GOOGLE_APPLICATION_CREDENTIALS = 'GOOGLE_APPLICATION_CREDENTIALS'; | ||
const GOOGLE_CLOUD_PROJECT = 'GOOGLE_CLOUD_PROJECT'; | ||
const PUBSUB_EMULATOR_HOST = 'PUBSUB_EMULATOR_HOST'; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you please use explicit constant access visibility? I also propose to rename these contants as it's not obvious what they represent. public const KEY_FILE_PATH_ENV_NAME = 'GOOGLE_APPLICATION_CREDENTIALS';
public const PROJECT_ID_ENV_NAME = 'GOOGLE_CLOUD_PROJECT';
public const EMULATOR_HOST_ENV_NAME = 'PUBSUB_EMULATOR_HOST'; |
||
|
||
private GpsConfigurationResolverInterface $gpsConfigurationResolver; | ||
|
||
public function __construct(GpsConfigurationResolverInterface $gpsConfigurationResolver) | ||
{ | ||
$this->gpsConfigurationResolver = $gpsConfigurationResolver; | ||
} | ||
} | ||
|
||
/** | ||
* {@inheritdoc} | ||
*/ | ||
*/ | ||
public function createTransport(string $dsn, array $options, SerializerInterface $serializer): TransportInterface | ||
{ | ||
$this->resolvePubSubEnvOptions($options); | ||
|
||
return new GpsTransport( | ||
new PubSubClient(), | ||
$this->gpsConfigurationResolver->resolve($dsn, $options), | ||
new PubSubClient(), | ||
$this->gpsConfigurationResolver->resolve($dsn, $options), | ||
$serializer | ||
); | ||
} | ||
|
||
} | ||
|
||
protected function resolvePubSubEnvOptions(array &$options): void | ||
{ | ||
$envMap = [ | ||
'projectId' => self::GOOGLE_CLOUD_PROJECT, | ||
'emulatorHost' => self::PUBSUB_EMULATOR_HOST, | ||
'keyFilePath' => self::GOOGLE_APPLICATION_CREDENTIALS | ||
]; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I propose to put it into private constant as: private const ENV_OPTIONS_MAP = [
'projectId' => self::PROJECT_ID_ENV_NAME,
'emulatorHost' => self::EMULATOR_HOST_ENV_NAME,
'keyFilePath' => self::KEY_FILE_PATH_ENV_NAME,
]; |
||
|
||
foreach ($envMap as $optKey => $envKey) { | ||
if (array_key_exists($optKey, $options)) { | ||
if (!empty($options[$optKey])) { | ||
putenv($envKey . '=' . $options[$optKey]); | ||
} | ||
unset($options[$optKey]); | ||
} | ||
} | ||
} | ||
|
||
/** | ||
* {@inheritdoc} | ||
*/ | ||
*/ | ||
public function supports(string $dsn, array $options): bool | ||
{ | ||
return str_starts_with($dsn, 'gps://'); | ||
} | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this whole part about using
->usePutenv
can be deleted. By this PR we are going to force use of putenv by resolving gps transport options and it's ok, because Google library can only work with globaly accessible env vars.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You don't need to use env vars anymore. You can pass all these options in when you instantiate the PubSubClient.
https://googleapis.github.io/google-cloud-php/#/docs/google-cloud/v0.172.0/pubsub/pubsubclient?method=__construct
This is pretty much what I did when I forked this project over to https://github.com/chrishemmings/gps-messenger-bundle