Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add keyFilePath, emulatorHost and projectId as configuration parameters #6

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 85 additions & 73 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,73 +1,85 @@
Google Pub/Sub transport implementation for Symfony Messenger
========

This bundle provides a simple implementation of Google Pub/Sub transport for Symfony Messenger.

The bundle requires only `symfony/messenger`, `google/cloud-pubsub` and `symfony/options-resolver` packages.
In contrast with [Enqueue GPS transport](https://github.com/php-enqueue/gps),
it doesn't require [Enqueue](https://github.com/php-enqueue)
and [some bridge](https://github.com/sroze/messenger-enqueue-transport#readme).
It supports ordering messages with `OrderingKeyStamp` and it's not outdated.

## Installation

### Step 1: Install the Bundle

From within container execute the following command to download the latest version of the bundle:

```console
$ composer require petitpress/gps-messenger-bundle --no-scripts
```

### Step 2: Configure environment variables

Official [Google Cloud PubSub SDK](https://github.com/googleapis/google-cloud-php-pubsub)
requires some globally accessible environment variables.

You might need to change default Symfony DotEnv instance to use `putenv`
as Google needs to access some variables through `getenv`. To do so, use putenv method in `config/bootstrap.php`:
```php
(new Dotenv())->usePutenv()->...
```

List of Google Pub/Sub configurable variables :
```dotenv
# use these for production environemnt:
GOOGLE_APPLICATION_CREDENTIALS='google-pubsub-credentials.json'
GCLOUD_PROJECT='project-id'

# use these for development environemnt (if you have installed Pub/Sub emulator):
PUBSUB_EMULATOR_HOST=http://localhost:8538
```

### Step 3: Configure Symfony Messenger
```yaml
# config/packages/messenger.yaml

framework:
messenger:
transports:
gps_transport:
dsn: 'gps://default'
options:
max_messages_pull: 10 # optional (default: 10)
topic: # optional (default name: messages)
name: 'messages'
queue: # optional (default the same as topic.name)
name: 'messages'
```
or:
```yaml
# config/packages/messenger.yaml

framework:
messenger:
transports:
gps_transport:
dsn: 'gps://default/messages?max_messages_pull=10'
```

### Step 4: Use available stamps if needed

* `OrderingKeyStamp`: use for keeping messages of the same context in order.
For more information, read an [official documentation](https://cloud.google.com/pubsub/docs/publisher#using_ordering_keys).
Google Pub/Sub transport implementation for Symfony Messenger
========

This bundle provides a simple implementation of Google Pub/Sub transport for Symfony Messenger.

The bundle requires only `symfony/messenger`, `google/cloud-pubsub` and `symfony/options-resolver` packages.
In contrast with [Enqueue GPS transport](https://github.com/php-enqueue/gps),
it doesn't require [Enqueue](https://github.com/php-enqueue)
and [some bridge](https://github.com/sroze/messenger-enqueue-transport#readme).
It supports ordering messages with `OrderingKeyStamp` and it's not outdated.

## Installation

### Step 1: Install the Bundle

From within container execute the following command to download the latest version of the bundle:

```console
$ composer require petitpress/gps-messenger-bundle --no-scripts
```

### Step 2: Configure environment variables

Official [Google Cloud PubSub SDK](https://github.com/googleapis/google-cloud-php-pubsub)
requires some globally accessible environment variables.

If you want to provide the PubSub authentication info through environment variables
you might need to change default Symfony DotEnv instance to use `putenv`
as Google needs to access some variables through `getenv`. To do so, use putenv method in `config/bootstrap.php`
(does no longer exist in Symfony 5.3 and above):
```php
(new Dotenv())->usePutenv()->...
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this whole part about using ->usePutenv can be deleted. By this PR we are going to force use of putenv by resolving gps transport options and it's ok, because Google library can only work with globaly accessible env vars.

Copy link

@chrishemmings chrishemmings Jan 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need to use env vars anymore. You can pass all these options in when you instantiate the PubSubClient.

https://googleapis.github.io/google-cloud-php/#/docs/google-cloud/v0.172.0/pubsub/pubsubclient?method=__construct

This is pretty much what I did when I forked this project over to https://github.com/chrishemmings/gps-messenger-bundle


List of Google Pub/Sub configurable variables :
```dotenv
# use these for production environemnt:
GOOGLE_APPLICATION_CREDENTIALS='google-pubsub-credentials.json'
GOOGLE_CLOUD_PROJECT='project-id'

# use these for development environemnt (if you have installed Pub/Sub emulator):
PUBSUB_EMULATOR_HOST=http://localhost:8538
```

If you want to use the bundle with Symfony Version 5.3 and above you need to configure those variables
inside the `config/packages/messenger.yaml`.

### Step 3: Configure Symfony Messenger
```yaml
# config/packages/messenger.yaml

framework:
messenger:
transports:
gps_transport:
dsn: 'gps://default'
options:
max_messages_pull: 10 # optional (default: 10)
topic: # optional (default name: messages)
name: 'messages'
queue: # optional (default the same as topic.name)
name: 'messages'

# optional (see google-cloud-php-pubsub documentation on GOOGLE_APPLICATION_CREDENTIALS)
keyFilePath: '%env(GOOGLE_APPLICATION_CREDENTIALS)%'
# optional (see google-cloud-php-pubsub documentation on PUBSUB_EMULATOR_HOST)
emulatorHost: '%env(PUBSUB_EMULATOR_HOST)%'
# mandatory (see google-cloud-php-pubsub documentation on GOOGLE_CLOUD_PROJECT)
projectId: '%env(GOOGLE_CLOUD_PROJECT)%'
```
or:
```yaml
# config/packages/messenger.yaml

framework:
messenger:
transports:
gps_transport:
dsn: 'gps://default/messages?max_messages_pull=10'
```

### Step 4: Use available stamps if needed

* `OrderingKeyStamp`: use for keeping messages of the same context in order.
For more information, read an [official documentation](https://cloud.google.com/pubsub/docs/publisher#using_ordering_keys).
1 change: 1 addition & 0 deletions Tests/Resources/credentials.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{}
94 changes: 94 additions & 0 deletions Tests/Transport/GpsTransportFactoryTest.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
<?php

namespace Transport;

use Google\Cloud\Core\Exception\GoogleException;
use PetitPress\GpsMessengerBundle\Transport\GpsConfigurationInterface;
use PetitPress\GpsMessengerBundle\Transport\GpsConfigurationResolverInterface;
use PetitPress\GpsMessengerBundle\Transport\GpsTransport;
use PetitPress\GpsMessengerBundle\Transport\GpsTransportFactory;
use PHPUnit\Framework\TestCase;
use Prophecy\Argument;
use Prophecy\PhpUnit\ProphecyTrait;
use Prophecy\Prophecy\ObjectProphecy;
use Symfony\Component\Messenger\Transport\Serialization\SerializerInterface;

class GpsTransportFactoryTest extends TestCase
{
use ProphecyTrait;

private ObjectProphecy $serializerProphecy;
private GpsTransportFactory $gpsTransportFactory;

protected function setUp(): void
{
parent::setUp();

$gpsConfigurtionProphecy = $this->prophesize(GpsConfigurationInterface::class);
$gpsCongigurationResolverProphecy = $this->prophesize(GpsConfigurationResolverInterface::class);
$this->serializerProphecy = $this->prophesize(SerializerInterface::class);

$gpsCongigurationResolverProphecy->resolve(Argument::any(), Argument::any())->willReturn($gpsConfigurtionProphecy->reveal());

$this->gpsTransportFactory = new GpsTransportFactory($gpsCongigurationResolverProphecy->reveal());
}

public function testCreateTransportFailsWithoutProjectId()
{
$dsn = 'gps://';
$options = [];

static::assertFalse(getenv(GpsTransportFactory::GOOGLE_CLOUD_PROJECT));

$this->expectException(GoogleException::class);

$this->gpsTransportFactory->createTransport($dsn, $options, $this->serializerProphecy->reveal());
}

public function testCreateTransportWithProjectIdFromEnvironmentVar()
{
$dsn = 'gps://';
$options = [];

putenv(GpsTransportFactory::GOOGLE_CLOUD_PROJECT . '=' . 'random');

$transport = $this->gpsTransportFactory->createTransport($dsn, $options, $this->serializerProphecy->reveal());

static::assertInstanceOf(GpsTransport::class, $transport);
static::assertEquals('random', getenv(GpsTransportFactory::GOOGLE_CLOUD_PROJECT));
static::assertFalse(getenv(GpsTransportFactory::GOOGLE_APPLICATION_CREDENTIALS));
static::assertFalse(getenv(GpsTransportFactory::PUBSUB_EMULATOR_HOST));
}

public function testCreateTransportWithProjectIdFromEnvironmentVarAndConfiguration()
{
$dsn = 'gps://';
$options = ['projectId' => 'specfic'];

putenv(GpsTransportFactory::GOOGLE_CLOUD_PROJECT . '=' . 'random');

$transport = $this->gpsTransportFactory->createTransport($dsn, $options, $this->serializerProphecy->reveal());

static::assertInstanceOf(GpsTransport::class, $transport);
static::assertEquals('specfic', getenv(GpsTransportFactory::GOOGLE_CLOUD_PROJECT));
static::assertFalse(getenv(GpsTransportFactory::GOOGLE_APPLICATION_CREDENTIALS));
static::assertFalse(getenv(GpsTransportFactory::PUBSUB_EMULATOR_HOST));
}

public function testCreateTransportWithEmulator()
{
$dsn = 'gps://';
$options = [
'projectId' => 'random',
'emulatorHost' => 'address://emulator/host',
'keyFilePath' => __DIR__ . '/../Resources/credentials.json'
];

$transport = $this->gpsTransportFactory->createTransport($dsn, $options, $this->serializerProphecy->reveal());

static::assertInstanceOf(GpsTransport::class, $transport);
static::assertEquals(__DIR__ . '/../Resources/credentials.json', getenv(GpsTransportFactory::GOOGLE_APPLICATION_CREDENTIALS));
static::assertEquals('random', getenv(GpsTransportFactory::GOOGLE_CLOUD_PROJECT));
static::assertEquals('address://emulator/host', getenv(GpsTransportFactory::PUBSUB_EMULATOR_HOST));
}
}
64 changes: 44 additions & 20 deletions Transport/GpsTransportFactory.php
Original file line number Diff line number Diff line change
@@ -1,43 +1,67 @@
<?php
declare(strict_types=1);
namespace PetitPress\GpsMessengerBundle\Transport;
<?php

declare(strict_types=1);

namespace PetitPress\GpsMessengerBundle\Transport;

use Google\Cloud\PubSub\PubSubClient;
use Symfony\Component\Messenger\Transport\Serialization\SerializerInterface;
use Symfony\Component\Messenger\Transport\TransportFactoryInterface;
use Symfony\Component\Messenger\Transport\TransportInterface;
use Symfony\Component\Messenger\Transport\TransportInterface;

/**
* @author Ronald Marfoldi <[email protected]>
*/
*/
final class GpsTransportFactory implements TransportFactoryInterface
{
private GpsConfigurationResolverInterface $gpsConfigurationResolver;

const GOOGLE_APPLICATION_CREDENTIALS = 'GOOGLE_APPLICATION_CREDENTIALS';
const GOOGLE_CLOUD_PROJECT = 'GOOGLE_CLOUD_PROJECT';
const PUBSUB_EMULATOR_HOST = 'PUBSUB_EMULATOR_HOST';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please use explicit constant access visibility? I also propose to rename these contants as it's not obvious what they represent.

public const KEY_FILE_PATH_ENV_NAME = 'GOOGLE_APPLICATION_CREDENTIALS';
public const PROJECT_ID_ENV_NAME = 'GOOGLE_CLOUD_PROJECT';
public const EMULATOR_HOST_ENV_NAME = 'PUBSUB_EMULATOR_HOST';


private GpsConfigurationResolverInterface $gpsConfigurationResolver;

public function __construct(GpsConfigurationResolverInterface $gpsConfigurationResolver)
{
$this->gpsConfigurationResolver = $gpsConfigurationResolver;
}
}

/**
* {@inheritdoc}
*/
*/
public function createTransport(string $dsn, array $options, SerializerInterface $serializer): TransportInterface
{
$this->resolvePubSubEnvOptions($options);

return new GpsTransport(
new PubSubClient(),
$this->gpsConfigurationResolver->resolve($dsn, $options),
new PubSubClient(),
$this->gpsConfigurationResolver->resolve($dsn, $options),
$serializer
);
}

}

protected function resolvePubSubEnvOptions(array &$options): void
{
$envMap = [
'projectId' => self::GOOGLE_CLOUD_PROJECT,
'emulatorHost' => self::PUBSUB_EMULATOR_HOST,
'keyFilePath' => self::GOOGLE_APPLICATION_CREDENTIALS
];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose to put it into private constant as:

private const ENV_OPTIONS_MAP = [
    'projectId' => self::PROJECT_ID_ENV_NAME,
    'emulatorHost' => self::EMULATOR_HOST_ENV_NAME,
    'keyFilePath' => self::KEY_FILE_PATH_ENV_NAME,
];


foreach ($envMap as $optKey => $envKey) {
if (array_key_exists($optKey, $options)) {
if (!empty($options[$optKey])) {
putenv($envKey . '=' . $options[$optKey]);
}
unset($options[$optKey]);
}
}
}

/**
* {@inheritdoc}
*/
*/
public function supports(string $dsn, array $options): bool
{
return str_starts_with($dsn, 'gps://');
}
}
}