Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: edge sdks should send events to bulk/environment endpoint #256

Merged
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
a2dbfe4
feat: implement useEnvironmentEndpoint logic by inspecting the sdk-ke…
Aug 26, 2023
cada2f6
fix: broken tests.
Aug 26, 2023
37bdbf4
fix: wrong events path for clientSideID
Aug 26, 2023
cfbc545
fix: added shell directive. added link-dev for cloudflare/example to …
Aug 28, 2023
2874194
chore: remove debug statements.
Aug 28, 2023
8bcd58a
Merge branch 'main' into yus/sc-214758/send-events-to-environment-end…
yusinto Aug 28, 2023
1908227
chore: better support for event sender analytics paths (#259)
yusinto Aug 28, 2023
11cb96a
chore: rollback unit test sdk- prefix changes.
yusinto Aug 28, 2023
a3d3132
chor: add comments
yusinto Aug 28, 2023
429e92a
Update packages/shared/sdk-server-edge/src/api/LDClient.ts
yusinto Aug 28, 2023
4741775
Merge branch 'main' into yus/sc-214758/send-events-to-environment-end…
Aug 29, 2023
f871209
chore: add release-please alpha config.
Aug 29, 2023
37ebde7
Update release-please-config.json
Aug 29, 2023
ad7d012
chore: flush events in waitUntil. improved handler types.
yusinto Aug 30, 2023
303e3ae
Merge branch 'main' into yus/sc-214758/send-events-to-environment-end…
yusinto Sep 21, 2023
44c8044
fix: remove link-dev and use yarn workspaces.
yusinto Sep 22, 2023
2abeb13
fix: unit tests due to missing ctx. update jest version.
yusinto Sep 22, 2023
8f1ad45
chore: fix flush callback arg types.
yusinto Sep 22, 2023
8b5c0d9
fix: close client to fix jest open handle warnings
yusinto Sep 22, 2023
6cb2165
Merge branch 'main' into yus/sc-214758/send-events-to-environment-end…
yusinto Nov 7, 2023
aae16d8
chore: resolve conflicts.
yusinto Nov 7, 2023
2acad70
chore: in cloudflare example, rename sdkKey to clientSideID.
yusinto Nov 7, 2023
5da020e
chore: added unit tests for edge sdk.
yusinto Nov 8, 2023
474cdf7
chore: update cloudflare sdk dep version
yusinto Nov 8, 2023
55bf9af
chore: set test data trackEvents to true
yusinto Nov 8, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions link-dev.sh
yusinto marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/bash

echo "===== Linking to cloudflare/example"

rm -rf packages/sdk/cloudflare/example/node_modules/@launchdarkly/js-sdk-common/dist
cp -r packages/shared/common/dist packages/sdk/cloudflare/example/node_modules/@launchdarkly/js-sdk-common/dist

rm -rf packages/sdk/cloudflare/example/node_modules/@launchdarkly/js-server-sdk-common/dist
cp -r packages/shared/sdk-server/dist packages/sdk/cloudflare/example/node_modules/@launchdarkly/js-server-sdk-common/dist

rm -rf packages/sdk/cloudflare/example/node_modules/@launchdarkly/js-server-sdk-common-edge/dist
cp -r packages/shared/sdk-server-edge/dist packages/sdk/cloudflare/example/node_modules/@launchdarkly/js-server-sdk-common-edge/dist

rm -rf packages/sdk/cloudflare/example/node_modules/@launchdarkly/cloudflare-server-sdk/dist
cp -r packages/sdk/cloudflare/dist packages/sdk/cloudflare/example/node_modules/@launchdarkly/cloudflare-server-sdk/dist
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
"packages/shared/akamai-edgeworker-sdk",
"packages/sdk/server-node",
"packages/sdk/cloudflare",
"packages/sdk/cloudflare/example",
"packages/sdk/vercel",
"packages/sdk/akamai-base",
"packages/sdk/akamai-base/example",
Expand All @@ -30,7 +29,8 @@
"contract-test-harness": "curl -s https://raw.githubusercontent.com/launchdarkly/sdk-test-harness/master/downloader/run.sh \\ | VERSION=v2 PARAMS=\"-url http://localhost:8000 -debug -stop-service-at-end $TEST_HARNESS_PARAMS\" sh",
"contract-tests": "npm run contract-test-service & npm run contract-test-harness",
"prettier": "npx prettier --write \"**/*.{js,ts,tsx,json,yaml,yml,md}\" --log-level warn",
"check": "yarn && yarn prettier && yarn lint && tsc && yarn build"
"check": "yarn && yarn prettier && yarn lint && tsc && yarn build",
"link-dev": "./link-dev.sh"
},
"devDependencies": {
"@trivago/prettier-plugin-sort-imports": "^4.1.1",
Expand Down
6 changes: 4 additions & 2 deletions packages/sdk/cloudflare/example/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,11 @@
"wrangler": "2.13.0"
},
"scripts": {
"build": "node build.js",
"build": "yarn && yarn link-dev && node build.js",
"start": "wrangler dev",
"deploy": "wrangler publish",
"test": "yarn build && node --experimental-vm-modules --no-warnings node_modules/jest/bin/jest.js"
"test": "yarn build && node --experimental-vm-modules --no-warnings node_modules/jest/bin/jest.js",
"clean": "rm -rf dist && rm -rf node_modules && rm -rf .yarn/cache && yarn build",
"link-dev": "cd ../../../../ && yarn link-dev"
}
}
14 changes: 10 additions & 4 deletions packages/sdk/cloudflare/example/src/index.ts
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To the Vercel and/or Akamai examples also need to be updated? I initially thought this change only applied to Cloudflare because only the Cloudflare example was changed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those examples can be updated, but not necessary. I'll leave that to @ldhenry .

Original file line number Diff line number Diff line change
@@ -1,24 +1,30 @@
/* eslint-disable no-console */
import { init as initLD } from '@launchdarkly/cloudflare-server-sdk';

export default {
async fetch(request: Request, env: Bindings): Promise<Response> {
ldhenry marked this conversation as resolved.
Show resolved Hide resolved
const sdkKey = 'test-sdk-key';
const clientSideID = 'client-side-id';
const flagKey = 'testFlag1';
const context = { kind: 'user', key: 'test-user-key-1', email: 'test@gmail.com' };
const context = { kind: 'org', key: 'org-key-cf', email: 'testcforg@gmail.com' };

// start using ld
const client = initLD(sdkKey, env.LD_KV);
const client = initLD(clientSideID, env.LD_KV, { sendEvents: true });
await client.waitForInitialization();
const flagValue = await client.variation(flagKey, context, false);
const flagDetail = await client.variationDetail(flagKey, context, false);
const allFlags = await client.allFlagsState(context);

// Gotcha: you must call flush otherwise events will not be sent to LD servers
// due to the ephemeral nature of edge workers.
await client.flush((err, res) => {
ldhenry marked this conversation as resolved.
Show resolved Hide resolved
console.log(`flushed events result: ${res}, error: ${err}`);
});

const resp = `
${flagKey}: ${flagValue}
detail: ${JSON.stringify(flagDetail)}
allFlags: ${JSON.stringify(allFlags)}`;

// eslint-disable-next-line
console.log(`------------- ${resp}`);
return new Response(`${resp}`);
},
Expand Down
12 changes: 6 additions & 6 deletions packages/sdk/cloudflare/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,20 +35,20 @@ export type { LDClient };
* (`new LDClient()/new LDClientImpl()/new LDClient()`); the SDK does not currently support
* this.
*
* @param clientSideID
* The client side ID. This is only used to query the kvNamespace above,
* not to connect with LaunchDarkly servers.
* @param kvNamespace
* The Cloudflare KV configured for LaunchDarkly.
* @param sdkKey
* The client side SDK key. This is only used to query the kvNamespace above,
* not to connect with LaunchDarkly servers.
* @param options
* Optional configuration settings. The only supported option is logger.
* @return
* The new {@link LDClient} instance.
*/
export const init = (sdkKey: string, kvNamespace: KVNamespace, options: LDOptions = {}) => {
export const init = (clientSideID: string, kvNamespace: KVNamespace, options: LDOptions = {}) => {
const logger = options.logger ?? BasicLogger.get();
return initEdge(sdkKey, createPlatformInfo(), {
featureStore: new EdgeFeatureStore(kvNamespace, sdkKey, 'Cloudflare', logger),
return initEdge(clientSideID, createPlatformInfo(), {
featureStore: new EdgeFeatureStore(kvNamespace, clientSideID, 'Cloudflare', logger),
logger,
...options,
});
Expand Down
15 changes: 15 additions & 0 deletions packages/shared/common/src/internal/events/LDInternalOptions.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
/**
* This is for internal use only.
*
* Edge sdks use clientSideID to query feature stores. They also send analytics
* using this clientSideID. This is a hybrid behavior because they are based
* on js-server-common, but uses the clientSideID instead of the sdkKey for the
* above reasons. These internal options allow the edge sdks to use the
* EventSender to send analytics to the correct LD endpoints using
* the clientSideId.
*/
export type LDInternalOptions = {
analyticsEventPath?: string;
diagnosticEventPath?: string;
includeAuthorizationHeader?: boolean;
};
10 changes: 9 additions & 1 deletion packages/shared/common/src/internal/events/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,13 @@ import InputCustomEvent from './InputCustomEvent';
import InputEvalEvent from './InputEvalEvent';
import InputEvent from './InputEvent';
import InputIdentifyEvent from './InputIdentifyEvent';
import type { LDInternalOptions } from './LDInternalOptions';

export { InputCustomEvent, InputEvalEvent, InputEvent, InputIdentifyEvent, EventProcessor };
export {
EventProcessor,
InputCustomEvent,
InputEvalEvent,
InputEvent,
InputIdentifyEvent,
LDInternalOptions,
};
29 changes: 28 additions & 1 deletion packages/shared/common/src/options/ServiceEndpoints.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,36 @@ export default class ServiceEndpoints {

public readonly events: string;

public constructor(streaming: string, polling: string, events: string) {
/** Valid paths are:
* /bulk
* /events/bulk/envId
* /mobile
*/
public readonly analyticsEventPath: string;

/** Valid paths are:
* /diagnostic
* /events/diagnostic/envId
* /mobile/events/diagnostic
*/
public readonly diagnosticEventPath: string;

// if true the sdk key will be included as authorization header
public readonly includeAuthorizationHeader: boolean;

public constructor(
streaming: string,
polling: string,
events: string,
analyticsEventPath: string = '/bulk',
diagnosticEventPath: string = '/diagnostic',
includeAuthorizationHeader: boolean = true,
) {
this.streaming = canonicalizeUri(streaming);
this.polling = canonicalizeUri(polling);
this.events = canonicalizeUri(events);
this.analyticsEventPath = analyticsEventPath;
this.diagnosticEventPath = diagnosticEventPath;
this.includeAuthorizationHeader = includeAuthorizationHeader;
}
}
13 changes: 9 additions & 4 deletions packages/shared/sdk-server-edge/src/api/LDClient.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { EventEmitter } from 'node:events';

import { Info, LDClientImpl, LDOptions } from '@launchdarkly/js-server-sdk-common';
import { Info, internal, LDClientImpl, LDOptions } from '@launchdarkly/js-server-sdk-common';

import EdgePlatform from '../platform';
import createCallbacks from './createCallbacks';
Expand All @@ -12,11 +12,16 @@ import createOptions from './createOptions';
export class LDClient extends LDClientImpl {
emitter: EventEmitter;

// sdkKey is only used to query featureStore, not to initialize with LD servers
constructor(sdkKey: string, platformInfo: Info, options: LDOptions) {
// clientSideID is only used to query the edge key-value store and send analytics, not to initialize with LD servers
constructor(clientSideID: string, platformInfo: Info, options: LDOptions) {
const em = new EventEmitter();
const platform = new EdgePlatform(platformInfo);
super('n/a', platform, createOptions(options), createCallbacks(em));
const internalOptions: internal.LDInternalOptions = {
analyticsEventPath: `/events/bulk/${clientSideID}`,
diagnosticEventPath: `/events/diagnostic/${clientSideID}`,
includeAuthorizationHeader: false,
};
super(clientSideID, platform, createOptions(options), createCallbacks(em), internalOptions);
this.emitter = em;
}
}
Expand Down
6 changes: 3 additions & 3 deletions packages/shared/sdk-server-edge/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ export type { LDClient, LDOptions, EdgeProvider };
*
* This is an internal API to be used directly only by LaunchDarkly Edge SDKs.
*/
export const init = (sdkKey: string, platformInfo: Info, options: LDOptionsInternal) => {
export const init = (clientSideID: string, platformInfo: Info, options: LDOptionsInternal) => {
// this throws if options are invalid
validateOptions(sdkKey, options);
validateOptions(clientSideID, options);

return new LDClient(sdkKey, platformInfo, options);
return new LDClient(clientSideID, platformInfo, options);
};
3 changes: 2 additions & 1 deletion packages/shared/sdk-server/src/LDClientImpl.ts
Original file line number Diff line number Diff line change
Expand Up @@ -120,13 +120,14 @@ export default class LDClientImpl implements LDClient {
private platform: Platform,
options: LDOptions,
callbacks: LDClientCallbacks,
internalOptions?: internal.LDInternalOptions,
) {
this.onError = callbacks.onError;
this.onFailed = callbacks.onFailed;
this.onReady = callbacks.onReady;

const { onUpdate, hasEventListeners } = callbacks;
const config = new Configuration(options);
const config = new Configuration(options, internalOptions);
if (!sdkKey && !config.offline) {
throw new Error('You must configure the client with an SDK key');
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,21 @@ export default function defaultHeaders(
sdkKey: string,
config: DefaultHeaderOptions,
info: Info,
includeAuthorizationHeader: boolean = true,
): { [key: string]: string } {
const sdkData = info.sdkData();
const headers: { [key: string]: string } = {
authorization: sdkKey,
'user-agent': `${sdkData.userAgentBase ? sdkData.userAgentBase : 'NodeJSClient'}/${
sdkData.version
}`,
};

// edge sdks sets this to false because they use the clientSideID
// and they don't need the authorization header
if (includeAuthorizationHeader) {
headers.authorization = sdkKey;
}

if (sdkData.wrapperName) {
headers['x-launchdarkly-wrapper'] = sdkData.wrapperVersion
? `${sdkData.wrapperName}/${sdkData.wrapperVersion}`
Expand Down
30 changes: 18 additions & 12 deletions packages/shared/sdk-server/src/events/EventSender.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,21 +28,27 @@ export default class EventSender implements subsystem.LDEventSender {
private crypto: Crypto;

constructor(config: EventSenderOptions, clientContext: ClientContext) {
const {
basicConfiguration: {
sdkKey,
serviceEndpoints: {
events,
analyticsEventPath,
diagnosticEventPath,
includeAuthorizationHeader,
},
},
platform: { info, requests, crypto },
} = clientContext;

this.defaultHeaders = {
...defaultHeaders(
clientContext.basicConfiguration.sdkKey,
config,
clientContext.platform.info,
),
...defaultHeaders(sdkKey, config, info, includeAuthorizationHeader),
};

this.eventsUri = `${clientContext.basicConfiguration.serviceEndpoints.events}/bulk`;

this.diagnosticEventsUri = `${clientContext.basicConfiguration.serviceEndpoints.events}/diagnostic`;

this.requests = clientContext.platform.requests;

this.crypto = clientContext.platform.crypto;
this.eventsUri = `${events}${analyticsEventPath}`;
this.diagnosticEventsUri = `${events}${diagnosticEventPath}`;
this.requests = requests;
this.crypto = crypto;
}

private async tryPostingEvents(
Expand Down
5 changes: 4 additions & 1 deletion packages/shared/sdk-server/src/options/Configuration.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import {
ApplicationTags,
internal,
LDClientContext,
LDLogger,
NumberWithMinimum,
Expand Down Expand Up @@ -209,7 +210,7 @@ export default class Configuration {

public readonly bigSegments?: LDBigSegmentsOptions;

constructor(options: LDOptions = {}) {
constructor(options: LDOptions = {}, internalOptions: internal.LDInternalOptions = {}) {
// The default will handle undefined, but not null.
// Because we can be called from JS we need to be extra defensive.
// eslint-disable-next-line no-param-reassign
Expand All @@ -228,6 +229,8 @@ export default class Configuration {
validatedOptions.streamUri,
validatedOptions.baseUri,
validatedOptions.eventsUri,
internalOptions.analyticsEventPath,
internalOptions.diagnosticEventPath,
);
this.eventsCapacity = validatedOptions.capacity;
this.timeout = validatedOptions.timeout;
Expand Down
4 changes: 3 additions & 1 deletion packages/store/node-server-sdk-dynamodb/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,9 @@ const client = LaunchDarkly.init('YOUR SDK KEY', config);
By default, the DynamoDB client will try to get your AWS credentials and region name from environment variables and/or local configuration files, as described in the AWS SDK documentation. You can also specify any valid [DynamoDB client options](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB.html#constructor-property) like this:

```typescript
const dynamoDBOptions = { credentials: { accessKeyId: 'YOUR KEY', secretAccessKey: 'YOUR SECRET' }};
const dynamoDBOptions = {
credentials: { accessKeyId: 'YOUR KEY', secretAccessKey: 'YOUR SECRET' },
};
const store = DynamoDBFeatureStore('YOUR TABLE NAME', { clientOptions: dynamoDBOptions });
```

Expand Down
4 changes: 3 additions & 1 deletion scripts/build-package.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#!/bin/bash

# Run this script like:
# ./scripts/build-package.sh

Expand All @@ -23,7 +25,7 @@ ESM_PACKAGE_JSON=$( jq -n \
--arg type "module" \
'{ name: $name, version: $version, type: $type }' )

tsc --module commonjs --outDir dist/cjs/
tsc --module commonjs --outDir dist/cjs/
echo "$CJS_PACKAGE_JSON" > dist/cjs/package.json

tsc --module es2022 --outDir dist/esm/
Expand Down
2 changes: 2 additions & 0 deletions scripts/doc-name.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#!/bin/bash

# Given a path get the name for the documentation.
# ./scripts/doc-name.sh packages/sdk/server-node
# Produces something like:
Expand Down
2 changes: 2 additions & 0 deletions scripts/package-name.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#!/bin/bash

# Given a path get the name of the package.
# ./scripts/package-name.sh packages/sdk/server-node
# Produces something like:
Expand Down
12 changes: 7 additions & 5 deletions scripts/publish-doc.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#!/bin/bash

# Run this script like:
# ./scripts/publish-doc.sh packages/sdk/node

Expand Down Expand Up @@ -55,24 +57,24 @@ set +e

while true
do

git pull origin gh-pages --no-edit # should accept the default message
after_pull_sha=$(git rev-parse HEAD)

# The first time this runs the head_sha will be empty and they will not match.
# If the push fails, then we pull again, and if the SHA does not change, then
# the push will not succeed.
if [ "$head_sha" == "$after_pull_sha" ]; then
echo "Failed to get changes. Could not publish docs."
exit 1
fi

head_sha=$after_pull_sha

if git push; then
break
fi

echo "Push failed, trying again."
done

Expand Down
Loading