Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DTP-963] Add support for customer provided typings for LiveObjects #1922

Merged
merged 3 commits into from
Nov 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 54 additions & 3 deletions ably.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2039,25 +2039,76 @@
/**
* Retrieves the root {@link LiveMap} object for state on a channel.
*
* A type parameter can be provided to describe the structure of the LiveObjects state on the channel. By default, it uses types from the globally defined `LiveObjectsTypes` interface.
*
* You can specify custom types for LiveObjects by defining a global `LiveObjectsTypes` interface with a `root` property that conforms to {@link LiveMapType}.
*
* Example:
*
* ```typescript
* import { LiveCounter } from 'ably';
*
* type MyRoot = {
* myTypedKey: LiveCounter;
* };
*
* declare global {
* export interface LiveObjectsTypes {
* root: MyRoot;
* }
* }
* ```
*
* @returns A promise which, upon success, will be fulfilled with a {@link LiveMap} object. Upon failure, the promise will be rejected with an {@link ErrorInfo} object which explains the error.
*/
getRoot(): Promise<LiveMap>;
getRoot<T extends LiveMapType = DefaultRoot>(): Promise<LiveMap<T>>;
VeskeR marked this conversation as resolved.
Show resolved Hide resolved
}

declare global {
/**
* A globally defined interface that allows users to define custom types for LiveObjects.
*/
export interface LiveObjectsTypes {
[key: string]: unknown;
}
}

/**
* Represents the type of data stored in a {@link LiveMap}.
* It maps string keys to scalar values ({@link StateValue}), or other LiveObjects.
*/
export type LiveMapType = { [key: string]: StateValue | LiveMap<LiveMapType> | LiveCounter | undefined };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just noticed that StateValue includes both the Buffer and Uint8Array types. Presumably the correct type is platform-dependent. Is there a way we can select the correct type from some type that denotes the platform? (I can appreciate that might be difficult)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just used the same approach we already had in the public types for the CipherKey and CipherKeyParams

ably-js/ably.d.ts

Lines 2527 to 2531 in 18a2559

export type CipherKeyParam = ArrayBuffer | Uint8Array | string; // if string must be base64-encoded
/**
* The type of the key returned by {@link Crypto.generateRandomKey}. Typed differently depending on platform (`Buffer` in Node.js, `ArrayBuffer` elsewhere).
*/
export type CipherKey = ArrayBuffer | Buffer;
, which is to include both types for node.js and browsers (though an additional comment in the docstring should be added to specify which type you will receive on which platform).

I think to make it platform dependant we would need to have two separate type declaration files, and configure their exports in package.json so node.js and browser environments get the appropriate one.

It is way beyond the scope of this PR and LiveObjects work, and probably has too little of an impact relative to the amount of work required to think about it right now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


/**
* The default type for the `root` object in the LiveObjects, based on the globally defined {@link LiveObjectsTypes} interface.
*
* - If no custom types are provided in `LiveObjectsTypes`, defaults to an untyped root map representation using the {@link LiveMapType} interface.
* - If a `root` type exists in `LiveObjectsTypes` and conforms to the {@link LiveMapType} interface, it is used as the type for the `root` object.
* - If the provided `root` type does not match {@link LiveMapType}, a type error message is returned.
*/
export type DefaultRoot =
// we need a way to know when no types were provided by the user.
// we expect a "root" property to be set on LiveObjectsTypes interface, e.g. it won't be "unknown" anymore
unknown extends LiveObjectsTypes['root']
? LiveMapType // no custom types provided; use the default untyped map representation for the root
: LiveObjectsTypes['root'] extends LiveMapType
? LiveObjectsTypes['root'] // "root" property exists, and it is of an expected type, we can use this interface for the root object in LiveObjects.
: `Provided type definition for the "root" object in LiveObjectsTypes is not of an expected LiveMapType`;

/**
* The `LiveMap` class represents a key/value map data structure, similar to a JavaScript Map, where all changes are synchronized across clients in realtime.
* Conflict-free resolution for updates follows Last Write Wins (LWW) semantics, meaning that if two clients update the same key in the map, the update with the most recent timestamp wins.
*
* Keys must be strings. Values can be another Live Object, or a primitive type, such as a string, number, boolean, or binary data (see {@link StateValue}).
*/
export declare interface LiveMap extends LiveObject<LiveMapUpdate> {
export declare interface LiveMap<T extends LiveMapType> extends LiveObject<LiveMapUpdate> {
/**
* Returns the value associated with a given key. Returns `undefined` if the key doesn't exist in a map.
*
* @param key - The key to retrieve the value for.
* @returns A {@link LiveObject}, a primitive type (string, number, boolean, or binary data) or `undefined` if the key doesn't exist in a map.
*/
get(key: string): LiveObject | StateValue | undefined;
get<TKey extends keyof T & string>(key: TKey): T[TKey];

/**
* Returns the number of key/value pairs in the map.
Expand Down Expand Up @@ -2429,7 +2480,7 @@
* and {@link ChannelOptions}, or returns the existing channel object.
*
* @experimental This is a preview feature and may change in a future non-major release.
* This experimental method allows you to create custom realtime data feeds by selectively subscribing

Check warning on line 2483 in ably.d.ts

View workflow job for this annotation

GitHub Actions / lint

Expected no lines between tags
* to receive only part of the data from the channel.
* See the [announcement post](https://pages.ably.com/subscription-filters-preview) for more information.
*
Expand Down
22 changes: 12 additions & 10 deletions src/plugins/liveobjects/livemap.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import deepEqual from 'deep-equal';

import type * as API from '../../../ably';
import { LiveObject, LiveObjectData, LiveObjectUpdate, LiveObjectUpdateNoop } from './liveobject';
import { LiveObjects } from './liveobjects';
import {
Expand Down Expand Up @@ -45,7 +46,7 @@ export interface LiveMapUpdate extends LiveObjectUpdate {
update: { [keyName: string]: 'updated' | 'removed' };
}

export class LiveMap extends LiveObject<LiveMapData, LiveMapUpdate> {
export class LiveMap<T extends API.LiveMapType> extends LiveObject<LiveMapData, LiveMapUpdate> {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Breaking Change Impact Confirmed: Generic Type Parameter Addition

The verification reveals that this is indeed a breaking change with significant impact:

  • Direct instantiations of LiveMap will need type parameters
  • The change affects core functionality in liveobjects.ts where getRoot<T>() returns LiveMap<T>
  • Multiple internal implementations reference and extend LiveMap
  • Type definitions in ably.d.ts show extensive usage of LiveMap in the public API

Key affected areas:

  • src/plugins/liveobjects/liveobjects.ts: Core implementation using LiveMap<T>
  • src/plugins/liveobjects/syncliveobjectsdatapool.ts: Internal data management
  • Type definitions affecting public API contracts
  • Test templates demonstrating expected usage patterns
🔗 Analysis chain

Breaking Change: Generic Type Parameter Addition

The addition of the generic type parameter T extends API.LiveMapType improves type safety but introduces a breaking change. Existing code that instantiates LiveMap directly will need to be updated to provide a type parameter.

Let's verify the impact:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for direct LiveMap instantiations that need updates
echo "=== Direct LiveMap instantiations ==="
rg "new LiveMap\(" --type ts

# Search for LiveMap type references that need updates
echo "=== LiveMap type references ==="
rg "LiveMap[^<]" --type ts

Length of output: 8684

constructor(
liveObjects: LiveObjects,
private _semantics: MapSemantics,
Expand All @@ -59,8 +60,8 @@ export class LiveMap extends LiveObject<LiveMapData, LiveMapUpdate> {
*
* @internal
*/
static zeroValue(liveobjects: LiveObjects, objectId: string): LiveMap {
return new LiveMap(liveobjects, MapSemantics.LWW, objectId);
static zeroValue<T extends API.LiveMapType>(liveobjects: LiveObjects, objectId: string): LiveMap<T> {
return new LiveMap<T>(liveobjects, MapSemantics.LWW, objectId);
}

/**
Expand All @@ -69,8 +70,8 @@ export class LiveMap extends LiveObject<LiveMapData, LiveMapUpdate> {
*
* @internal
*/
static fromStateObject(liveobjects: LiveObjects, stateObject: StateObject): LiveMap {
const obj = new LiveMap(liveobjects, stateObject.map?.semantics!, stateObject.objectId);
static fromStateObject<T extends API.LiveMapType>(liveobjects: LiveObjects, stateObject: StateObject): LiveMap<T> {
const obj = new LiveMap<T>(liveobjects, stateObject.map?.semantics!, stateObject.objectId);
obj.overrideWithStateObject(stateObject);
return obj;
}
Expand All @@ -82,24 +83,25 @@ export class LiveMap extends LiveObject<LiveMapData, LiveMapUpdate> {
* then you will get a reference to that Live Object if it exists in the local pool, or undefined otherwise.
* If the value is not an objectId, then you will get that value.
*/
get(key: string): LiveObject | StateValue | undefined {
// force the key to be of type string as we only allow strings as key in a map
get<TKey extends keyof T & string>(key: TKey): T[TKey] {
VeskeR marked this conversation as resolved.
Show resolved Hide resolved
const element = this._dataRef.data.get(key);

if (element === undefined) {
return undefined;
return undefined as T[TKey];
VeskeR marked this conversation as resolved.
Show resolved Hide resolved
}

if (element.tombstone === true) {
return undefined;
return undefined as T[TKey];
}

// data exists for non-tombstoned elements
const data = element.data!;

if ('value' in data) {
return data.value;
return data.value as T[TKey];
} else {
return this._liveObjects.getPool().get(data.objectId);
return this._liveObjects.getPool().get(data.objectId) as T[TKey];
}
VeskeR marked this conversation as resolved.
Show resolved Hide resolved
}

Expand Down
9 changes: 7 additions & 2 deletions src/plugins/liveobjects/liveobjects.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,18 @@ export class LiveObjects {
this._bufferedStateOperations = [];
}

async getRoot(): Promise<LiveMap> {
/**
* When called without a type variable, we return a default root type which is based on globally defined LiveObjects interface.
* A user can provide an explicit type for the getRoot method to explicitly set the LiveObjects type structure on this particular channel.
* This is useful when working with LiveObjects on multiple channels with different underlying data.
*/
async getRoot<T extends API.LiveMapType = API.DefaultRoot>(): Promise<LiveMap<T>> {
// SYNC is currently in progress, wait for SYNC sequence to finish
if (this._syncInProgress) {
await this._eventEmitter.once(LiveObjectsEvents.SyncCompleted);
}

return this._liveObjectsPool.get(ROOT_OBJECT_ID) as LiveMap;
return this._liveObjectsPool.get(ROOT_OBJECT_ID) as LiveMap<T>;
}

/**
Expand Down
21 changes: 21 additions & 0 deletions test/package/browser/template/src/ably.config.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import { LiveCounter, LiveMap } from 'ably';

type CustomRoot = {
numberKey: number;
stringKey: string;
booleanKey: boolean;
couldBeUndefined?: string;
mapKey?: LiveMap<{
foo: 'bar';
nestedMap?: LiveMap<{
baz: 'qux';
}>;
}>;
counterKey?: LiveCounter;
};

declare global {
export interface LiveObjectsTypes {
root: CustomRoot;
}
}
42 changes: 32 additions & 10 deletions test/package/browser/template/src/index-liveobjects.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
import * as Ably from 'ably';
import LiveObjects from 'ably/liveobjects';
import { CustomRoot } from './ably.config';
import { createSandboxAblyAPIKey } from './sandbox';

type ExplicitRootType = {
someOtherKey: string;
};

globalThis.testAblyPackage = async function () {
const key = await createSandboxAblyAPIKey({ featureFlags: ['enableChannelState'] });

Expand All @@ -11,12 +16,27 @@ globalThis.testAblyPackage = async function () {
// check liveObjects can be accessed
const liveObjects = channel.liveObjects;
await channel.attach();
// root should be a LiveMap object
const root: Ably.LiveMap = await liveObjects.getRoot();
// expect root to be a LiveMap instance with LiveObjects types defined via the global LiveObjectsTypes interface
// also checks that we can refer to the LiveObjects types exported from 'ably' by referencing a LiveMap interface
const root: Ably.LiveMap<CustomRoot> = await liveObjects.getRoot();

// check root has expected LiveMap TypeScript type methods
const size: number = root.size();

// check root is recognized as LiveMap TypeScript type
root.get('someKey');
root.size();
// check custom user provided typings via LiveObjectsTypes are working:
// keys on a root:
const aNumber: number = root.get('numberKey');
const aString: string = root.get('stringKey');
const aBoolean: boolean = root.get('booleanKey');
const couldBeUndefined: string | undefined = root.get('couldBeUndefined');
// live objects on a root:
const counter: Ably.LiveCounter | undefined = root.get('counterKey');
const map: LiveObjectsTypes['root']['mapKey'] = root.get('mapKey');
// check string literal types works
// need to use nullish coalescing as we didn't actually create any data on the root,
// so the next calls would fail. we only need to check that TypeScript types work
const foo: 'bar' = map?.get('foo')!;
const baz: 'qux' = map?.get('nestedMap')?.get('baz')!;

// check LiveMap subscription callback has correct TypeScript types
const { unsubscribe } = root.subscribe(({ update }) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: could we add tighter types for the allowed key strings in the update object? (Okay to create a separate ticket for this)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expand All @@ -31,13 +51,15 @@ globalThis.testAblyPackage = async function () {
});
unsubscribe();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we add a check for unsubscribeAll?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same on the counter

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it wasn't my intention to check every existing public method, as it can get quite bloaty, but instead to check some of the most interesting typing parts, e.g. customer provided typings and some tricky ones like subscription update objects.
the presence and functionality of the public methods is tested by regular unit/integration tests, not by this package test.


// check LiveCounter types also behave as expected
const counter = root.get('randomKey') as Ably.LiveCounter | undefined;
// use nullish coalescing as we didn't actually create a counter object on the root,
// so the next calls would fail. we only need to check that TypeScript types work
const value: number = counter?.value();
// check LiveCounter type also behaves as expected
// same deal with nullish coalescing
const value: number = counter?.value()!;
const counterSubscribeResponse = counter?.subscribe(({ update }) => {
const shouldBeANumber: number = update.inc;
});
counterSubscribeResponse?.unsubscribe();

// check can provide custom types for the getRoot method, ignoring global LiveObjectsTypes interface
const explicitRoot: Ably.LiveMap<ExplicitRootType> = await liveObjects.getRoot<ExplicitRootType>();
const someOtherKey: string = explicitRoot.get('someOtherKey');
};
1 change: 1 addition & 0 deletions test/package/browser/template/src/tsconfig.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
{
"include": ["**/*.ts", "**/*.tsx"],
"compilerOptions": {
"strictNullChecks": true,
"resolveJsonModule": true,
"esModuleInterop": true,
"module": "esnext",
Expand Down
3 changes: 2 additions & 1 deletion typedoc.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,6 @@
"TypeAlias",
"Variable",
"Namespace"
]
],
"intentionallyNotExported": ["__global.LiveObjectsTypes"]
}
Loading