Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support RPCv2 CBOR wire protocol #887

Open
wants to merge 31 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
b236c1a
add CBOR support to ServiceUtils (WireProtocol, AWSProtocol)
dayaffe Nov 8, 2024
c6a872c
add configurable protocol resolution with a default priority list
dayaffe Nov 8, 2024
37fdc0e
make function makeQueryCompatibleError generic to work across multipl…
dayaffe Nov 13, 2024
cdbaadb
Merge branch 'main' into day/rpcv2-cbor
dayaffe Dec 2, 2024
10dccdd
current version only has 3/69 failing protocol tests
dayaffe Dec 19, 2024
50ceb88
ALL TESTS PASSING WOOOO
dayaffe Dec 20, 2024
4eec99b
Merge branch 'main' into day/rpcv2-cbor
dayaffe Dec 24, 2024
ebde137
all tests passing now
dayaffe Dec 24, 2024
6b1e91c
add getter for private context
dayaffe Dec 24, 2024
e587f61
Merge branch 'main' into day/rpcv2-cbor
dayaffe Dec 24, 2024
7a321a0
add addUserAgentMiddleware fun to mocks
dayaffe Dec 24, 2024
3e1f884
expose only service name instead of whole context
dayaffe Dec 24, 2024
079a6c3
implement addUserAgentMiddleware in CBOR mock
dayaffe Dec 24, 2024
b8475ce
fix some lint issues
dayaffe Dec 26, 2024
36628b4
fix some lint issues
dayaffe Dec 26, 2024
9b2880b
fix imports
dayaffe Dec 26, 2024
f14c31b
more import fixing
dayaffe Dec 26, 2024
2601702
more lint fixes
dayaffe Dec 26, 2024
2713fbc
more lint
dayaffe Dec 26, 2024
2de2171
try reorder
dayaffe Dec 26, 2024
2b3120e
ran ktlintFormat
dayaffe Dec 26, 2024
5bc3d23
Merge branch 'main' into day/rpcv2-cbor
dayaffe Dec 26, 2024
d7da330
address PR comments
dayaffe Dec 31, 2024
6a607ae
ignore timestampFormat on member for generating reading/writing closures
dayaffe Dec 31, 2024
dde4eba
change format type
dayaffe Dec 31, 2024
40f271f
change format type
dayaffe Dec 31, 2024
3accc45
short circuit timestamp format resolution for cbor
dayaffe Dec 31, 2024
537b93c
address PR comments
dayaffe Jan 8, 2025
9b756c9
Merge branch 'main' into day/rpcv2-cbor
dayaffe Jan 8, 2025
7181815
add comparator test
dayaffe Jan 9, 2025
e5d1a56
Merge branch 'main' into day/rpcv2-cbor
dayaffe Jan 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ let package = Package(
.library(name: "SmithyStreams", targets: ["SmithyStreams"]),
.library(name: "SmithyChecksumsAPI", targets: ["SmithyChecksumsAPI"]),
.library(name: "SmithyChecksums", targets: ["SmithyChecksums"]),
.library(name: "SmithyCBOR", targets: ["SmithyCBOR"]),
.library(name: "SmithyWaitersAPI", targets: ["SmithyWaitersAPI"]),
.library(name: "SmithyTestUtil", targets: ["SmithyTestUtil"]),
],
Expand Down Expand Up @@ -92,6 +93,7 @@ let package = Package(
"SmithyStreams",
"SmithyChecksumsAPI",
"SmithyChecksums",
"SmithyCBOR",
.product(name: "AwsCommonRuntimeKit", package: "aws-crt-swift"),
],
resources: [
Expand Down Expand Up @@ -140,7 +142,7 @@ let package = Package(
),
.target(
name: "SmithyTestUtil",
dependencies: ["ClientRuntime", "SmithyHTTPAPI", "SmithyIdentity"]
dependencies: ["ClientRuntime", "SmithyHTTPAPI", "SmithyIdentity", "SmithyCBOR"]
),
.target(
name: "SmithyIdentity",
Expand Down Expand Up @@ -222,6 +224,14 @@ let package = Package(
.product(name: "AwsCommonRuntimeKit", package: "aws-crt-swift")
]
),
.target(
name: "SmithyCBOR",
dependencies: [
"SmithyReadWrite",
"SmithyTimestamps",
.product(name: "AwsCommonRuntimeKit", package: "aws-crt-swift")
]
),
.target(
name: "SmithyWaitersAPI"
),
Expand All @@ -235,6 +245,10 @@ let package = Package(
],
resources: [ .process("Resources") ]
),
.testTarget(
name: "SmithyCBORTests",
dependencies: ["SmithyCBOR", "ClientRuntime", "SmithyTestUtil"]
),
.testTarget(
name: "SmithyHTTPClientTests",
dependencies: [
Expand Down
222 changes: 222 additions & 0 deletions Sources/SmithyCBOR/Reader/Reader.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,222 @@
//
// Copyright Amazon.com Inc. or its affiliates.
// All Rights Reserved.
//
// SPDX-License-Identifier: Apache-2.0
//

import AwsCommonRuntimeKit
import Foundation

@_spi(SmithyReadWrite) import protocol SmithyReadWrite.SmithyReader
@_spi(SmithyReadWrite) import enum SmithyReadWrite.ReaderError
@_spi(SmithyReadWrite) import protocol SmithyReadWrite.SmithyWriter
@_spi(Smithy) import struct Smithy.Document
@_spi(Smithy) import protocol Smithy.SmithyDocument
@_spi(SmithyTimestamps) import enum SmithyTimestamps.TimestampFormat
@_spi(SmithyTimestamps) import struct SmithyTimestamps.TimestampFormatter

@_spi(SmithyReadWrite)
public final class Reader: SmithyReader {
public typealias NodeInfo = String

public let cborValue: CBORType?
public let nodeInfo: NodeInfo
public internal(set) var children: [Reader] = []
public internal(set) weak var parent: Reader?
public var hasContent: Bool { cborValue != nil && cborValue != .null }

public init(nodeInfo: NodeInfo, cborValue: CBORType?, parent: Reader? = nil) {
self.nodeInfo = nodeInfo
self.cborValue = cborValue
self.parent = parent
self.children = Self.children(from: cborValue, parent: self)
}

public static func from(data: Data) throws -> Reader {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this function calls CBORDecoder which is defined in CRT for actually decoding individual types

let decoder = try CBORDecoder(data: [UInt8](data))
let rootValue: CBORType
if decoder.hasNext() {
rootValue = try decoder.popNext()
} else {
rootValue = .null
}
return Reader(nodeInfo: "", cborValue: rootValue, parent: nil)
}

private static func children(from cborValue: CBORType?, parent: Reader) -> [Reader] {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function helps form a graph for nested and complex types

var children = [Reader]()
switch cborValue {
case .map(let map):
for (key, value) in map {
let child = Reader(nodeInfo: key, cborValue: value, parent: parent)
children.append(child)
}
case .array(let array):
for (index, value) in array.enumerated() {
let child = Reader(nodeInfo: "\(index)", cborValue: value, parent: parent)
children.append(child)
}
sichanyoo marked this conversation as resolved.
Show resolved Hide resolved
default:
break
}
return children
}

public subscript(nodeInfo: NodeInfo) -> Reader {
if let match = children.first(where: { $0.nodeInfo == nodeInfo }) {
return match
} else {
return Reader(nodeInfo: nodeInfo, cborValue: nil, parent: self)
}
}

public func readIfPresent() throws -> String? {
switch cborValue {
case .text(let string):
return string
case .indef_text_start:
// Handle concatenation of indefinite-length text
var combinedText = ""
for child in children {
if let chunk = try child.readIfPresent() as String? {
combinedText += chunk
}
}
return combinedText
case .bytes(let data):
return String(data: data, encoding: .utf8)
default:
return nil
}
}

public func readIfPresent() throws -> Int8? {
switch cborValue {
case .int(let intValue): return Int8(intValue)
case .uint(let uintValue): return Int8(uintValue)
default: return nil
}
}

public func readIfPresent() throws -> Int16? {
switch cborValue {
case .int(let intValue): return Int16(intValue)
case .uint(let uintValue): return Int16(uintValue)
default: return nil
}
}

public func readIfPresent() throws -> Int? {
switch cborValue {
case .int(let intValue): return Int(intValue)
case .uint(let uintValue): return Int(uintValue)
default: return nil
}
}

public func readIfPresent() throws -> Float? {
switch cborValue {
case .double(let doubleValue): return Float(doubleValue)
case .int(let intValue): return Float(intValue)
case .uint(let uintValue): return Float(uintValue)
default: return nil
}
}

public func readIfPresent() throws -> Double? {
switch cborValue {
case .double(let doubleValue): return doubleValue
case .int(let intValue): return Double(intValue)
case .uint(let uintValue): return Double(uintValue)
default: return nil
}
}

public func readIfPresent() throws -> Bool? {
switch cborValue {
case .bool(let boolValue): return boolValue
default: return nil
}
}

public func readIfPresent() throws -> Data? {
switch cborValue {
case .bytes(let data): return data
case .text(let string): return Data(base64Encoded: string)
default: return nil
}
}

public func readIfPresent() throws -> Document? {
// No operation. Smithy document not supported in CBOR
return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there future plans to support Document? Document type is super-important in schema-based serde.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe in RPCv3 so not anytime in the near future. Not supporting Document type is part of the SEP https://code.amazon.com/packages/AwsDrSeps/blobs/main/--/seps/accepted/shared/rpcv2-cbor/protocol-selection_smithy-rpc-v2.md

}

public func readIfPresent<T>() throws -> T? where T: RawRepresentable, T.RawValue == Int {
guard let rawValue: Int = try readIfPresent() else { return nil }
return T(rawValue: rawValue)
}

public func readIfPresent<T>() throws -> T? where T: RawRepresentable, T.RawValue == String {
guard let rawValue: String = try readIfPresent() else { return nil }
return T(rawValue: rawValue)
}

public func readTimestampIfPresent(format: TimestampFormat) throws -> Date? {
switch cborValue {
case .double(let doubleValue):
return Date(timeIntervalSince1970: doubleValue)
case .int(let intValue):
return Date(timeIntervalSince1970: Double(intValue))
case .uint(let uintValue):
return Date(timeIntervalSince1970: Double(uintValue))
case .text(let string):
return TimestampFormatter(format: format).date(from: string)
case .date(let dateValue):
return dateValue // Directly return the date value
default:
return nil
}
}

public func readMapIfPresent<Value>(
valueReadingClosure: (Reader) throws -> Value,
keyNodeInfo: NodeInfo,
valueNodeInfo: NodeInfo,
isFlattened: Bool
) throws -> [String: Value]? {
guard let cborValue else { return nil }
guard case .map(let map) = cborValue else { return nil }
var dict = [String: Value]()
for (key, _) in map {
let reader = self[key]
do {
let value = try valueReadingClosure(reader)
dict[key] = value
} catch ReaderError.requiredValueNotPresent {
if !(try reader.readNullIfPresent() ?? false) { throw ReaderError.requiredValueNotPresent }
}
}
return dict
}

public func readListIfPresent<Member>(
memberReadingClosure: (Reader) throws -> Member,
memberNodeInfo: NodeInfo,
isFlattened: Bool
) throws -> [Member]? {
guard let cborValue else { return nil }
guard case .array = cborValue else { return nil }
return try children.map { child in
return try memberReadingClosure(child)
}
}

public func readNullIfPresent() throws -> Bool? {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the doc comment on readNullIfPresent():

    /// Attempts to read a `null` value from the source document.
    /// - Returns: `true` if the value read is null, `false` if a value is present but it is not null, `nil` if no value is present.

You should first unwrap cborValue & return nil if it's not there, then return T/F based on whether the present value is a CBOR null.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in latest revision

guard let value = cborValue else {
return nil
}
return value == .null
}
}
Loading
Loading