Skip to content

Commit

Permalink
Add support for partially decoding arrays (#107)
Browse files Browse the repository at this point in the history
* save point - start adding support for lossy array decoding

* clean up some documentation, add sendable conformance to retry config

* Change concurrency checking to complete

* dont force unwrap final resource when auto decoding lossy array

* remove convenience extension that wasn't working properly

* Move FailableDecodable into LossyArray, rework LossyArray a ltitle for easier extension

* switch array decoding to be request-level

this is kind of a save point - doesnt work with smart unwrap yet

* add lossy decoding for smart unwraps

* update changelog - bump version to 2.1.0
  • Loading branch information
brendanlensink authored Dec 14, 2022
1 parent 5ce356e commit c6059d0
Show file tree
Hide file tree
Showing 12 changed files with 206 additions and 26 deletions.
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,16 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).

## [2.1.0] 14-12-22
- [104] Add support for partially decoding arrays through new `arrayDecodingStrategy` parameter on `Request`.
- [106] Fix `RetryConfiguration` not being marked as `Sendable`.


## [2.0.0] 18-11-22
- [77] Rework networking to use async/await by default.
- [96] Add support + documentation for basic GraphQL requests.


## [1.0.0] 12-9-21
### Added
- [60] Add better handling for printing request bodies to the console, including some default redaction for sensitive parameters.
Expand All @@ -18,19 +24,23 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- [76] Add an option to set json en/decoding strategies in the Netable config constructor in addition to per-request.
- [79] Add `requestFailedDelegate` and `requestFailedPublisher` to users to handle errors globally in addition to in `request` completion callbacks. Bumps minimum iOS version to 13.0.


## [0.10.3] - 12-01-21
### Changed
- Fixed an issue with logging successful requests that was preventing finalized data from being printed properly.
- Fixed a couple small tpyos.


## [0.10.2] - 10-08-20
### Added
- Added support for `DELETE` requests.


## [0.10.1] - 16-07-20
### Changed
- Fixed some properties in the new logging not being marked as "public".


## [0.10.0] - 16-07-20
### Added
- Requests are now automatically retried for (some) failures. The new RetryConfiguration struct controls the exact mechanisms for retrying.
Expand Down
14 changes: 13 additions & 1 deletion Netable/Netable.xcodeproj/project.pbxproj
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
C639674D28E4F4CD00ADAE3E /* Netable+Equatable.swift in Sources */ = {isa = PBXBuildFile; fileRef = C639674C28E4F4CD00ADAE3E /* Netable+Equatable.swift */; };
C639674F28E4F58100ADAE3E /* String+FullyQualifiedURL.swift in Sources */ = {isa = PBXBuildFile; fileRef = C639674E28E4F58100ADAE3E /* String+FullyQualifiedURL.swift */; };
C639675128E5F89D00ADAE3E /* Netable+Error.swift in Sources */ = {isa = PBXBuildFile; fileRef = C639675028E5F89D00ADAE3E /* Netable+Error.swift */; };
C64ADA47293F9ED900695444 /* ArrayDecodeStrategy.swift in Sources */ = {isa = PBXBuildFile; fileRef = C64ADA46293F9ED900695444 /* ArrayDecodeStrategy.swift */; };
C64EAB6926F2828E0093850A /* Post.swift in Sources */ = {isa = PBXBuildFile; fileRef = C64EAB6826F2828E0093850A /* Post.swift */; };
C64EAB6B26F283440093850A /* GetPostsRequest.swift in Sources */ = {isa = PBXBuildFile; fileRef = C64EAB6A26F283440093850A /* GetPostsRequest.swift */; };
C64EAB6D26F29AFD0093850A /* UnauthorizedRequest.swift in Sources */ = {isa = PBXBuildFile; fileRef = C64EAB6C26F29AFD0093850A /* UnauthorizedRequest.swift */; };
Expand Down Expand Up @@ -67,6 +68,7 @@
C692789D26F147FC00917E65 /* GetUserDetailsRequest.swift in Sources */ = {isa = PBXBuildFile; fileRef = C692789C26F147FC00917E65 /* GetUserDetailsRequest.swift */; };
C6953F42241A95830044D278 /* LogDestination.swift in Sources */ = {isa = PBXBuildFile; fileRef = C6953F41241A95830044D278 /* LogDestination.swift */; };
C6A455EE2912D77600C1C20E /* ErrorService.swift in Sources */ = {isa = PBXBuildFile; fileRef = C6A455ED2912D77600C1C20E /* ErrorService.swift */; };
C6DA3354293822230076F693 /* LossyArray.swift in Sources */ = {isa = PBXBuildFile; fileRef = C6DA3353293822230076F693 /* LossyArray.swift */; };
C6F4CFB026D582E8004E6BB8 /* RequestFailedDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = C6F4CFAF26D582E8004E6BB8 /* RequestFailedDelegate.swift */; };
C6F4CFB626D598B3004E6BB8 /* README.md in Resources */ = {isa = PBXBuildFile; fileRef = C6F4CFB526D598B3004E6BB8 /* README.md */; };
/* End PBXBuildFile section */
Expand Down Expand Up @@ -136,6 +138,7 @@
C639674C28E4F4CD00ADAE3E /* Netable+Equatable.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = "Netable+Equatable.swift"; sourceTree = "<group>"; };
C639674E28E4F58100ADAE3E /* String+FullyQualifiedURL.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = "String+FullyQualifiedURL.swift"; sourceTree = "<group>"; };
C639675028E5F89D00ADAE3E /* Netable+Error.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = "Netable+Error.swift"; sourceTree = "<group>"; };
C64ADA46293F9ED900695444 /* ArrayDecodeStrategy.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ArrayDecodeStrategy.swift; sourceTree = "<group>"; };
C64EAB6826F2828E0093850A /* Post.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Post.swift; sourceTree = "<group>"; };
C64EAB6A26F283440093850A /* GetPostsRequest.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = GetPostsRequest.swift; sourceTree = "<group>"; };
C64EAB6C26F29AFD0093850A /* UnauthorizedRequest.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = UnauthorizedRequest.swift; sourceTree = "<group>"; };
Expand Down Expand Up @@ -166,6 +169,7 @@
C692789C26F147FC00917E65 /* GetUserDetailsRequest.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = GetUserDetailsRequest.swift; sourceTree = "<group>"; };
C6953F41241A95830044D278 /* LogDestination.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LogDestination.swift; sourceTree = "<group>"; };
C6A455ED2912D77600C1C20E /* ErrorService.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ErrorService.swift; sourceTree = "<group>"; };
C6DA3353293822230076F693 /* LossyArray.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LossyArray.swift; sourceTree = "<group>"; };
C6F4CFAF26D582E8004E6BB8 /* RequestFailedDelegate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = RequestFailedDelegate.swift; sourceTree = "<group>"; };
C6F4CFB526D598B3004E6BB8 /* README.md */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = net.daringfireball.markdown; name = README.md; path = ../../README.md; sourceTree = "<group>"; };
/* End PBXFileReference section */
Expand Down Expand Up @@ -245,6 +249,7 @@
B8C928A223E9FBEC00DB2B37 /* HTTPMethod.swift */,
B8C9288423E9F68000DB2B37 /* Info.plist */,
C6953F41241A95830044D278 /* LogDestination.swift */,
C6DA3353293822230076F693 /* LossyArray.swift */,
B8C9288323E9F68000DB2B37 /* Netable.h */,
B8C928A823E9FDCC00DB2B37 /* Netable.swift */,
C6F4CFB526D598B3004E6BB8 /* README.md */,
Expand Down Expand Up @@ -282,9 +287,10 @@
C639674B28E4F4BF00ADAE3E /* Helper */ = {
isa = PBXGroup;
children = (
C64ADA46293F9ED900695444 /* ArrayDecodeStrategy.swift */,
C639674C28E4F4CD00ADAE3E /* Netable+Equatable.swift */,
C639674E28E4F58100ADAE3E /* String+FullyQualifiedURL.swift */,
C639675028E5F89D00ADAE3E /* Netable+Error.swift */,
C639674E28E4F58100ADAE3E /* String+FullyQualifiedURL.swift */,
);
path = Helper;
sourceTree = "<group>";
Expand Down Expand Up @@ -592,9 +598,11 @@
B8C9289F23E9FB1500DB2B37 /* URLRequest+EncodeParameters.swift in Sources */,
C61DC6FC28CFDF3F0089E912 /* GraphQLRequest.swift in Sources */,
B8C928A123E9FBA100DB2B37 /* Request.swift in Sources */,
C6DA3354293822230076F693 /* LossyArray.swift in Sources */,
C65289F526D01829009D486B /* Config.swift in Sources */,
C639674D28E4F4CD00ADAE3E /* Netable+Equatable.swift in Sources */,
C6953F42241A95830044D278 /* LogDestination.swift in Sources */,
C64ADA47293F9ED900695444 /* ArrayDecodeStrategy.swift in Sources */,
B8C9289D23E9FA0E00DB2B37 /* Error.swift in Sources */,
C639674F28E4F58100ADAE3E /* String+FullyQualifiedURL.swift in Sources */,
B8C928A323E9FBEC00DB2B37 /* HTTPMethod.swift in Sources */,
Expand Down Expand Up @@ -663,6 +671,7 @@
PRODUCT_BUNDLE_IDENTIFIER = com.steamclock.NetableExample;
PRODUCT_NAME = NetableExample;
SUPPORTS_MACCATALYST = YES;
SWIFT_STRICT_CONCURRENCY = complete;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
};
Expand All @@ -685,6 +694,7 @@
PRODUCT_BUNDLE_IDENTIFIER = com.steamclock.NetableExample;
PRODUCT_NAME = NetableExample;
SUPPORTS_MACCATALYST = YES;
SWIFT_STRICT_CONCURRENCY = complete;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
};
Expand Down Expand Up @@ -837,6 +847,7 @@
SKIP_INSTALL = YES;
SUPPORTS_MACCATALYST = YES;
SWIFT_OPTIMIZATION_LEVEL = "-Onone";
SWIFT_STRICT_CONCURRENCY = minimal;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
};
Expand Down Expand Up @@ -866,6 +877,7 @@
PRODUCT_NAME = "$(TARGET_NAME:c99extidentifier)";
SKIP_INSTALL = YES;
SUPPORTS_MACCATALYST = YES;
SWIFT_STRICT_CONCURRENCY = minimal;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
};
Expand Down
20 changes: 20 additions & 0 deletions Netable/Netable/Helper/ArrayDecodeStrategy.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
//
// ArrayDecodeStrategy.swift
// Netable
//
// Created by Brendan on 2022-12-06.
// Copyright © 2022 Steamclock Software. All rights reserved.
//

import Foundation

/// Strategy to use when decoding top-level arrays of values
/// By default, if you try to decode an array objects and one of the objects fails to decode, the entire array fails to decode.
/// Instead, this allows you to partially decode arrays and only return the well-formed elements.
/// For lossy decoding nested arrays, we recommend checking out [Better Codable](https://github.com/marksands/BetterCodable).
public enum ArrayDecodeStrategy {
/// If any element of the array fails to decode, the whole array fails.
case standard
/// Decode the array, omitting any elements that fail to decode.
case lossy
}
46 changes: 46 additions & 0 deletions Netable/Netable/LossyArray.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
//
// LossyArray.swift
// Netable
//
// Created by Brendan on 2022-11-30.
// Copyright © 2022 Steamclock Software. All rights reserved.
//

import Foundation

/// Adapted from https://stackoverflow.com/a/46369152

/// Array container that allows for partial decoding of elements.
/// If an element of the array fails to decode, it will be omitted rather than the rest of the array failing to decode.
public struct LossyArray<Element> {
/// All elements of the array that decoded successfully.
public var elements: [Element]

public init(elements: [Element]) {
self.elements = elements
}
}

extension LossyArray: Decodable where Element: Decodable {
/// Decode non-optional item into an optional element.
public struct FailableDecodable<Element: Decodable>: Decodable {
public var element: Element?

/// Decode an element and set it to `nil` if decoding fails.
public init(from decoder: Decoder) throws {
let container = try decoder.singleValueContainer()
element = try? container.decode(Element.self)
}
}

/// Attempt to decode the contents of an array, omitting any results that fail to decode.
public init(from decoder: Decoder) throws {
var elements = [Element?]()
var container = try decoder.unkeyedContainer()
while !container.isAtEnd {
let item = try container.decode(FailableDecodable<Element>.self).element
elements.append(item)
}
self.elements = elements.compactMap { $0 }
}
}
77 changes: 71 additions & 6 deletions Netable/Netable/Request.swift
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ public protocol Request: Sendable {
/// See `FallbackDecoderViewController` for an example.
associatedtype FallbackResource: Sendable = AnyObject

/// Allows for top-level arrays to be partially decoded if some elements fail to decode.
var arrayDecodeStrategy: ArrayDecodeStrategy { get }

/// Any headers that should be included with the request.
/// Note that these will be set _after_ any global headers,
/// and will thus take precedence if there's a duplicated key
Expand Down Expand Up @@ -71,6 +74,10 @@ public extension Request {
return Set<String>()
}

var arrayDecodeStrategy: ArrayDecodeStrategy {
.standard
}

func unredactedParameters(defaultEncodingStrategy: JSONEncoder.KeyEncodingStrategy) -> [String: String] {
var output = [String: String]()

Expand Down Expand Up @@ -114,6 +121,32 @@ public extension Request where FinalResource == RawResource {
}
}

public extension Request where
RawResource: Sequence,
RawResource: Decodable,
RawResource.Element: Decodable
{
func decode(_ data: Data?, defaultDecodingStrategy: JSONDecoder.KeyDecodingStrategy) async throws -> RawResource {
let decoder = JSONDecoder()
decoder.dateDecodingStrategy = .iso8601
decoder.keyDecodingStrategy = jsonKeyDecodingStrategy ?? defaultDecodingStrategy

guard let data = data else { throw NetableError.noData }

do {
guard arrayDecodeStrategy == .lossy else {
return try decoder.decode(RawResource.self, from: data)
}

let decoded = try decoder.decode(LossyArray<RawResource.Element>.self, from: data)

return decoded.elements as! Self.RawResource
} catch {
throw NetableError.decodingError(error, data)
}
}
}

public extension Request where RawResource: Decodable {
func decode(_ data: Data?, defaultDecodingStrategy: JSONDecoder.KeyDecodingStrategy) async throws -> RawResource {
let decoder = JSONDecoder()
Expand All @@ -122,11 +155,9 @@ public extension Request where RawResource: Decodable {

do {
if RawResource.self == Empty.self {
let raw = try decoder.decode(RawResource.self, from: Empty.data)
return raw
return try decoder.decode(RawResource.self, from: Empty.data)
} else if let data = data {
let raw = try decoder.decode(RawResource.self, from: data)
return raw
return try decoder.decode(RawResource.self, from: data)
}
} catch {
throw NetableError.decodingError(error, data)
Expand All @@ -150,6 +181,42 @@ extension CodingUserInfoKey {
static let smartUnwrapKey = CodingUserInfoKey(rawValue: "smartUnwrapKey")!
}

public extension Request where
RawResource == SmartUnwrap<FinalResource>,
FinalResource: Sequence,
FinalResource: Decodable,
FinalResource.Element: Decodable
{
func decode(_ data: Data?, defaultDecodingStrategy: JSONDecoder.KeyDecodingStrategy) async throws -> RawResource {
guard let data = data else {
throw NetableError.noData
}

do {
let decoder = JSONDecoder()
decoder.userInfo = [
.smartUnwrapKey: smartUnwrapKey
]

guard arrayDecodeStrategy == .lossy else {
return try decoder.decode(SmartUnwrap<FinalResource>.self, from: data)
}

let decodedType = try decoder.decode(SmartUnwrap<LossyArray<FinalResource.Element>>.self, from: data).decodedType
guard let finalResource = decodedType.elements as? FinalResource,
let rawResource = SmartUnwrap(decodedType: finalResource) as? SmartUnwrap<FinalResource> else {
throw NetableError.resourceExtractionError("Failed to smart unwrap lossy decodable type. This is an internal error.")
}

return rawResource
} catch {
throw NetableError.decodingError(error, data)
}
}
}



public extension Request where RawResource == SmartUnwrap<FinalResource> {
func decode(_ data: Data?, defaultDecodingStrategy: JSONDecoder.KeyDecodingStrategy) async throws -> RawResource {
guard let data = data else {
Expand Down Expand Up @@ -204,5 +271,3 @@ public extension Request where RawResource: Decodable, FallbackResource: Decodab
public struct Empty: Codable {
public static let data = "{}".data(using: .utf8)!
}


6 changes: 3 additions & 3 deletions Netable/Netable/RetryConfiguration.swift
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@

import Foundation

public struct RetryConfiguration {
public struct RetryConfiguration: Sendable {
/// Specify types of networking errors to retry
public enum Errors {
public enum Errors: Sendable {
/// No retries will happen
case none

Expand All @@ -21,7 +21,7 @@ public struct RetryConfiguration {
case transport

/// Test the errors with a user supplied closure. Custom errors are limited in the same way that ".all" is, there are certain types of errors (request formatting errors, cancellation, network timeouts) that this will NOT be called for and there is no option to retry. Note: will be called on a background thread so closure must be thread safe
case custom(retryTest: (NetableError) -> Bool)
case custom(retryTest: @Sendable (NetableError) -> Bool)

internal func shouldRetry(_ error: NetableError) -> Bool {
switch self {
Expand Down
4 changes: 4 additions & 0 deletions Netable/Netable/SmartUnwrap.swift
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@ public struct SmartUnwrap<T: Decodable>: Decodable {
}
}

public init(decodedType: T) {
self.decodedType = decodedType
}

public init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: DynamicCodingKeys.self)
let smartUnwrapKey = decoder.userInfo[.smartUnwrapKey] as? String
Expand Down
3 changes: 2 additions & 1 deletion Netable/NetableExample/Repository/PostRepository.swift
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,8 @@ class PostRepository {
func getPosts() {
Task { @MainActor in
do {
let posts = try await netable.request(GetPostsRequest())
let posts = try await self.netable.request(GetPostsRequest())
print(posts)
self.posts.send(posts)
} catch {
print(error)
Expand Down
2 changes: 2 additions & 0 deletions Netable/NetableExample/Request/GetPostsRequest.swift
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ struct GetPostsRequest: Request {

var path = "all"

var arrayDecodeStrategy: ArrayDecodeStrategy { .lossy }

var unredactedParameterKeys: Set<String> {
["title", "content"]
}
Expand Down
5 changes: 2 additions & 3 deletions Netable/NetableExample/Resources/JsonResponse/posts.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
{
"posts": [
"posts":[
{
"title": "first post",
"content": "em ipsum dolor sit amet, consectetur adipiscing elit. Proin in mattis magna. Pellentesque ac tortor nec lectus auctor egestas. Proin erat felis, finibus vitae condimentum at, viverra sed arcu."
"title": "first post"
},
{
"title": "second post",
Expand Down
Loading

0 comments on commit c6059d0

Please sign in to comment.