Merge F&F release branch into main (#31)

* Secure Vault autofill Pixel support (#26) * add delegate calls after autofilling * upd with AutofillType * Secure Vault additions (#28) * Finish note support. * Note database migration * Begin working on identities * Further work on Identities * Consolidate database migrations. * Fill out remaining credit card functions. * Remove unneeded credit card properties. * Further autofill API work. * Support cardholder name. * Fix unit test compilation failures. * Remove an incorrect marker comment. * Add better credit card validation. * Bump the Autofill commit. * Add tests for the new autofill Get methods. * Clean up the database access calls. * Clean up a number of SwiftLint warnings and errors. * Update the Autofill commit. * Fill in the autofill type delegate methods. * Ignore the release script. * Bump autofill Signed-off-by: Emanuele Feliziani <[email protected]> Co-authored-by: Emanuele Feliziani <[email protected]> * Bump autofill to version 3.3.0 (#29) * Bump autofill to version 3.3.0 Signed-off-by: Emanuele Feliziani <[email protected]> * Update the duckduckgo-autofill submodule commit. Co-authored-by: Sam Symons <[email protected]> * Add requiresRunInPageContentWorld to run UserScripts in page world (#30) * Ensure that the WKContentWorld usage compiles on iOS. (#32) * Fix lastName in identity (#33) * Update the credit card table (#34) * Upgrade the credit card number field to L2. * Correctly provide the last name for Identity autofill. * Allow the SecureVaultFactory to return the crypto classes only. * Ensure that addressStreet2 is encoded and read correctly. * Update the migration to remove the old column. * Generate a display title and subtitle for notes. * Add a new error type for if no L1 key is available. * Move content blocking code to BSK (#35) * Initial refactoring of content blocking to enable sharing between platforms * Extracted Content Rules script * Tweaks based on Mac OS X implementation * Expand API to enable checking for result of the requested compilation * Add option to obtain PrivacyFeature settings dictionary * Add Surrogates user script * API Tweaks * Fix tests * Content Blocking reference tests * More Pixels * Add tests for User Scripts and Tracker allowlist * Reuse sha256 code in tests * Better Privacy Config tests * Add autoconsent as a configurable feature (#39) * Remove unused TrackingProtectionStats dead code (#41) * Add identities autofill (#38) * Bump autofill pointer Signed-off-by: Emanuele Feliziani <[email protected]> * Bump autofill revision Signed-off-by: Emanuele Feliziani <[email protected]> * Bump autofill to latest version Signed-off-by: Emanuele Feliziani <[email protected]> * Bump autofill revision (#46) Signed-off-by: Emanuele Feliziani <[email protected]> * Support content scope as a user script (#45) * Remove content scope files causing warnings (#49) * Navigator credentials move (#47) * Fix unprotected domains calculation for about:blank urls (#51) * Add Resolver tests and fix cname resolving (#50) * Multiple blocking rule lists (#53) * Initial implementation to support multiple rules lists * Make API open for inheritance * Fix few bugs, update tests * CTL delegate WIP * Update ContentBlockerRulesUserScript.swift * Update ContentBlockerRulesUserScript.swift * Refactor Content Blocker User Script * Add missing change information * Expand error reporting API * Unit tests for error reporting * Workaround for missing TDS at startup issue * Move model processing to work queue * Kill the app only in case main TDS rule list fails to compile * Allow for non-TDS rule lists to fail compilation in case setup fails * Add check for embedded TDS compilation Co-authored-by: Bartek Waresiak <[email protected]> Co-authored-by: [email protected] <[email protected]> * Add platform.name to BrowserServices kit (#48) * Add platform.name to BrowserServices kit * be explicit about supported platform names * add clickToPlay config (#56) Co-authored-by: [email protected] <[email protected]> * Enable unit tests via Xcode (#57) * Have all bundle resources copied to the root directory. * Remove a duplicate file. * Move embedded TDS to platform code (#58) * Filtering various suggestion types from Top Hits section of suggestions (#55) * Filtering various suggestion types from Top Hits section of suggestions * Suggesting bookmarks instead of history entries with the same URL * Allowing bookmarks to be in the Top Hits if they replaced history entry * isDownload flag removed * Edge case with bookmark not allowed in Top Hits resolved * Array renamed to all * Fix Privcy Config API to correcty take into account feature type (#62) * Update config to reflect iOS features (#64) * Update Autofill submodule pointer with iOS fix (#65) * Update the duckduckgo-autofill submodule commit. * Resolve package warnings Co-authored-by: Bartek Waresiak <[email protected]> Co-authored-by: Alexey Martemyanov <[email protected]> Co-authored-by: Emanuele Feliziani <[email protected]> Co-authored-by: Jonathan Kingston <[email protected]> Co-authored-by: bwaresiak <[email protected]> Co-authored-by: Sam Macbeth <[email protected]> Co-authored-by: Brad Slayter <[email protected]> Co-authored-by: [email protected] <[email protected]> Co-authored-by: Shane Osbourne <[email protected]> Co-authored-by: Lucas Adamski <[email protected]> Co-authored-by: Tomas Strba <[email protected]>
duckduckgo · Feb 9, 2022 · b86c166 · b86c166
1 parent 5eb7cde
commit b86c166
Show file tree

Hide file tree

Showing 75 changed files with 10,035 additions and 105 deletions.
diff --git a/.gitmodules b/.gitmodules
@@ -1,3 +1,6 @@
 [submodule "Sources/BrowserServicesKit/Resources/duckduckgo-autofill"]
 	path = Sources/BrowserServicesKit/Resources/duckduckgo-autofill
 	url = https://github.com/duckduckgo/duckduckgo-autofill
+[submodule "Sources/BrowserServicesKit/Resources/content-scope-scripts"]
+	path = Sources/BrowserServicesKit/Resources/content-scope-scripts
+	url = https://github.com/duckduckgo/content-scope-scripts
diff --git a/Package.resolved b/Package.resolved
@@ -9,6 +9,24 @@
           "revision": "9efe5a515acff8b73f69a31a65fc2bce2a823219",
           "version": "1.1.0"
         }
+      },
+      {
+        "package": "swift-argument-parser",
+        "repositoryURL": "https://github.com/apple/swift-argument-parser",
+        "state": {
+          "branch": null,
+          "revision": "6b2aa2748a7881eebb9f84fb10c01293e15b52ca",
+          "version": "0.5.0"
+        }
+      },
+      {
+        "package": "TrackerRadarKit",
+        "repositoryURL": "https://github.com/duckduckgo/TrackerRadarKit",
+        "state": {
+          "branch": null,
+          "revision": "5f4caf35b8418700a48c64c7c61eb43308c8dacc",
+          "version": "1.0.3"
+        }
       }
     ]
   },

diff --git a/Package.swift b/Package.swift
@@ -11,18 +11,33 @@ let package = Package(
         .macOS("10.15")
     ],
     products: [
-        .library(name: "BrowserServicesKit", targets: ["BrowserServicesKit"]),
+        .library(name: "BrowserServicesKit", targets: ["BrowserServicesKit"])
     ],
     dependencies: [
-        .package(name: "GRDB", url: "https://github.com/duckduckgo/GRDB.swift.git", .exact("1.1.0"))
+        .package(name: "GRDB", url: "https://github.com/duckduckgo/GRDB.swift.git", .exact("1.1.0")),
+        .package(url: "https://github.com/duckduckgo/TrackerRadarKit", .exact("1.0.3"))
     ],
     targets: [
         .target(
             name: "BrowserServicesKit",
             dependencies: [
-                "GRDB"
+                "GRDB",
+                "TrackerRadarKit"
             ],
             exclude: [
+                "Resources/content-scope-scripts/README.md",
+                "Resources/content-scope-scripts/package-lock.json",
+                "Resources/content-scope-scripts/package.json",
+                "Resources/content-scope-scripts/LICENSE.md",
+                "Resources/content-scope-scripts/src/",
+                "Resources/content-scope-scripts/unit-test/",
+                "Resources/content-scope-scripts/integration-test/",
+                "Resources/content-scope-scripts/scripts/",
+                "Resources/content-scope-scripts/inject/",
+                "Resources/content-scope-scripts/lib/",
+                "Resources/content-scope-scripts/build/chrome/",
+                "Resources/content-scope-scripts/build/firefox/",
+                "Resources/content-scope-scripts/build/integration/",
                 "Resources/duckduckgo-autofill/Gruntfile.js",
                 "Resources/duckduckgo-autofill/package.json",
                 "Resources/duckduckgo-autofill/package-lock.json",
@@ -34,18 +49,28 @@ let package = Package(
                 "Resources/duckduckgo-autofill/jest.setup.js",
                 "Resources/duckduckgo-autofill/dist/autofill-host-styles_chrome.css",
                 "Resources/duckduckgo-autofill/jest.config.js",
-                "Resources/duckduckgo-autofill/jest-test-environment.js"
+                "Resources/duckduckgo-autofill/jest-test-environment.js",
+                "Resources/duckduckgo-autofill/scripts/release.js",
+                "Resources/duckduckgo-autofill/jesthtmlreporter.config.json",
+                "Resources/duckduckgo-autofill/types.d.ts",
+                "Resources/duckduckgo-autofill/tsconfig.json",
+                "Resources/duckduckgo-autofill/docs/real-world-html-tests.md",
+                "Resources/duckduckgo-autofill/docs/matcher-configuration.md"
             ],
             resources: [
-                .process("Resources/duckduckgo-autofill/dist/autofill.js")
+                .process("Resources/duckduckgo-autofill/dist/autofill.js"),
+                .process("ContentBlocking/UserScripts/contentblockerrules.js"),
+                .process("ContentBlocking/UserScripts/surrogates.js"),
+                .process("Resources/content-scope-scripts/build/apple/contentScope.js")
             ]),
         .testTarget(
             name: "BrowserServicesKitTests",
             dependencies: [
                 "BrowserServicesKit"
             ],
             resources: [
-                .copy("UserScript/testUserScript.js")
+                .process("UserScript/testUserScript.js"),
+                .process("Resources")
             ])
     ]
 )
diff --git a/Sources/BrowserServicesKit/Autofill/AutofillUserScript+Email.swift b/Sources/BrowserServicesKit/Autofill/AutofillUserScript+Email.swift
@@ -34,7 +34,6 @@ public protocol AutofillEmailDelegate: AnyObject {
 
 extension AutofillUserScript {
 
-
     func emailCheckSignedInStatus(_ message: WKScriptMessage, _ replyHandler: MessageReplyHandler) {
         let signedIn = emailDelegate?.autofillUserScriptDidRequestSignedInStatus(self) ?? false
         let signedInString = String(signedIn)
@@ -97,5 +96,4 @@ extension AutofillUserScript {
         }
     }
 
-
 }
diff --git a/Sources/BrowserServicesKit/Autofill/AutofillUserScript+SecureVault.swift b/Sources/BrowserServicesKit/Autofill/AutofillUserScript+SecureVault.swift
@@ -21,28 +21,155 @@ import WebKit
 
 public protocol AutofillSecureVaultDelegate: AnyObject {
 
+    func autofillUserScript(_: AutofillUserScript, didRequestAutoFillInitDataForDomain domain: String, completionHandler: @escaping (
+        [SecureVaultModels.WebsiteAccount],
+        [SecureVaultModels.Identity],
+        [SecureVaultModels.CreditCard]
+    ) -> Void)
+
     func autofillUserScript(_: AutofillUserScript, didRequestPasswordManagerForDomain domain: String)
     func autofillUserScript(_: AutofillUserScript, didRequestStoreCredentialsForDomain domain: String, username: String, password: String)
     func autofillUserScript(_: AutofillUserScript, didRequestAccountsForDomain domain: String,
                             completionHandler: @escaping ([SecureVaultModels.WebsiteAccount]) -> Void)
     func autofillUserScript(_: AutofillUserScript, didRequestCredentialsForAccount accountId: Int64,
                             completionHandler: @escaping (SecureVaultModels.WebsiteCredentials?) -> Void)
+    func autofillUserScript(_: AutofillUserScript, didRequestCreditCardWithId creditCardId: Int64,
+                            completionHandler: @escaping (SecureVaultModels.CreditCard?) -> Void)
+    func autofillUserScript(_: AutofillUserScript, didRequestIdentityWithId identityId: Int64,
+                            completionHandler: @escaping (SecureVaultModels.Identity?) -> Void)
 
 }
 
 extension AutofillUserScript {
 
-    struct RequestVaultAccountsResponse: Codable {
+    // MARK: - Response Objects
 
-        struct Account: Codable {
-            let id: Int64
-            let username: String
-            let lastUpdated: TimeInterval
+    struct IdentityObject: Codable {
+        let id: Int64
+        let title: String
+
+        let firstName: String?
+        let middleName: String?
+        let lastName: String?
+
+        let birthdayDay: Int?
+        let birthdayMonth: Int?
+        let birthdayYear: Int?
+
+        let addressStreet: String?
+        let addressStreet2: String?
+        let addressCity: String?
+        let addressProvince: String?
+        let addressPostalCode: String?
+        let addressCountryCode: String?
+
+        let phone: String?
+        let emailAddress: String?
+
+        static func from(identity: SecureVaultModels.Identity) -> IdentityObject? {
+            guard let id = identity.id else { return nil }
+
+            return IdentityObject(id: id,
+                                  title: identity.title,
+                                  firstName: identity.firstName,
+                                  middleName: identity.middleName,
+                                  lastName: identity.lastName,
+                                  birthdayDay: identity.birthdayDay,
+                                  birthdayMonth: identity.birthdayMonth,
+                                  birthdayYear: identity.birthdayYear,
+                                  addressStreet: identity.addressStreet,
+                                  addressStreet2: identity.addressStreet2,
+                                  addressCity: identity.addressCity,
+                                  addressProvince: identity.addressProvince,
+                                  addressPostalCode: identity.addressPostalCode,
+                                  addressCountryCode: identity.addressCountryCode,
+                                  phone: identity.homePhone, // Replace with single "phone number" column
+                                  emailAddress: identity.emailAddress)
+        }
+    }
+
+    struct CreditCardObject: Codable {
+        let id: Int64
+        let title: String
+        let displayNumber: String
+
+        let cardName: String?
+        let cardNumber: String?
+        let cardSecurityCode: String?
+        let expirationMonth: Int?
+        let expirationYear: Int?
+
+        static func from(card: SecureVaultModels.CreditCard) -> CreditCardObject? {
+            guard let id = card.id else { return nil }
+
+            return CreditCardObject(id: id,
+                                    title: card.title,
+                                    displayNumber: card.displayName,
+                                    cardName: card.cardholderName,
+                                    cardNumber: card.cardNumber,
+                                    cardSecurityCode: card.cardSecurityCode,
+                                    expirationMonth: card.expirationMonth,
+                                    expirationYear: card.expirationYear)
+        }
+
+        /// Provides a minimal summary of the card, suitable for presentation in the credit card selection list. This intentionally omits secure data, such as card number and cardholder name.
+        static func autofillInitializationValueFrom(card: SecureVaultModels.CreditCard) -> CreditCardObject? {
+            guard let id = card.id else { return nil }
+
+            return CreditCardObject(id: id,
+                                    title: card.title,
+                                    displayNumber: card.displayName,
+                                    cardName: nil,
+                                    cardNumber: nil,
+                                    cardSecurityCode: nil,
+                                    expirationMonth: nil,
+                                    expirationYear: nil)
+        }
+    }
+
+    struct CredentialObject: Codable {
+        let id: Int64
+        let username: String
+    }
+
+    // MARK: - Responses
+
+    // swiftlint:disable nesting
+    struct RequestAutoFillInitDataResponse: Codable {
+
+        struct AutofillInitSuccess: Codable {
+            let credentials: [CredentialObject]
+            let creditCards: [CreditCardObject]
+            let identities: [IdentityObject]
         }
 
-        let success: [Account]
+        let success: AutofillInitSuccess
+        let error: String?
+
+    }
+    // swiftlint:enable nesting
+
+    struct RequestAutoFillCreditCardResponse: Codable {
+
+        let success: CreditCardObject
+        let error: String?
+
     }
 
+    struct RequestAutoFillIdentityResponse: Codable {
+
+        let success: IdentityObject
+        let error: String?
+
+    }
+
+    struct RequestVaultAccountsResponse: Codable {
+
+        let success: [CredentialObject]
+
+    }
+
+    // swiftlint:disable nesting
     struct RequestVaultCredentialsResponse: Codable {
 
         struct Credential: Codable {
@@ -54,6 +181,33 @@ extension AutofillUserScript {
 
         let success: Credential
 
+    }
+    // swiftlint:enable nesting
+
+    // MARK: - Message Handlers
+
+    func pmGetAutoFillInitData(_ message: WKScriptMessage, _ replyHandler: @escaping MessageReplyHandler) {
+
+        let domain = hostProvider.hostForMessage(message)
+        vaultDelegate?.autofillUserScript(self, didRequestAutoFillInitDataForDomain: domain) { accounts, identities, cards in
+            let credentials: [CredentialObject] = accounts.compactMap {
+                guard let id = $0.id else { return nil }
+                return .init(id: id, username: $0.username)
+            }
+
+            let identities: [IdentityObject] = identities.compactMap(IdentityObject.from(identity:))
+            let cards: [CreditCardObject] = cards.compactMap(CreditCardObject.autofillInitializationValueFrom(card:))
+
+            let success = RequestAutoFillInitDataResponse.AutofillInitSuccess(credentials: credentials,
+                                                                              creditCards: cards,
+                                                                              identities: identities)
+
+            let response = RequestAutoFillInitDataResponse(success: success, error: nil)
+            if let json = try? JSONEncoder().encode(response), let jsonString = String(data: json, encoding: .utf8) {
+                replyHandler(jsonString)
+            }
+        }
+
     }
 
     func pmStoreCredentials(_ message: WKScriptMessage, _ replyHandler: @escaping MessageReplyHandler) {
@@ -74,9 +228,9 @@ extension AutofillUserScript {
     func pmGetAccounts(_ message: WKScriptMessage, _ replyHandler: @escaping MessageReplyHandler) {
 
         vaultDelegate?.autofillUserScript(self, didRequestAccountsForDomain: hostProvider.hostForMessage(message)) { credentials in
-            let credentials: [RequestVaultAccountsResponse.Account] = credentials.compactMap {
+            let credentials: [CredentialObject] = credentials.compactMap {
                 guard let id = $0.id else { return nil }
-                return .init(id: id, username: $0.username, lastUpdated: $0.lastUpdated.timeIntervalSince1970)
+                return .init(id: id, username: $0.username)
             }
 
             let response = RequestVaultAccountsResponse(success: credentials)
@@ -110,6 +264,54 @@ extension AutofillUserScript {
         }
     }
 
+    func pmGetCreditCard(_ message: WKScriptMessage, _ replyHandler: @escaping MessageReplyHandler) {
+        guard let body = message.body as? [String: Any],
+              let id = body["id"] as? String,
+              let cardId = Int64(id) else {
+            return
+        }
+
+        vaultDelegate?.autofillUserScript(self, didRequestCreditCardWithId: Int64(cardId)) {
+            guard let card = $0, let cardObject = CreditCardObject.from(card: card) else { return }
+
+            let response = RequestAutoFillCreditCardResponse(success: cardObject, error: nil)
+
+            if let json = try? JSONEncoder().encode(response), let jsonString = String(data: json, encoding: .utf8) {
+                replyHandler(jsonString)
+            }
+        }
+    }
+
+    func pmGetIdentity(_ message: WKScriptMessage, _ replyHandler: @escaping MessageReplyHandler) {
+        guard let body = message.body as? [String: Any],
+              let id = body["id"] as? String,
+              let accountId = Int64(id) else {
+            return
+        }
+
+        vaultDelegate?.autofillUserScript(self, didRequestIdentityWithId: Int64(accountId)) {
+            guard let identity = $0, let identityObject = IdentityObject.from(identity: identity) else { return }
+
+            let response = RequestAutoFillIdentityResponse(success: identityObject, error: nil)
+
+            if let json = try? JSONEncoder().encode(response), let jsonString = String(data: json, encoding: .utf8) {
+                replyHandler(jsonString)
+            }
+        }
+    }
+
+    // MARK: Open Management Views
+
+    func pmOpenManageCreditCards(_ message: WKScriptMessage, _ replyHandler: @escaping MessageReplyHandler) {
+        vaultDelegate?.autofillUserScript(self, didRequestPasswordManagerForDomain: hostProvider.hostForMessage(message))
+        replyHandler(nil)
+    }
+
+    func pmOpenManageIdentities(_ message: WKScriptMessage, _ replyHandler: @escaping MessageReplyHandler) {
+        vaultDelegate?.autofillUserScript(self, didRequestPasswordManagerForDomain: hostProvider.hostForMessage(message))
+        replyHandler(nil)
+    }
+
     func pmOpenManagePasswords(_ message: WKScriptMessage, _ replyHandler: @escaping MessageReplyHandler) {
         vaultDelegate?.autofillUserScript(self, didRequestPasswordManagerForDomain: hostProvider.hostForMessage(message))
         replyHandler(nil)