Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PM-1724] Sqlite WASM #924

Open
wants to merge 52 commits into
base: main
Choose a base branch
from
Open

[PM-1724] Sqlite WASM #924

wants to merge 52 commits into from

Conversation

Hinton
Copy link
Member

@Hinton Hinton commented Jul 22, 2024

🎟️ Tracking

https://bitwarden.atlassian.net/browse/PM-1724

📔 Objective

Implement persisted state backed by SQLite on our supported platforms. It provides an abstraction layer with a similar interface as the rusqlite crate with some minor differences required to support the web assembly interface.

⏰ Reminders before review

  • Contributor guidelines followed
  • All formatters and local linters executed and passed
  • Written new unit and / or integration tests where applicable
  • Protected functional changes with optionality (feature flags)
  • Used internationalization (i18n) for all UI strings
  • CI builds passed
  • Communicated to DevOps any deployment requirements
  • Updated any necessary documentation (Confluence, contributing docs) or informed the documentation
    team

🦮 Reviewer guidelines

  • 👍 (:+1:) or similar for great changes
  • 📝 (:memo:) or ℹ️ (:information_source:) for notes or general info
  • ❓ (:question:) for questions
  • 🤔 (:thinking:) or 💭 (:thought_balloon:) for more open inquiry that's not quite a confirmed
    issue and could potentially benefit from discussion
  • 🎨 (:art:) for suggestions / improvements
  • ❌ (:x:) or ⚠️ (:warning:) for more significant problems or concerns needing attention
  • 🌱 (:seedling:) or ♻️ (:recycle:) for future improvements or indications of technical debt
  • ⛏ (:pick:) for minor or nitpick changes

Hinton added 14 commits March 7, 2024 11:24
# Conflicts:
#	Cargo.lock
#	crates/bitwarden/src/vault/cipher/mod.rs
# Conflicts:
#	Cargo.lock
#	crates/bitwarden-core/src/.gitignore
#	crates/bitwarden-vault/src/cipher/repository.rs
#	crates/bitwarden-vault/src/sync.rs
#	crates/bitwarden/Cargo.toml
#	crates/bitwarden/src/client/client.rs
#	crates/bitwarden/src/client/mod.rs
#	crates/bitwarden/src/error.rs
#	crates/bitwarden/src/vault/client_vault.rs
Copy link
Contributor

github-actions bot commented Jul 22, 2024

Logo
Checkmarx One – Scan Summary & Detailse9099662-ca85-4a90-92a0-f09a3354a03b

No New Or Fixed Issues Found

@Hinton Hinton changed the base branch from ps/sqlite to main July 29, 2024 10:49
Hinton added 5 commits July 29, 2024 12:51
# Conflicts:
#	Cargo.lock
#	crates/bitwarden-core/Cargo.toml
#	crates/bitwarden-core/src/client/internal.rs
#	crates/bitwarden-vault/Cargo.toml
Copy link

codecov bot commented Aug 13, 2024

Codecov Report

Attention: Patch coverage is 80.32787% with 96 lines in your changes missing coverage. Please review.

Project coverage is 58.78%. Comparing base (6f63639) to head (914f016).
Report is 43 commits behind head on main.

Files with missing lines Patch % Lines
crates/bitwarden-db-tests/src/lib.rs 0.00% 50 Missing ⚠️
crates/bitwarden-db/src/migrator.rs 85.91% 10 Missing ⚠️
crates/bitwarden-vault/src/cipher/repository.rs 96.02% 6 Missing ⚠️
crates/bitwarden-db/src/sqlite.rs 96.45% 5 Missing ⚠️
crates/bitwarden-c/src/c.rs 0.00% 3 Missing ⚠️
crates/bitwarden-db-tests/bin/main.rs 0.00% 3 Missing ⚠️
crates/bitwarden-wasm/src/client.rs 0.00% 3 Missing ⚠️
crates/bw/src/main.rs 0.00% 3 Missing ⚠️
crates/bitwarden-core/src/client/client.rs 86.66% 2 Missing ⚠️
crates/bitwarden-json/src/client.rs 0.00% 2 Missing ⚠️
... and 6 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #924      +/-   ##
==========================================
+ Coverage   57.96%   58.78%   +0.81%     
==========================================
  Files         197      203       +6     
  Lines       13651    14085     +434     
==========================================
+ Hits         7913     8280     +367     
- Misses       5738     5805      +67     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Hinton Hinton marked this pull request as ready for review August 13, 2024 08:47
@Hinton Hinton requested review from a team as code owners August 13, 2024 08:47
@Hinton Hinton requested review from dani-garcia and MGibson1 August 13, 2024 08:48
Copy link
Contributor

@michalchecinski michalchecinski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Hinton Hinton changed the title Sqlite WASM [PM-1724] Sqlite WASM Aug 13, 2024
@Hinton Hinton mentioned this pull request Aug 19, 2024
Hinton added a commit that referenced this pull request Aug 19, 2024
## 🎟️ Tracking

<!-- Paste the link to the Jira or GitHub issue or otherwise describe /
point to where this change is coming from. -->

## 📔 Objective

<!-- Describe what the purpose of this PR is, for example what bug
you're fixing or new feature you're adding. -->

The #972 PR requires MSRV 1.73, and #924 will require. MSRV 1.75. To
avoid upgrading twice I suggest immediately changing to 1.75. This is
noted as a change in both the CLI and bitwarden change logs.

## ⏰ Reminders before review

- Contributor guidelines followed
- All formatters and local linters executed and passed
- Written new unit and / or integration tests where applicable
- Protected functional changes with optionality (feature flags)
- Used internationalization (i18n) for all UI strings
- CI builds passed
- Communicated to DevOps any deployment requirements
- Updated any necessary documentation (Confluence, contributing docs) or
informed the documentation
  team

## 🦮 Reviewer guidelines

<!-- Suggested interactions but feel free to use (or not) as you desire!
-->

- 👍 (`:+1:`) or similar for great changes
- 📝 (`:memo:`) or ℹ️ (`:information_source:`) for notes or general info
- ❓ (`:question:`) for questions
- 🤔 (`:thinking:`) or 💭 (`:thought_balloon:`) for more open inquiry
that's not quite a confirmed
  issue and could potentially benefit from discussion
- 🎨 (`:art:`) for suggestions / improvements
- ❌ (`:x:`) or ⚠️ (`:warning:`) for more significant problems or
concerns needing attention
- 🌱 (`:seedling:`) or ♻️ (`:recycle:`) for future improvements or
indications of technical debt
- ⛏ (`:pick:`) for minor or nitpick changes
# Conflicts:
#	.github/workflows/minimum-rust-version.yml
#	crates/bitwarden-c/src/c.rs
#	crates/bitwarden-core/Cargo.toml
#	crates/bitwarden-core/src/mobile/crypto.rs
#	crates/bitwarden-py/src/client.rs
#	crates/bitwarden-uniffi/src/lib.rs
#	crates/bitwarden-vault/Cargo.toml
#	crates/bitwarden/Cargo.toml
Copy link
Member

@dani-garcia dani-garcia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've got some comments, but looking pretty good

crates/bitwarden-json/Cargo.toml Outdated Show resolved Hide resolved
crates/bitwarden-napi/src/client.rs Outdated Show resolved Hide resolved
@@ -32,13 +32,13 @@ pub struct Client(bitwarden::Client);
impl Client {
/// Initialize a new instance of the SDK client
#[uniffi::constructor]
pub fn new(settings: Option<ClientSettings>) -> Arc<Self> {
pub async fn factory(settings: Option<ClientSettings>) -> Arc<Self> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

factory seems like a confusing name, I would almost expect this function to return a factory itself. Thoughts on using something else like create?

@@ -79,6 +117,8 @@ impl Client {
})),
external_client,
encryption_settings: RwLock::new(None),
#[cfg(feature = "state")]
db: Arc::new(Mutex::new(db)),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the Mutex around the Database here is needed, all the functions in the DatabaseTrait only take &self as far as I could see.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe, but ideally you shouldn't run multiple queries at the same time if we want to support transactions.


#[tokio::main(flavor = "current_thread")]
async fn main() {
run_tests().await;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests seem to leave the test.sqlite file hanging around after they finish when I run them with cargo run --bin bitwarden_db_tests.

That then causes the next run of the tests for me to fail, should we remove it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this crate be a part of the SDK or is it just a test to check that the POC works? Because if it's the former it would be nice to have it checked in CI

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should keep it around since it's a good suite to test the database behavior.

async fn get_version(&self) -> Result<usize, DatabaseError>;
async fn set_version(&self, version: usize) -> Result<(), DatabaseError>;

async fn execute_batch(&self, sql: &str) -> Result<(), DatabaseError>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to have execute_batch and query_map documented here, instead of having to click though into the rusqlite documentation

crates/bitwarden-db/src/migrator.rs Outdated Show resolved Hide resolved
@@ -66,6 +91,19 @@ impl Client {
api_key: None,
};

#[cfg(feature = "state")]
let db = {
let db = Database::default()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we pass some path to the database here? Otherwise two simultaneous SDK clients will overwrite each other.

pub async fn replace_all(&self, ciphers: &[Cipher]) -> Result<(), CipherRepositoryError> {
let guard = self.db.lock().await;

guard.execute("DELETE FROM ciphers", []).await?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice to be able to run the DELETE FROM and INSERT INTO in the same transaction, to be able to rollback the whole thing in cases there's an error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants