Merge Upstream Changes

Gemfile.lock

@@ -94,7 +94,7 @@ GEM
     redis-client (0.22.2)
       connection_pool
     regexp_parser (2.9.2)
-    rexml (3.3.8)
+    rexml (3.3.9)
     rubocop (1.66.1)
       json (~> 2.3)
       language_server-protocol (>= 3.17.0)
@@ -121,15 +121,15 @@ GEM
       bigdecimal
       concurrent-ruby (~> 1.0, >= 1.0.2)
     simpleidn (0.2.3)
-    sorbet (0.5.11608)
-      sorbet-static (= 0.5.11608)
-    sorbet-runtime (0.5.11608)
-    sorbet-static (0.5.11608-aarch64-linux)
-    sorbet-static (0.5.11608-universal-darwin)
-    sorbet-static (0.5.11608-x86_64-linux)
-    sorbet-static-and-runtime (0.5.11608)
-      sorbet (= 0.5.11608)
-      sorbet-runtime (= 0.5.11608)
+    sorbet (0.5.11641)
+      sorbet-static (= 0.5.11641)
+    sorbet-runtime (0.5.11641)
+    sorbet-static (0.5.11641-aarch64-linux)
+    sorbet-static (0.5.11641-universal-darwin)
+    sorbet-static (0.5.11641-x86_64-linux)
+    sorbet-static-and-runtime (0.5.11641)
+      sorbet (= 0.5.11641)
+      sorbet-runtime (= 0.5.11641)
     spoom (1.5.0)
       erubi (>= 1.10.0)
       prism (>= 0.28.0)

README.md

@@ -167,148 +167,162 @@ Regardless of how you obtained or built the image, follow these steps to run One
 
 OnetimeSecret should now be running and accessible at `http://localhost:3000`.
 
+
+Ah yes, the classic sudo paradox! Here's my attempt to handle this clearly:
+
+
 ### Manual Installation
 
-If you prefer to work with the source code directly, you can install OnetimeSecret manually. Follow these steps:
+This guide covers installing OnetimeSecret manually, whether you're working with an existing development environment or starting from a fresh system.
 
-#### 1. Get the Code
+#### Prerequisites
 
-Choose one of these methods:
+Required components:
+- [Redis Server](https://redis.io/download) (version 5 or higher)
+- [Ruby](https://www.ruby-lang.org/en/downloads/) (version 3.1 or higher)
+- [Bundler](https://bundler.io/) (version 2.5.x)
+- [Node.js](https://nodejs.org/en/download/) (version 20 or higher)
+- [pnpm](https://pnpm.io/installation) (version 9.2 or higher)
+- Essential build tools and development libraries
 
-* Download the [latest release](https://github.com/onetimesecret/onetimesecret/archive/refs/tags/latest.tar.gz)
-* Clone the repository:
+#### Installation Steps
 
-  ```bash
-  git clone https://github.com/onetimesecret/onetimesecret.git
-  ```
+##### 1. Prepare Your Environment
+
+First, verify if you have the required dependencies:
 
-#### 2. Install System Dependencies
+```bash
+ruby --version       # Should be 3.1+
+bundler --version    # Should be 2.5.x
+node --version       # Should be 20+
+pnpm --version       # Should be 9.2+
+redis-server -v      # Should be 5+
+```
 
-Follow these general steps to install the required system dependencies:
+For a fresh system installation, follow these steps:
 
-1. Install [Redis Server 5+](https://redis.io/download)
-2. Install [Ruby 3.1+](https://www.ruby-lang.org/en/downloads/) and [bundler 2.5*](https://bundler.io/)
-3. Install [Node.js 20+](https://nodejs.org/en/download/) and [pnpm 9.2+](https://pnpm.io/installation)
-4. Install additional required packages: build-essential, libyaml-dev, libffi-dev
+> [!Important]
+> If starting with a minimal system (like a fresh Debian container), install `sudo` first:
+>
+> ```bash
+> # Only if starting as root on a minimal system
+> apt update && apt install -y sudo
+> ```
 
-For Debian/Ubuntu systems, you can use the following commands:
+Install system dependencies:
 
 ```bash
-# Update package list and install basic dependencies
+# For Debian/Ubuntu systems:
 sudo apt update
-sudo apt install -y git redis-server build-essential libyaml-dev libffi-dev
-
-# Install Ruby 3.1
-sudo apt install -y ruby3.1 ruby3.1-dev
+sudo apt install -y git curl build-essential libyaml-dev libffi-dev redis-server ruby3.1 ruby3.1-dev
 
-# Install Node.js and pnpm
+# Install package managers
+sudo gem install bundler
 curl -fsSL https://deb.nodesource.com/setup_22.x | sudo -E bash -
 sudo apt install -y nodejs
 sudo npm install -g pnpm@latest
-```
 
-Note: After installation, make sure Redis is running with `service redis-server status`. Start it if needed with `service redis-server start`.
+# Start Redis server
+sudo service redis-server start
+```
 
-For other operating systems, please refer to the official documentation for each dependency to install the correct versions.
+> **Note:** If you see audit-related errors when installing pnpm with sudo, this is normal in containers or minimal systems where audit capabilities are limited.
 
-#### 3. Initialize the config files
+##### 2. Get the Source Code
 
 ```bash
+git clone https://github.com/onetimesecret/onetimesecret.git
 cd onetimesecret
-git rev-parse --short HEAD > .commit_hash.txt
-cp --preserve --no-clobber ./etc/config.example.yaml ./etc/config.yaml
 ```
 
-#### 4. Install Ruby Dependencies
+##### 3. Install Dependencies
 
 ```bash
-sudo gem install bundler
-bundle config set --local without 'development test'
-bundle update --bundler
+# Install Ruby dependencies
 bundle install
-```
 
-#### 5. Install JavaScript Dependencies
-
-```bash
-pnpm install --frozen-lockfile
+# Install Node.js dependencies
+pnpm install
 ```
 
-#### 6. Build the Frontend
+##### 4. Initialize Configuration
 
 ```bash
-pnpm run build
+git rev-parse --short HEAD > .commit_hash.txt
+cp -p ./etc/config.example.yaml ./etc/config.yaml
 ```
 
+##### 5. Choose Your Running Mode
 
-#### 7. Run the Web Application
+You can run the application in two ways:
 
-There are two main ways to run the application, depending on your development needs:
+###### Option A: Standard Mode (Static Frontend, Choose RACK_ENV)
 
-##### Option A: Without Vite Dev Server (Production-like or Simple Development)
+Best for production or development without frontend changes:
 
-1. For production or simple development without frontend changes:
+1. Build frontend assets (optional, pre-built assets included):
 
-   ```bash
-   RACK_ENV=production bundle exec thin -R config.ru -p 3000 start
-   ```
+```bash
+pnpm run build:local
+```
 
-   Or, for a development environment with a static frontend and Ruby live reloading:
+2. Set development mode to false in `etc/config.yaml`:
 
-   ```bash
-   RACK_ENV=development bundle exec thin -R config.ru -p 3000 start
-   ```
+```yaml
+:development:
+  :enabled: false
+```
 
-   Ensure `development.enabled` is set to `false` in `etc/config.yaml`:
+3. Start the server (choose environment as needed):
 
-   ```yaml
-   :development:
-     :enabled: false
-   ```
+```bash
+# For production
+RACK_ENV=production bundle exec thin -R config.ru -p 3000 start
 
-   This uses pre-built frontend assets in the `dist/assets` directory.
+# Or for backend development
+RACK_ENV=development bundle exec thin -R config.ru -p 3000 start
+```
 
-##### Option B: With Vite Dev Server (Active Frontend Development)
+###### Option B: Frontend Development Mode
 
-1. Set `development.enabled` to `true` in `etc/config.yaml`:
+Best for active frontend development with live reloading:
 
+1. Enable development mode in `etc/config.yaml`:
    ```yaml
    :development:
      :enabled: true
    ```
 
-2. Run the Thin server in development mode:
-
+2. Start the main server:
    ```bash
    RACK_ENV=development bundle exec thin -R config.ru -p 3000 start
    ```
 
-3. In a separate terminal, start the Vite dev server:
-
+3. Start the Vite dev server (in a separate terminal):
    ```bash
    pnpm run dev
    ```
 
-   This enables live reloading of frontend assets.
+#### Technical Details for Frontend Development
 
-The application determines whether to use development or production assets based on the `development.enabled` setting. In development mode with the Vite server running, frontend assets are loaded dynamically:
+When running in development mode (Option B), the application uses Vite's dev server for dynamic asset loading and hot module replacement. Here's how it works:
 
-```html
-{{#frontend_development}}
-<script type="module" src="{{ frontend_host }}/dist/main.ts"></script>
-<script type="module" src="{{ frontend_host }}/dist/@vite/client"></script>
-{{/frontend_development}}
-```
-
-In production mode, it uses the built files in `dist/assets`:
+- In development mode (`development.enabled: true`), the application loads assets dynamically from the Vite dev server:
+  ```html
+  {{#frontend_development}}
+  <script type="module" src="{{ frontend_host }}/dist/main.ts"></script>
+  <script type="module" src="{{ frontend_host }}/dist/@vite/client"></script>
+  {{/frontend_development}}
+  ```
 
-```html
-{{^frontend_development}}
-{{{vite_assets}}}
-{{/frontend_development}}
-```
+- In production mode (`development.enabled: false`), it uses pre-built static assets:
+  ```html
+  {{^frontend_development}}
+  {{{vite_assets}}}
+  {{/frontend_development}}
+  ```
 
-Choose the option that best fits your development workflow and needs.
+This setup enables features like hot module replacement and instant updates during frontend development, while ensuring optimal performance in production.
 
 ## Configuration
 
@@ -319,7 +333,7 @@ OnetimeSecret requires a `config.yaml` file for all installations. Environment v
 1. Create the configuration file:
 
    ```bash
-   cp --preserve --no-clobber ./etc/config.example.yaml ./etc/config.yaml
+   cp -p ./etc/config.example.yaml ./etc/config.yaml
    ```
 
 2. Review and edit `./etc/config.yaml` as needed. At minimum, update the secret key and back it up securely.
@@ -370,7 +384,7 @@ For various deployment scenarios, including Docker setups and local development,
 1. Create the .env file:
 
    ```bash
-   cp --preserve --no-clobber .env.example .env
+   cp -p .env.example .env
    ```
 
 2. Edit the `.env` file with your desired configuration.

bin/ots

@@ -15,6 +15,10 @@ require 'drydock'
 class OT::CLI::Definition
   extend Drydock
 
+  before do
+    puts "" # separate our content from the bootstrapping banner
+  end
+
   global :v, :verbose, 'Increase output' do
     @verbose ||= 0
     @verbose += 1
@@ -41,7 +45,19 @@ class OT::CLI::Definition
   command move_keys: OT::CLI
 
   usage 'ots customers'
+  option :l, :list, "List customer domains (by count)"
+  option :c, :check, "Show customer records where custid and email do not match (obscured)"
   command customers: OT::CLI
+
+  usage 'ots domains'
+  option :l, :list, "List domains"
+  command domains: OT::CLI
+
+  usage 'ots revalidate-domains'
+  about 'Revalidate domain verification status. Optionally specify a domain and/or customer_id.'
+  option :d, :domain, String, "Domain to revalidate"
+  option :c, :custid, String, "Customer ID to revalidate"
+  command revalidate_domains: OT::CLI
 end
 
 begin

config.ru

@@ -21,6 +21,15 @@ end
 # Load Paths
 $LOAD_PATH.unshift(File.join(app_root, 'lib'))
 
+# Freshly installed operating systems don't always have their locale settings
+# figured out. By setting this to UTF-8, we ensure that:
+# - All file I/O operations default to UTF-8 encoding.
+# - Network I/O operations treat data as UTF-8 encoded.
+# - Standard input/output (STDIN, STDOUT, STDERR) uses UTF-8 encoding.
+# - Strings created from external sources default to UTF-8 encoding.
+# This helps maintain consistency and reduces encoding-related issues.
+Encoding.default_external = Encoding::UTF_8
+
 # Required Libraries
 require 'rack/content_length'
 require 'rack/contrib'

etc/config.example.yaml

@@ -1,4 +1,3 @@
----
 :site:
   :host: <%= ENV['HOST'] || 'localhost:3000' %>
   :domains:
@@ -62,8 +61,30 @@
     # These options will be presented to users when they create a new secret
     # Format: String of integers representing seconds
     :ttl_options: <%= (ENV['TTL_OPTIONS'] || nil) %>
+# Redis Configuration
 :redis:
+  # Main Redis connection URI - Specify full connection string including auth
+  # Format: redis://[:password@]host[:port]/[db-number]
+  # Examples:
+  #   - redis://mypassword@localhost:6379/0        # Simple password auth
+  #   - redis://user:pass@localhost:6379/0         # Username/password auth
+  #   - redis://redis.example.com:6379/0          # No auth (development only)
   :uri: <%= ENV['REDIS_URL'] || 'redis://[email protected]:6379/0' %>
+  # Database Mapping Configuration
+  # These mappings allow separating data across different Redis logical
+  # databases (0-15). These can be arranged any way you like.
+  # Note: If using a Redis provider with single-database limitation (e.g., Upstash),
+  # set all values to 0.
+  :dbs:
+    :session: 1
+    :splittest: 1
+    :custom_domain: 6
+    :customer: 6
+    :subdomain: 6
+    :metadata: 7
+    :email_receipt: 8
+    :secret: 8
+    :feedback: 11
 :colonels:
   # Email addresses listed below will be granted automatic
   # administrative privileges upon account creation. These