Skip to content
This repository has been archived by the owner on Nov 16, 2023. It is now read-only.

Latest commit

 

History

History
521 lines (447 loc) · 15.5 KB

README.md

File metadata and controls

521 lines (447 loc) · 15.5 KB
Raw

crux-api

A Chrome UX Report API wrapper that supports batching, handles errors, and provides types.

Motivation: CrUX API is a fantastic tool to get RUM data without installing any script. While using the API in Treo, we discovered a few complex cases like API errors, rate limits, not found entries, a complicated multipart response from the batch API, URLs normalization, and TypeScript notations. We decided to build the crux-api library to makes it easier to work with the CrUX API.

Features:

  • A tiny (450 bytes) wrapper for Chrome UX Report API;
  • Batch API support for up to 1000 records per one request;
  • TypeScript notations for options and responses;
  • Isomorphic: works in a browser and node.js;
  • Returns null for the 404 (CrUX data not found) response;
  • Automatic retry when hits the API rate limits: 429 (Quota exceeded);
  • URL normalization helper to match the CrUX API index;

Usage

Install:

npm install crux-api

Fetch URL-level data for a various form factors and connections:

import { createQueryRecord } from 'crux-api'
const queryRecord = createQueryRecord({ key: CRUX_API_KEY })

const res1 = await queryRecord({ url: 'https://www.github.com/' }) // fetch all dimensions
const res2 = await queryRecord({ url: 'https://www.github.com/explore', formFactor: 'DESKTOP' }) // fetch data for desktop devices

Use the CrUX Batch API to combine up to 1000 requests and get results in less than 1 second:

import { createBatch } from 'crux-api/batch'
const batch = createBatch({ key: CRUX_API_KEY })

const records = await batch([
  { url: 'https://github.com/', formFactor: 'MOBILE', effectiveConnectionType: '4G' },
  { url: 'https://github.com/marketplace', formFactor: 'DESKTOP' },
  { url: 'https://www.github.com/explore', formFactor: 'TABLET' },
  // ... up to 1000 records.
])

Fetch origin-level data in node.js using node-fetch:

import { createQueryRecord } from 'crux-api'
import nodeFetch from 'node-fetch'
const queryRecord = createQueryRecord({ key: process.env.CRUX_API_KEY, fetch: nodeFetch })

const res1 = await queryRecord({ origin: 'https://github.com' })
const res2 = await queryRecord({
  origin: 'https://www.github.com/marketplace?type=actions',
  formFactor: 'DESKTOP',
  effectiveConnectionType: '4G',
})

Result is null or an object with queryRecord response body, ex:

{
  "record": {
    "key": {
      "formFactor": "DESKTOP",
      "effectiveConnectionType": "4G",
      "url": "https://github.com/marketplace"
    },
    "metrics": {
      "first_contentful_paint": {
        "histogram": [
          { "start": 0, "end": 1000, "density": 0.454180602006688 },
          { "start": 1000, "end": 3000, "density": 0.52575250836120291 },
          { "start": 3000, "density": 0.020066889632107024 }
        ],
        "percentiles": {
          "p75": 1365
        }
      },
      "cumulative_layout_shift": { ... },
      "first_input_delay": { ... },
      "largest_contentful_paint": { ... },
    }
  },
  "urlNormalizationDetails": {
    "originalUrl": "https://github.com/marketplace?type=actions",
    "normalizedUrl": "https://github.com/marketplace"
  }
}

API

Single Record Request

createQueryRecord(createOptions)

Returns a queryRecord function.

  • createOptions.key (required) - CrUX API key, use https://goo.gle/crux-api-key to generate a new key;
  • createOptions.fetch (optional, default: window.fetch) - pass a WHATWG fetch implementation for a non-browser environment;

queryRecord(queryOptions)

Fetches CrUX API using queryRecord options:

  • queryOptions.url or queryOptions.origin (required) – the main identifier for a record lookup;
  • queryOptions.formFactor (optional, defaults to all form factors) - the form factor dimension: PHONE | DESKTOP | TABLET;
  • queryOptions.effectiveConnectionType (optional, defaults to all connections) - the effective network class: 4G | 3G | 2G | slow-2G | offline.

Returns a Promise with a raw queryRecord response or null when the data is not found.

import { createQueryRecord } from 'crux-api'

const queryRecord = createQueryRecord({ key: process.env.CRUX_API_KEY })
const res = await queryRecord({
  url: 'https://github.com/marketplace?type=actions',
  formFactor: 'DESKTOP',
  effectiveConnectionType: '4G',
})

// res -> URL-level data for https://github.com/marketplace

Batch Request

crux-api/batch uses the CrUX Batch API, which allows combining 1000 calls in a single batch request. It's a separate namespace because the API is different, and it's bigger (850 bytes) due to the complexity of constructing and parsing multipart requests.

Note: A set of n requests batched together counts toward your usage limit as n requests, not as one request. That's why the sometimes a batch response contains 429 responses. But the crux-api automatically retries these responses, aiming always to return the data you need.

createBatch(createOptions)

Accepts the same createOptions as the createQueryRecord and returns a batch function.

batch(batchOptions)

Accepts an array of queryRecord options and returns an array with an exact position for each record. If the record doesn't exist in CrUX index, the value set to null. If some requests hit rate-limit, batch will retry them after a short timeout.

import { createBatch } from 'crux-api/batch'
import nodeFetch from 'node-fetch'

const batch = createBatch({ key: process.env.CRUX_KEY, fetch: nodeFetch })
const res = await batch([
  { origin: 'https://example.com' },
  { url: 'https://github.com/', formFactor: 'DESKTOP' },
  { origin: 'https://fooo.bar' },
])

// res[0] -> origin-level data for https://example.com
// res[1] -> URL-level data for https://github.com/ on desktop devices
// res[2] -> null (invalid origin that not found in the CrUX index)

normalizeUrl(url)

Normalize a URL to match the CrUX API internal index. It is a URL's origin + pathname (source).

import { normalizeUrl } from 'crux-api'

console.log(normalizeUrl('https://github.com/marketplace?type=actions')) // https://github.com/marketplace (removes search params)
console.log(normalizeUrl('https://github.com')) // https://github.com/ (adds "/" to the end)

CrUX API Responses

The crux-api is designed to return data and automatically handles errors. It returns an object for 200 responses, retries after 429, and returns null for 404. For 400 and 5xx it throws a relevant error.

Below are all known responses of Chrome UX Report API for easier debugging and development (The API documentation is vague about errors, please, submit an issue, if you know other responses).

✅ 200: URL-level data
curl -d url='https://github.com/marketplace?type=actions' \
     -d effectiveConnectionType=4G \
     -d formFactor=PHONE \
     'https://chromeuxreport.googleapis.com/v1/records:queryRecord?key=CRUX_API_KEY'
{
  "record": {
    "key": {
      "formFactor": "PHONE",
      "effectiveConnectionType": "4G",
      "url": "https://github.com/marketplace"
    },
    "metrics": {
      "cumulative_layout_shift": {
        "histogram": [
          {
            "start": "0.00",
            "end": "0.10",
            "density": 0.74598930481283388
          },
          {
            "start": "0.10",
            "end": "0.25",
            "density": 0.17112299465240635
          },
          {
            "start": "0.25",
            "density": 0.082887700534759287
          }
        ],
        "percentiles": {
          "p75": "0.11"
        }
      },
      "first_contentful_paint": {
        "histogram": [
          {
            "start": 0,
            "end": 1000,
            "density": 0.454180602006688
          },
          {
            "start": 1000,
            "end": 3000,
            "density": 0.52575250836120291
          },
          {
            "start": 3000,
            "density": 0.020066889632107024
          }
        ],
        "percentiles": {
          "p75": 1365
        }
      },
      "first_input_delay": {
        "histogram": [
          {
            "start": 0,
            "end": 100,
            "density": 0.812922614575508
          },
          {
            "start": 100,
            "end": 300,
            "density": 0.1750563486100678
          },
          {
            "start": 300,
            "density": 0.012021036814425257
          }
        ],
        "percentiles": {
          "p75": 38
        }
      },
      "largest_contentful_paint": {
        "histogram": [
          {
            "start": 0,
            "end": 2500,
            "density": 0.95027247956403227
          },
          {
            "start": 2500,
            "end": 4000,
            "density": 0.039509536784741124
          },
          {
            "start": 4000,
            "density": 0.010217983651226175
          }
        ],
        "percentiles": {
          "p75": 1583
        }
      }
    }
  },
  "urlNormalizationDetails": {
    "originalUrl": "https://github.com/marketplace?type=actions",
    "normalizedUrl": "https://github.com/marketplace"
  }
}
✅ 200: Origin-level data
curl -d origin='https://github.com' \
     -d formFactor=DESKTOP \
     'https://chromeuxreport.googleapis.com/v1/records:queryRecord?key=CRUX_API_KEY'
{
  "record": {
    "key": {
      "formFactor": "DESKTOP",
      "origin": "https://github.com"
    },
    "metrics": {
      "first_input_delay": {
        "histogram": [
          {
            "start": 0,
            "end": 100,
            "density": 0.99445638646905821
          },
          {
            "start": 100,
            "end": 300,
            "density": 0.004072858920692389
          },
          {
            "start": 300,
            "density": 0.0014707546102500305
          }
        ],
        "percentiles": {
          "p75": 19
        }
      },
      "largest_contentful_paint": {
        "histogram": [
          {
            "start": 0,
            "end": 2500,
            "density": 0.88479181369088589
          },
          {
            "start": 2500,
            "end": 4000,
            "density": 0.0809809456598438
          },
          {
            "start": 4000,
            "density": 0.034227240649258875
          }
        ],
        "percentiles": {
          "p75": 1775
        }
      },
      "cumulative_layout_shift": {
        "histogram": [
          {
            "start": "0.00",
            "end": "0.10",
            "density": 0.869868589370856
          },
          {
            "start": "0.10",
            "end": "0.25",
            "density": 0.076636818234678356
          },
          {
            "start": "0.25",
            "density": 0.053494592394464843
          }
        ],
        "percentiles": {
          "p75": "0.05"
        }
      },
      "first_contentful_paint": {
        "histogram": [
          {
            "start": 0,
            "end": 1000,
            "density": 0.46447119924457247
          },
          {
            "start": 1000,
            "end": 3000,
            "density": 0.48642587346553579
          },
          {
            "start": 3000,
            "density": 0.049102927289896459
          }
        ],
        "percentiles": {
          "p75": 1572
        }
      }
    }
  }
}
🛑 400 INVALID_ARGUMENT: API key not valid, please pass a valid API key
curl -d origin='https://github.com' \
     'https://chromeuxreport.googleapis.com/v1/records:queryRecord?key=INVALID_KEY'
{
  "error": {
    "code": 400,
    "message": "API key not valid. Please pass a valid API key.",
    "status": "INVALID_ARGUMENT",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.Help",
        "links": [
          {
            "description": "Google developers console",
            "url": "https://console.developers.google.com"
          }
        ]
      }
    ]
  }
}
🛑 400 INVALID_ARGUMENT: Invalid value at 'form_factor'/'ect'
curl -d url='https://github.com/' \
     -d formFactor=mobile  \
     'https://chromeuxreport.googleapis.com/v1/records:queryRecord?key=CRUX_API_KEY'
{
  "error": {
    "code": 400,
    "message": "Invalid value at 'form_factor' (type.googleapis.com/google.chrome.uxreport.v1.FormFactor), \"mobile\"",
    "status": "INVALID_ARGUMENT",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.BadRequest",
        "fieldViolations": [
          {
            "field": "form_factor",
            "description": "Invalid value at 'form_factor' (type.googleapis.com/google.chrome.uxreport.v1.FormFactor), \"mobile\""
          }
        ]
      }
    ]
  }
}
🛑 404 NOT_FOUND: chrome ux report data not found
curl -d url='https://github.com/search' \
     'https://chromeuxreport.googleapis.com/v1/records:queryRecord?key=CRUX_API_KEY'
{
  "error": {
    "code": 404,
    "message": "chrome ux report data not found",
    "status": "NOT_FOUND"
  }
}
🛑 429 RESOURCE_EXHAUSTED: Quota exceeded limit 'Queries per 100 seconds' of service
curl -d url='https://github.com/search' \
     'https://chromeuxreport.googleapis.com/v1/records:queryRecord?key=CRUX_API_KEY'
{
  "code": 429,
  "message": "Quota exceeded for quota group 'default' and limit 'Queries per 100 seconds' of service 'chromeuxreport.googleapis.com' for consumer 'project_number:00000000000000'.",
  "status": "RESOURCE_EXHAUSTED",
  "details": [
    {
      "@type": "type.googleapis.com/google.rpc.Help",
      "links": [
        {
          "description": "Google developer console API key",
          "url": "https://console.developers.google.com/project/00000000000000/apiui/credential"
        }
      ]
    }
  ]
}

Credits

Sponsored by Treo - Page speed monitoring made simple.