Expose HTTP Gateway when running in browser (eg. Service Worker)

[lidel]

• Version: v0.33.1
• Platform: Browser
• Subsystem: Workers / WebExtensions

Type:

Feature

Severity:

Medium

Description:

TL;DR

js-ipfs running in browser context does not provide gateway functionality (browser can't open raw TCP sockets). What if we expose it in JS as const response = await node.httpGateway(request) so we can reuse it in Web Workers and other browser contexts?

Background on HTTP Gateways

HTTP Gateway is exposed by go-ipfs and js-ipfs running in Node to bridge the gap between old web and IPFS. People without native IPFS client can access IPFS resources over HTTP Gateways. HTTP is an important step in migration path and general interoperability with the web: it provides a known abstraction for accessing IPFS, takes care of setting correct content-type, resolving long paths, hides sharding structures, and generates HTML with directory listings.

One of key challenges js-ipfs has when running in browser context is exposing HTTP Gateway. JS running on websites can't open TCP ports, and due to this HTTP Gateway is not available in such node.

We would like to reuse HTTP Gateway feature in browser contexts by exposing it as an utility function to improve developer experience when used in Service Worker.

Problem

Lack of local HTTP Gateway in browser context impacts multiple projects:

• native protocol handler prototype in ipfs-companion
• service-worker-gateway and js-ipfs in Service Worker in general
• recent progressive p2p app experiments and related "Fix 3" as means to use js-ipfs in Brave

They all live in web browser and need an easy way to open IPFS resource in HTTP-like fashion.
Unfortunately, without HTTP Gateway, opening an arbitrary resource requires manual IPFS tree traversal, support for sharded directories, and content-type detection, which all translates to multiple calls to low level IPFS APIs or awkwardly reusing parts of js-ipfs-http-response with some additional glue code. In general, is a brittle process, a lot of code gets duplicated. Not a productive experience.

Solutions

The obvious route is to improve js-ipfs-http-response and move duplicated code and boilerplate there, so it can be shared across mentioned and other projects. Most of this will happen anyway, however I started thinking if there is anything else we could do to improve developer experience in Worker contexts.

Here is an idea: expose HTTP Gateway as a programmatic interface

• js-ipfs detects it runs in browser context and instead of exposing Gateway on TCP socket it exposes gateway functionality as a JS function (eg. node.httpGateway(..)).
• The function accepts an IPFS Path or a request object and returns a response object with headers and payload, making it extremely easy to use in places like ServiceWorker without any additional heavy-duty code.

Why is this valuable?

Web developers would not need to know or worry about any dependency. If you have js-ipfs instance running on your page, Gateway function is just there, ready to be used, no need to write custom HTTP-like resolver.

Would love to hear some thoughts on feasibility of this.

[lidel]

Where to start

• Right now gateway code is tightly coupled with HTTP server (iirc js-ipfs internally uses hapi). As the first step, gateway.js needs to be evaluated on what will be a better approach:
  • (A) create a thin abstraction on top of checkCID, handler, afterHandler, or
  • (B) make a new, generic gateway interface and switch the existing TCP-based Gateway (mentioned methods) to use it internally

(B) is probably the better idea, it makes gateway framework-agnostic.

[achingbrain]

js-ipfs is being deprecated in favor of Helia. You can #4336 and read the migration guide.

This is implemented as https://github.com/ipfs-shipyard/helia-service-worker-gateway