Skip to content

Commit

Permalink
docs: Change singularity documentation
Browse files Browse the repository at this point in the history
Signed-off-by: zhaoxinxin <[email protected]>
  • Loading branch information
Liam-Zhao committed Apr 8, 2024
1 parent 26463b5 commit 3a11da3
Show file tree
Hide file tree
Showing 33 changed files with 66 additions and 60 deletions.
1 change: 1 addition & 0 deletions docs/getting-started/installation/binaries.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: binaries
title: Binaries
slug: /getting-started/installation/binaries/
---

This guide shows how to install the Dragonfly. Dragonfly can be installed either from source, or from pre-built binary releases.
Expand Down
1 change: 1 addition & 0 deletions docs/getting-started/installation/helm-charts.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: helm-charts
title: Helm Charts
slug: /getting-started/installation/helm-charts/
---

Documentation for deploying Dragonfly on kubernetes using helm.
Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started/quick-start.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
id: quick-start
title: Quick Start
slug: /getting-started/quick-start/
slug: /getting-started/quick-start
---

Table of contents:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
id: containerd
title: containerd
slug: /setup/runtime/containerd
slug: /operations/integrations/container-runtime/containerd/
---

Documentation for setting Dragonfly's container runtime to containerd.
Expand Down
2 changes: 1 addition & 1 deletion docs/operations/integrations/container-runtime/cri-o.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
id: cri-o
title: CRI-O
slug: /setup/runtime/cri-o
slug: /operations/integrations/container-runtime/cri-o/
---

Use dfget daemon as registry mirror for CRI-O
Expand Down
2 changes: 1 addition & 1 deletion docs/operations/integrations/container-runtime/docker.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
id: docker
title: Docker
slug: /setup/runtime/docker
slug: /operations/integrations/container-runtime/docker/
---

Use dfget daemon as HTTP proxy for docker daemon
Expand Down
1 change: 1 addition & 0 deletions docs/operations/integrations/container-runtime/nydus.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: nydus
title: Nydus
slug: /operations/integrations/container-runtime/nydus/
---

This document will help you experience how to use dragonfly with nydus.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
---
id: singularity
title: Singularity/Apptainer
slug: /setup/runtime/singularity
---

This documentation will help you to integrate Dragonfly into
Singularity/Apptainer container runtime.

Table of contents:

- [Oras Resource Client Mode](./singularity/oras_resource_client.md)
- [Oras Resource Client Mode](./singularity/oras-resource-client.md)
- [Proxy Mode](./singularity/proxy.md)
Original file line number Diff line number Diff line change
@@ -1,22 +1,18 @@
---
id: singularity-oras-resource-client
title: Oras Resource Client Mode
slug: /setup/runtime/singularity/oras-resource-client
title: ORAS Resource Client
slug: /operations/integrations/container-runtime/singularity/oras-resource-client/
---

## Oras Resource Client {#oras-resource-client}

We can use oras resource client to pull image using Dragonfly, the key is the scheme used for the resource client.
Documentation for setting Dragonfly's container runtime to ORAS Resource Client.

This method of image pull through Dragonfly is more efficient when compared to proxy method as
it avoids TLS termination, reduces CPU time and download time as it creates hardlink(insead of copy)
for subsequent file download after downloading the image from source for first time.

## Quick Start {#quick-start}

### Step 1: Configure dfget daemon {#step-1-configure-dfget-daemon}
## Configure dfget daemon {#step-1-configure-dfget-daemon}

To use oras resource client to pull image ensure below configuraion in `/etc/dragonfly/dfget.yaml:`.
To use ORAS Resource Client to pull image ensure below configuraion in `/etc/dragonfly/dfget.yaml`:

```yaml
# Peer task storage option.
Expand All @@ -33,7 +29,7 @@ storage:
# Set to ture for reusing underlying storage for same task id.
multiplex: true

# The singularity oras resources, most of it is same with https scheme
# The singularity ORAS resources, most of it is same with https scheme.
oras:
proxy:
dialTimeout: 30s
Expand All @@ -46,17 +42,15 @@ oras:
insecureSkipVerify: true
```
### Step 2: Pull images through oras resource client {#step-2-pull-images-through-oras-resource-client}
Through the above steps, we can start to validate if Dragonfly works as expected.
## ORAS Resource Client downloads images through Dragonfly {#step-2-pull-images-through-oras-resource-client}
And you can pull the image through oras resource client as below:
And you can pull the image through ORAS Resource Client as below:
```shell
dfget -u "oras://hostname/path/image:tag" -O /path/to/output
```

### Step 3: Validate Dragonfly {#step-3-validate-dragonfly}
### Verify {#verify}

You can execute the following command to
check if the image is distributed via Dragonfly.
Expand All @@ -65,7 +59,7 @@ check if the image is distributed via Dragonfly.
grep "peer task done" /var/log/dragonfly/daemon/core.log
```

If the output of command above has content like
The expected output is as follows:

```shell
{
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
---
id: singularity-proxy
title: Proxy Mode
slug: /setup/runtime/singularity/proxy
title: HTTP Proxy
slug: /operations/integrations/container-runtime/singularity/proxy/
---

Uses dfget daemon as proxy for Singularity/Apptainer.
Documentation for setting up Dragonfly's container runtime as an http proxy for Singularity/Apptainer.

## Step 1: Generate CA certificate {#step-1-generate-ca-certificate}
## Generate CA certificate {#step-1-generate-ca-certificate}

Generate an RSA private key.

Expand Down Expand Up @@ -57,7 +57,7 @@ openssl x509 -req -days 36500 -extfile openssl.conf \
-extensions v3_ca -in ca.csr -signkey ca.key -out ca.crt
```

## Step 2: Configure dfget daemon {#step-2-configure-dfget-daemon}
## Configure dfget daemon {#step-2-configure-dfget-daemon}

To use dfget daemon as proxy, first you need to append a proxy rule in
`/etc/dragonfly/dfget.yaml`, This will proxy
Expand Down Expand Up @@ -90,17 +90,15 @@ hijackHTTPS:
- regx: <your.private.registry>
```
## Step 3: Pull images with proxy {#step-4-pull-images-with-proxy}
Through the above steps, we can start to validate if Dragonfly works as expected.
## Pull images with proxy {#step-4-pull-images-with-proxy}
And you can pull the image through proxy as below:
```bash
no_proxy='' NO_PROXY='' HTTPS_PROXY=127.0.0.1:65001 singularity pull oras://hostname/path/image:tag
```

## Step 4: Validate Dragonfly {#step-4-validate-dragonfly}
### Verify {#verify}

You can execute the following command to
check if the image is distributed via Dragonfly.
Expand All @@ -109,7 +107,7 @@ check if the image is distributed via Dragonfly.
grep "peer task done" /var/log/dragonfly/daemon/core.log
```

If the output of command above has content like
The expected output is as follows:

```shell
{
Expand Down
1 change: 1 addition & 0 deletions docs/operations/integrations/container-runtime/stargz.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: stargz
title: eStargz
slug: /operations/integrations/container-runtime/stargz/
---

This document will help you experience how to use dragonfly with eStargz.
Expand Down
1 change: 1 addition & 0 deletions docs/operations/integrations/git-lfs.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: git-lfs
title: Git LFS
slug: /operations/integrations/git-lfs/
---

## What is Git LFS?
Expand Down
1 change: 1 addition & 0 deletions docs/operations/integrations/harbor.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: harbor
title: Harbor
slug: /operations/integrations/harbor/
---

This document will help you experience how to use dragonfly with harbor.
Expand Down
1 change: 1 addition & 0 deletions docs/operations/integrations/hugging-face.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: hugging-face
title: Hugging Face
slug: /operations/integrations/hugging-face/
---

This document will help you experience how to use dragonfly with hugging face.
Expand Down
1 change: 1 addition & 0 deletions docs/operations/integrations/torchserve.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: torchserve
title: TorchServe
slug: /operations/integrations/torchserve/
---

This document will help you experience how to use dragonfly with [TorchServe](https://github.com/pytorch/serve).
Expand Down
1 change: 1 addition & 0 deletions docs/operations/integrations/triton-server.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: triton-server
title: Triton Server
slug: /operations/integrations/triton-server/
---

This document will help you experience how to use dragonfly with [TritonServe](https://github.com/pytorch/serve).
Expand Down
2 changes: 1 addition & 1 deletion docs/operations/integrations/upgrade.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
id: upgrade
title: Upgrade
slug: /setup/upgrade
slug: /operations/integrations/upgrade/
---

## Upgrade the cluster deployed by Helm {#upgrade-the-cluster-deployed-by-helm}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: binaries
title: 使用二进制文件安装
slug: /getting-started/installation/binaries/
---

文档的目标是帮助您快速开始使用源码或构建的二进制版本部署 Dragonfly。
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: helm-charts
title: Helm Charts
slug: /getting-started/installation/helm-charts/
---

文档的目标是帮助您快速开始使用 Helm Charts 部署 Dragonfly。
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
id: containerd
title: containerd
slug: /setup/runtime/containerd
slug: /operations/integrations/container-runtime/containerd/
---

文档的目标是帮助您将 Dragonfly 的容器运行时设置为 containerd。
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
id: cri-o
title: CRI-O
slug: /setup/runtime/cri-o
slug: /operations/integrations/container-runtime/cri/
---

使用 dfdaemon 支持 CRI-O
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
id: docker
title: Docker
slug: /setup/runtime/docker
slug: /operations/integrations/container-runtime/docker/
---

使用 dfget daemon 作为 docker daemon 的 http 代理
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: nydus
title: Nydus
slug: /operations/integrations/container-runtime/nydus/
---

本文档将帮助您将 Dragonfly 与 Nydus 一起使用。
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
---
id: singularity
title: Singularity/Apptainer
slug: /setup/runtime/singularity
---

该文档将帮助你集成 Dragonfly 到 Singularity/Apptainer。

集成方式:

- [Oras 资源客户端模式](./singularity/oras_resource_client.md)
- [Oras 资源客户端模式](./singularity/oras-resource-client.md)
- [HTTP Proxy 模式](./singularity/proxy.md)
Original file line number Diff line number Diff line change
@@ -1,26 +1,22 @@
---
id: singularity-oras-resource-client
title: Oras 资源客户端模式
slug: /setup/runtime/singularity/oras-resource-client
title: ORAS 资源客户端
slug: /operations/integrations/container-runtime/singularity/oras-resource-client/
---

## Oras 资源客户端

使用 oras 的资源客户端使用 Dragonfly 拉取镜像。
文档的目标是帮助您将 Dragonfly 的容器运行时设置为 ORAS 的资源客户端。

通过 Dragonfly 资源客户端拉取镜像的这种方法比代理方法相比更加高效,因为它避免了 TLS,减少了 CPU 资源使用量和下载时间,
因为它在首次从源下载镜像后为后续文件下载创建了硬链接,而不是复制一份完全一样的资源。

## 快速开始

### 步骤 1: 配置 dfdaemon
## 配置 Dfdaemon

下面为镜像仓库的 dfdaemon 配置,在路径 `/etc/dragonfly/dfget.yaml`:
下面为镜像仓库的 Dfdaemon 配置,在路径 `/etc/dragonfly/dfget.yaml`:

```yaml
# Peer task storage option.
storage:
# Task data expire time,
# Task data expire time.
# when there is no access to a task data, this task will be gc.
taskExpireTime: 6h
strategy: io.d7y.storage.v2.advance.
Expand All @@ -32,7 +28,7 @@ storage:
# Set to ture for reusing underlying storage for same task id.
multiplex: true

# The singularity oras resources, most of it is same with https scheme
# The singularity oras resources, most of it is same with https scheme.
oras:
proxy:
dialTimeout: 30s
Expand All @@ -45,15 +41,15 @@ oras:
insecureSkipVerify: true
```
### 步骤 2: 使用 oras 资源客户端下载镜像
## ORAS 资源客户端通过 Dragonfly 下载镜像
使用以下命令拉取镜像:
```shell
dfget -u "oras://hostname/path/image:tag" -O /path/to/output
```

### 步骤 3: 验证 Dragonfly 拉取成功
### 验证镜像下载成功

可以查看日志,判断镜像正常拉取。

Expand Down
Loading

0 comments on commit 3a11da3

Please sign in to comment.