Skip to content

Commit

Permalink
update nvidia-ai-endpoints notebook vlm examples (add asset id)
Browse files Browse the repository at this point in the history
  • Loading branch information
mattf committed Sep 9, 2024
1 parent 3e19339 commit 4da68fb
Showing 1 changed file with 74 additions and 0 deletions.
74 changes: 74 additions & 0 deletions libs/ai-endpoints/docs/chat/nvidia_ai_endpoints.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,80 @@
")"
]
},
{
"cell_type": "markdown",
"id": "25e8db7c",
"metadata": {},
"source": [
"#### Passing an image as an NVCF asset\n",
"\n",
"If your image is sufficiently large or you will pass it multiple times in a chat conversation, you may upload it once and reference it in your chat conversation.\n",
"\n",
"See https://docs.nvidia.com/cloud-functions/user-guide/latest/cloud-function/assets.html for details about how upload the image."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "091f7fce",
"metadata": {},
"outputs": [],
"source": [
"import requests\n",
"\n",
"content_type = \"image/jpg\"\n",
"description = \"example-image-from-lc-nv-ai-e-notebook\"\n",
"\n",
"create_response = requests.post(\n",
" \"https://api.nvcf.nvidia.com/v2/nvcf/assets\",\n",
" headers={\n",
" \"Authorization\": f\"Bearer {os.environ['NVIDIA_API_KEY']}\",\n",
" \"accept\": \"application/json\",\n",
" \"Content-Type\": \"application/json\",\n",
" },\n",
" json={\n",
" \"contentType\": content_type,\n",
" \"description\": description\n",
" }\n",
")\n",
"create_response.raise_for_status()\n",
"\n",
"upload_response = requests.put(\n",
" create_response.json()[\"uploadUrl\"],\n",
" headers={\n",
" \"Content-Type\": content_type,\n",
" \"x-amz-meta-nvcf-asset-description\": description,\n",
" },\n",
" data=image_content,\n",
")\n",
"upload_response.raise_for_status()\n",
"\n",
"asset_id = create_response.json()[\"assetId\"]\n",
"asset_id"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5c24be59",
"metadata": {},
"outputs": [],
"source": [
"llm.invoke(\n",
" [\n",
" HumanMessage(\n",
" content=[\n",
" {\"type\": \"text\", \"text\": \"Describe this image\"},\n",
" {\n",
" \"type\": \"image_url\",\n",
" \"image_url\": {\"url\": f\"data:{content_type};asset_id,{asset_id}\"},\n",
" },\n",
" ]\n",
" )\n",
" ]\n",
")"
]
},
{
"cell_type": "markdown",
"id": "0573dd1f-9a17-4c99-ab2a-8d930b89d283",
Expand Down

0 comments on commit 4da68fb

Please sign in to comment.