URLからアップロードができないファイルがある #14216

fruitriin · 2024-07-15T08:58:48Z

💡 Summary

ドライブのURLの制約がvarchar(512)だが、現実にはそれより長いことがあるらしい

🥰 Expected Behavior

そんなにカジュアルにエラーになってほしくない

🤬 Actual Behavior

 misskey@misskeydb ERROR:  value too long for type character varying(512)
misskey@misskeydb statement: with "cte"("id",
"userId",
"userHost",
"md5",
"name",
"type",
"size",
"com
ment",
"blurhash",
"properties",
"storedInternal",
"url",
"thumbnailUrl",
"webpublicUrl",
"webpublicType",
"accessKey",
"thumbnailAccessKey", 
"webpublicAccessKey",
"uri",
"src",
"folderId",
"isSensitive",
"maybeSensitive",
"maybePorn",
"isLink",
"requestHeaders",
"requestIp") as (INS
ERT
into
	"drive_file" as "t"("id",
	"userId",
	"userHost",
	"md5",
	"name",
	"type",
	"size",
	"comment",
	"blurhash",
	"properties",
	"storedInternal",
	"url",
	"thumbnailUrl",
	"webpublicUrl",
	"webpublicType",
	"accessKey",
	"thumbnailAccessKey",
	"webpublicAccessKey",
	"uri",
	"src",
	"folderId",
	"i
sSensitive",
	"maybeSensitive",
	"maybePorn",
	"isLink",
	"requestHeaders",
	"requestIp")
values ($1,
$2,
$3,
$4,
$5,
$6,
$7,
$8,
$9,
$10,
$11,
$12
,
default,
default,
default,
$13,
$14,
$15,
$16,
$17,
$18,
$19,
$20,
$21,
$22,
$23,
$24) returning "id",
"userId",
"userHost",
"md5",
"name", 
"type",
"size",
"comment",
"blurhash",
"properties",
"storedInternal",
"url",
"thumbnailUrl",
"webpublicUrl",
"webpublicType",
"accessKey",
"t
humbnailAccessKey",
"webpublicAccessKey",
"uri",
"src",
"folderId",
"isSensitive",
"maybeSensitive",
"maybePorn",
"isLink",
"requestHeaders", 
"requestIp")
select
	"MiDriveFile"."id" as "MiDriveFile_id",
	"MiDriveFile"."userId" as "MiDriveFile_userId",
	"MiDriveFile"."userHost" as "MiDri
veFile_userHost",
	"MiDriveFile"."md5" as "MiDriveFile_md5",
	"MiDriveFile"."name" as "MiDriveFile_name",
	"MiDriveFile"."type" as "MiDriveFile_t
ype",
	"MiDriveFile"."size" as "MiDriveFile_size",
	"MiDriveFile"."comment" as "MiDriveFile_comment",
	"MiDriveFile"."blurhash" as "MiDriveFile_b
lurhash",
	"MiDriveFile"."properties" as "MiDriveFile_properties",
	"MiDriveFile"."storedInternal" as "MiDriveFile_storedInternal",
	"MiDriveFile
"."url" as "MiDriveFile_url",
	"MiDriveFile"."thumbnailUrl" as "MiDriveFile_thumbnailUrl",
	"MiDriveFile"."webpublicUrl" as "MiDriveFile_webpubl
icUrl",
	"MiDriveFile"."webpublicType" as "MiDriveFile_webpublicType",
	"MiDriveFile"."accessKey" as "MiDriveFile_accessKey",
	"MiDriveFile"."thu
mbnailAccessKey" as "MiDriveFile_thumbnailAccessKey",
	"MiDriveFile"."webpublicAccessKey" as "MiDriveFile_webpublicAccessKey",
	"MiDriveFile"."u
ri" as "MiDriveFile_uri",
	"MiDriveFile"."src" as "MiDriveFile_src",
	"MiDriveFile"."folderId" as "MiDriveFile_folderId",
	"MiDriveFile"."isSensi
tive" as "MiDriveFile_isSensitive",
	"MiDriveFile"."maybeSensitive" as "MiDriveFile_maybeSensitive",
	"MiDriveFile"."maybePorn" as "MiDriveFile_
maybePorn",
	"MiDriveFile"."isLink" as "MiDriveFile_isLink",
	"MiDriveFile"."requestHeaders" as "MiDriveFile_requestHeaders",
	"MiDriveFile"."req
uestIp" as "MiDriveFile_requestIp"
from
	"cte" "MiDriveFile"

📝 Steps to Reproduce

Jul 15 16:50:32 MisskeySystems misskey[679689]:     query: 'WITH "cte"("id", "userId", "userHost", "md5", "name", "type", "size", "comment", "blurhash", "properties", "storedInternal", "url", "thumbnailUrl", "webpublicUrl", "webpublicType", "accessKey", "thumbnailAccessKey", "webpublicAccessKey", "uri", "src", "folderId", "isSensitive", "maybeSensitive", "maybePorn", "isLink", "requestHeaders", "requestIp") AS (INSERT INTO "drive_file" AS "t"("id", "userId", "userHost", "md5", "name", "type", "size", "comment", "blurhash", "properties", "storedInternal", "url", "thumbnailUrl", "webpublicUrl", "webpublicType", "accessKey", "thumbnailAccessKey", "webpublicAccessKey", "uri", "src", "folderId", "isSensitive", "maybeSensitive", "maybePorn", "isLink", "requestHeaders", "requestIp") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, DEFAULT, DEFAULT, DEFAULT, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24) RETURNING "id", "userId", "userHost", "md5", "name", "type", "size", "comment", "blurhash", "properties", "storedInternal", "url", "thumbnailUrl", "webpublicUrl", "webpublicType", "accessKey", "thumbnailAccessKey", "webpublicAccessKey", "uri", "src", "folderId", "isSensitive", "maybeSensitive", "maybePorn", "isLink", "requestHeaders", "requestIp") SELECT "MiDriveFile"."id" AS "MiDriveFile_id", "MiDriveFile"."userId" AS "MiDriveFile_userId", "MiDriveFile"."userHost" AS "MiDriveFile_userHost", "MiDriveFile"."md5" AS "MiDriveFile_md5", "MiDriveFile"."name" AS "MiDriveFile_name", "MiDriveFile"."type" AS "MiDriveFile_type", "MiDriveFile"."size" AS "MiDriveFile_size", "MiDriveFile"."comment" AS "MiDriveFile_comment", "MiDriveFile"."blurhash" AS "MiDriveFile_blurhash", "MiDriveFile"."properties" AS "MiDriveFile_properties", "MiDriveFile"."storedInternal" AS "MiDriveFile_storedInternal", "MiDriveFile"."url" AS "MiDriveFile_url", "MiDriveFile"."thumbnailUrl" AS "MiDriveFile_thumbnailUrl", "MiDriveFile"."webpublicUrl" AS "MiDriveFile_webpublicUrl", "MiDriveFile"."webpublicType" AS "MiDriveFile_webpublicType", "MiDriveFile"."accessKey" AS "MiDriveFile_accessKey", "MiDriveFile"."thumbnailAccessKey" AS "MiDriveFile_thumbnailAccessKey", "MiDriveFile"."webpublicAccessKey" AS "MiDriveFile_webpublicAccessKey", "MiDriveFile"."uri" AS "MiDriveFile_uri", "MiDriveFile"."src" AS "MiDriveFile_src", "MiDriveFile"."folderId" AS "MiDriveFile_folderId", "MiDriveFile"."isSensitive" AS "MiDriveFile_isSensitive", "MiDriveFile"."maybeSensitive" AS "MiDriveFile_maybeSensitive", "MiDriveFile"."maybePorn" AS "MiDriveFile_maybePorn", "MiDriveFile"."isLink" AS "MiDriveFile_isLink", "MiDriveFile"."requestHeaders" AS "MiDriveFile_requestHeaders", "MiDriveFile"."requestIp" AS "MiDriveFile_requestIp" FROM "cte" "MiDriveFile"',
Jul 15 16:50:32 MisskeySystems misskey[679689]:     driverError: error: value too long for type character varying(512)
Jul 15 16:50:32 MisskeySystems misskey[679689]:         at async PostgresQueryRunner.query (/root/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:184:25)
Jul 15 16:50:32 MisskeySystems misskey[679689]:         at async DriveService.addFile (file:///root/misskey/packages/backend/built/core/DriveService.js:501:24)
Jul 15 16:50:32 MisskeySystems misskey[679689]:         at async DriveService.uploadFromUrl (file:///root/misskey/packages/backend/built/core/DriveService.js:701:31)
Jul 15 16:50:32 MisskeySystems misskey[679689]:     at PostgresQueryRunner.query (/root/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
Jul 15 16:50:32 MisskeySystems misskey[679689]:     at async DriveService.addFile (file:///root/misskey/packages/backend/built/core/DriveService.js:501:24)
Jul 15 16:50:32 MisskeySystems misskey[679689]:     at async DriveService.uploadFromUrl (file:///root/misskey/packages/backend/built/core/DriveService.js:701:31)
Jul 15 16:50:32 MisskeySystems misskey[679689]:         '    at PostgresQueryRunner.query (/root/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)\n' +
Jul 15 16:50:32 MisskeySystems misskey[679689]:         '    at async DriveService.addFile (file:///root/misskey/packages/backend/built/core/DriveService.js:501:24)\n' +
Jul 15 16:50:32 MisskeySystems misskey[679689]:         '    at async DriveService.uploadFromUrl (file:///root/misskey/packages/backend/built/core/DriveService.js:701:31)\n' +
Jul 15 16:50:32 MisskeySystems misskey[679689]:       '    at PostgresQueryRunner.query (/root/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)\n' +
Jul 15 16:50:32 MisskeySystems misskey[679689]:       '    at async DriveService.addFile (file:///root/misskey/packages/backend/built/core/DriveService.js:501:24)\n' +
Jul 15 16:50:32 MisskeySystems misskey[679689]:       '    at async DriveService.uploadFromUrl (file:///root/misskey/packages/backend/built/core/DriveService.js:701:31)\n' +

💻 Frontend Environment

* Model and OS of the device(s):
* Browser:
* Server URL:
* Misskey:

🛰 Backend Environment (for server admin)

* Installation Method or Hosting Service:
* Misskey: 2024.5.0 or later
* Node:
* PostgreSQL:
* Redis:
* OS and Architecture:

Do you want to address this bug yourself?

Yes, I will patch the bug myself and send a pull request

The text was updated successfully, but these errors were encountered:

KisaragiEffective · 2024-07-15T09:00:14Z

個人的に: 512文字よりも長いことはあまりない気がします……？

fruitriin · 2024-07-15T09:56:07Z

アプリケーション側で不正な値として弾くのは正常な動作だけど、DBがエラーを投げるのはハンドリング不足だと思う

KisaragiEffective · 2024-07-15T09:57:11Z

たしかに、それなら一理ある気がします

Sayamame-beans · 2024-07-22T21:07:40Z

FYI: #14292 で512文字より長いファイルURLを観測 ("URLからアップロード"にこれを入れることは無いとは思いますが)

tai-cha · 2024-07-30T06:32:16Z

#14323 がマージされ上限が上がるので緩和はされるかも（もちろんハンドリングとかはまだしていないのでCloseするようにはしていない）

fruitriin added the ⚠️bug? This might be a bug label Jul 15, 2024

KisaragiEffective added this to [BUG TRACKER] Drive Jul 15, 2024

tai-cha mentioned this issue Jul 23, 2024

Threadsのリソースurlが長すぎて入らない Secineralyr/misskey.dream#70

Closed

tai-cha mentioned this issue Jul 30, 2024

Fix(backend): ドライブのファイルのurl, uri, src の上限引き上げ #14323

Merged

5 tasks

samunohito added the [Feat] Drive Drive related issue label Nov 27, 2024

samunohito added this to [実験中] 管理用 Nov 27, 2024

github-project-automation bot moved this to Todo in [実験中] 管理用 Nov 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

URLからアップロードができないファイルがある #14216

URLからアップロードができないファイルがある #14216

fruitriin commented Jul 15, 2024

KisaragiEffective commented Jul 15, 2024

fruitriin commented Jul 15, 2024

KisaragiEffective commented Jul 15, 2024

Sayamame-beans commented Jul 22, 2024 •

edited

Loading

tai-cha commented Jul 30, 2024

URLからアップロードができないファイルがある #14216

URLからアップロードができないファイルがある #14216

Comments

fruitriin commented Jul 15, 2024

💡 Summary

🥰 Expected Behavior

🤬 Actual Behavior

📝 Steps to Reproduce

💻 Frontend Environment

🛰 Backend Environment (for server admin)

Do you want to address this bug yourself?

KisaragiEffective commented Jul 15, 2024

fruitriin commented Jul 15, 2024

KisaragiEffective commented Jul 15, 2024

Sayamame-beans commented Jul 22, 2024 • edited Loading

tai-cha commented Jul 30, 2024

Sayamame-beans commented Jul 22, 2024 •

edited

Loading