Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] "server has same asset" error on videos #544

Open
helgehatt opened this issue Nov 27, 2024 · 5 comments
Open

[BUG] "server has same asset" error on videos #544

helgehatt opened this issue Nov 27, 2024 · 5 comments

Comments

@helgehatt
Copy link

I have several videos which are not uploaded. I suspect it is because of files with identical names. I have made a minimal example with these files:

-rw-rw-r-- 1 helge helge 1.6M Nov 26 18:05 IMG_7717.HEIC
-rw-rw-r-- 1 helge helge  809 Nov 26 18:05 IMG_7717.HEIC.json
-rw-rw-r-- 1 helge helge 611K Nov 26 18:40 IMG_7717.JPG
-rw-rw-r-- 1 helge helge  787 Nov 26 18:40 IMG_7717.JPG.json
-rw-rw-r-- 1 helge helge 752K Nov 26 18:12 IMG_7717.MOV
-rw-rw-r-- 1 helge helge  707 Nov 26 18:11 IMG_7717.MOV.json

Here are the logs from running immich-go upload -google-photos -dry-run *:

 INFO | Browsing google take out archive... | time="2024-11-27T08:14.20 CET"
 INFO | scanned image file | file=IMG_7717.HEIC time="2024-11-27T08:14.20 CET"
 INFO | scanned sidecar file | file=IMG_7717.HEIC.json type="asset metadata" title=IMG_7717.HEIC time="2024-11-27T08:14.20 CET"
 INFO | scanned image file | file=IMG_7717.JPG time="2024-11-27T08:14.20 CET"
 INFO | scanned sidecar file | file=IMG_7717.JPG.json type="asset metadata" title=IMG_7717.JPG time="2024-11-27T08:14.20 CET"
 INFO | scanned video file | file=IMG_7717.MOV time="2024-11-27T08:14.20 CET"
 INFO | scanned sidecar file | file=IMG_7717.MOV.json type="asset metadata" title=IMG_7717.MOV time="2024-11-27T08:14.20 CET"
 INFO | associated metadata file | file=IMG_7717.HEIC json=IMG_7717.HEIC.json size=1603420 matcher=normalMatch time="2024-11-27T08:14.20 CET"
 INFO | associated metadata file | file=IMG_7717.JPG json=IMG_7717.JPG.json size=625352 matcher=normalMatch time="2024-11-27T08:14.20 CET"
 INFO | associated metadata file | file=IMG_7717.MOV json=IMG_7717.MOV.json size=769376 matcher=normalMatch time="2024-11-27T08:14.20 CET"
 INFO | server has same asset | file=IMG_7717.HEIC reason="An asset with the same name:\"IMG_7717.HEIC\", date:\"2022-06-17 15:00:39\" and size:1.5 MB exists on the server. No need to upload." time="2024-11-27T08:15.02 CET"
 INFO | server has same asset | file=IMG_7717.MOV reason="An asset with the same name:\"IMG_7717.HEIC\", date:\"2022-06-17 15:00:39\" and size:1.5 MB exists on the server. No need to upload." time="2024-11-27T08:15.02 CET"
 INFO | server has same asset | file=IMG_7717.JPG reason="An asset with the same name:\"IMG_7717.JPG\", date:\"2022-01-01 11:29:45\" and size:610.7 KB exists on the server. No need to upload." time="2024-11-27T08:15.02 CET"

Notice the second last line, where the MOV file is identified as the HEIC file.

The file sizes are not identical, nor are the dates:

// IMG_7717.HEIC.json
{
  "title": "IMG_7717.HEIC",
  "description": "",
  "imageViews": "16",
  "creationTime": {
    "timestamp": "1655490219",
    "formatted": "Jun 17, 2022, 6:23:39 PM UTC"
  },
  "photoTakenTime": {
    "timestamp": "1655470839",
    "formatted": "Jun 17, 2022, 1:00:39 PM UTC"
  },
  ...
}

// IMG_7717.MOV.json
{
  "title": "IMG_7717.MOV",
  "description": "",
  "imageViews": "0",
  "creationTime": {
    "timestamp": "1732636306",
    "formatted": "Nov 26, 2024, 3:51:46 PM UTC"
  },
  "photoTakenTime": {
    "timestamp": "1648931303",
    "formatted": "Apr 2, 2022, 8:28:23 PM UTC"
  },
  ...
}
@helgehatt
Copy link
Author

Works fine if the other photos with same name are not uploaded simultaneously:

 INFO | Browsing google take out archive... | time="2024-11-27T08:34.52 CET"
 INFO | scanned video file | file=IMG_7717.MOV time="2024-11-27T08:34.53 CET"
 INFO | scanned sidecar file | file=IMG_7717.MOV.json type="asset metadata" title=IMG_7717.MOV time="2024-11-27T08:34.53 CET"
 INFO | associated metadata file | file=IMG_7717.MOV json=IMG_7717.MOV.json size=769376 matcher=normalMatch time="2024-11-27T08:34.53 CET"
 INFO | uploaded | file=IMG_7717.MOV "capture date"="2022-04-02 22:28:23 +0200 CEST" time="2024-11-27T08:35.35 CET"

Maybe related to live photos, matching video to image file?

@simulot
Copy link
Owner

simulot commented Nov 28, 2024

That's strange.
I have completely rewritten the code.
Maybe you can try the release candidate:
https://github.com/simulot/immich-go/releases/tag/v0.23.0-RC

@helgehatt
Copy link
Author

Seems to work with the release candidate. Small question though, the log includes three lines with Stacked file=.:. Would this perform any stacking? Or are all files a stack with at least one photo?

2024-11-29 09:31:50 INF Dry-run mode enabled. No changes will be made to the server.
2024-11-29 09:31:50 INF Can't set the takeout tag: no .zip file in the arguments
2024-11-29 09:31:50 INF scanned image file file=.:IMG_7717.HEIC
2024-11-29 09:31:50 INF scanned sidecar file file=.:IMG_7717.HEIC.json type=asset metadata title=IMG_7717.HEIC
2024-11-29 09:31:50 INF scanned image file file=.:IMG_7717.JPG
2024-11-29 09:31:50 INF scanned sidecar file file=.:IMG_7717.JPG.json type=asset metadata title=IMG_7717.JPG
2024-11-29 09:31:50 INF scanned video file file=.:IMG_7717.MOV
2024-11-29 09:31:50 INF scanned sidecar file file=.:IMG_7717.MOV.json type=asset metadata title=IMG_7717.MOV
2024-11-29 09:31:50 INF associated metadata file file=.:IMG_7717.HEIC json=IMG_7717.HEIC.json matcher=normalMatch
2024-11-29 09:31:50 INF associated metadata file file=.:IMG_7717.JPG json=IMG_7717.JPG.json matcher=normalMatch
2024-11-29 09:31:50 INF associated metadata file file=.:IMG_7717.MOV json=IMG_7717.MOV.json matcher=normalMatch
2024-11-29 09:32:37 INF server has same asset file=.:IMG_7717.JPG reason=An asset with the same name:"IMG_7717.JPG", date:"2022-01-01 11:29:45" and size:610.7 KB exists on the server. No need to upload.
2024-11-29 09:32:37 INF uploaded file=.:IMG_7717.MOV
2024-11-29 09:32:37 INF server has same asset file=.:IMG_7717.HEIC reason=An asset with the same name:"IMG_7717.HEIC", date:"2022-06-17 15:00:39" and size:1.5 MB exists on the server. No need to upload.
2024-11-29 09:32:37 INF Stacked file=.:IMG_7717.JPG
2024-11-29 09:32:37 INF Stacked file=.:IMG_7717.MOV
2024-11-29 09:32:37 INF Stacked file=.:IMG_7717.HEIC
2024-11-29 09:32:44 INF 
2024-11-29 09:32:44 INF Input analysis:
2024-11-29 09:32:44 INF ---------------
2024-11-29 09:32:44 INF scanned image file                      :       2
2024-11-29 09:32:44 INF scanned video file                      :       1
2024-11-29 09:32:44 INF scanned sidecar file                    :       3
2024-11-29 09:32:44 INF discarded file                          :       0
2024-11-29 09:32:44 INF unsupported file                        :       0
2024-11-29 09:32:44 INF file duplicated in the input            :       0
2024-11-29 09:32:44 INF associated metadata file                :       3
2024-11-29 09:32:44 INF missing associated metadata file        :       0
2024-11-29 09:32:44 INF 
2024-11-29 09:32:44 INF Uploading:
2024-11-29 09:32:44 INF ----------
2024-11-29 09:32:44 INF uploaded                                :       1
2024-11-29 09:32:44 INF upload error                            :       0
2024-11-29 09:32:44 INF file not selected                       :       0
2024-11-29 09:32:44 INF server's asset upgraded with the input  :       0
2024-11-29 09:32:44 INF server has same asset                   :       2
2024-11-29 09:32:44 INF server has a better asset               :       0
2024-11-29 09:32:44 INF 
2024-11-29 09:32:44 INF Dry-run mode enabled. No changes were made to the server.

@simulot
Copy link
Owner

simulot commented Nov 29, 2024 •
edited
Loading

Seems to work with the release candidate.

Good to know! Thanks for testing

Or are all files a stack with at least one photo?

Um... no
Coud you share he complete log file?

@helgehatt
Copy link
Author

helgehatt commented Nov 29, 2024 •
edited
Loading

Here's the complete log. Doesn't say much more though.

2024-11-29 09:31:50 INF immich-go version:dev,  commit:d180f8731bcb543a8ce72101a20c1d295ae181c8, date:2024-11-24T15:18:15Z
2024-11-29 09:31:50 INF Command: from-google-photos [flags] <takeout-*.zip> | <takeout-folder>
2024-11-29 09:31:50 INF Flags:
2024-11-29 09:31:50 INF  --api-key=*************************************
2024-11-29 09:31:50 INF  --api-trace=false
2024-11-29 09:31:50 INF  --ban-file='@eaDir/', '@__thumb/', 'SYNOFILE_THUMB_*.*', 'Lightroom Catalog/', 'thumbnails/', '.DS_Store/', '._*.*'
2024-11-29 09:31:50 INF  --capture-date-method=
2024-11-29 09:31:50 INF  --client-timeout=5m0s
2024-11-29 09:31:50 INF  --date-range=unset
2024-11-29 09:31:50 INF  --device-uuid=ds380
2024-11-29 09:31:50 INF  --dry-run=true
2024-11-29 09:31:50 INF  --exclude-extensions=
2024-11-29 09:31:50 INF  --exiftool-enabled=false
2024-11-29 09:31:50 INF  --exiftool-path=
2024-11-29 09:31:50 INF  --exiftool-timezone=Local
2024-11-29 09:31:50 INF  --from-album-name=
2024-11-29 09:31:50 INF  --help=false
2024-11-29 09:31:50 INF  --include-archived=true
2024-11-29 09:31:50 INF  --include-extensions=
2024-11-29 09:31:50 INF  --include-partner=true
2024-11-29 09:31:50 INF  --include-trashed=false
2024-11-29 09:31:50 INF  --include-unmatched=false
2024-11-29 09:31:50 INF  --include-untitled-albums=false
2024-11-29 09:31:50 INF  --log-file=/home/helge/.cache/immich-go/immich-go_2024-11-29_09-31-50.log
2024-11-29 09:31:50 INF  --log-level=INFO
2024-11-29 09:31:50 INF  --log-type=text
2024-11-29 09:31:50 INF  --manage-burst=
2024-11-29 09:31:50 INF  --manage-epson-fastfoto=false
2024-11-29 09:31:50 INF  --manage-heic-jpeg=
2024-11-29 09:31:50 INF  --manage-raw-jpeg=
2024-11-29 09:31:50 INF  --no-ui=false
2024-11-29 09:31:50 INF  --partner-shared-album=
2024-11-29 09:31:50 INF  --server=******************
2024-11-29 09:31:50 INF  --session-tag=false
2024-11-29 09:31:50 INF  --skip-verify-ssl=false
2024-11-29 09:31:50 INF  --sync-albums=true
2024-11-29 09:31:50 INF  --tag=[]
2024-11-29 09:31:50 INF  --takeout-tag=true
2024-11-29 09:31:50 INF  --time-zone=
2024-11-29 09:31:50 INF Arguments:
2024-11-29 09:31:50 INF   "IMG_7717.HEIC"
2024-11-29 09:31:50 INF   "IMG_7717.HEIC.json"
2024-11-29 09:31:50 INF   "IMG_7717.JPG"
2024-11-29 09:31:50 INF   "IMG_7717.JPG.json"
2024-11-29 09:31:50 INF   "IMG_7717.MOV"
2024-11-29 09:31:50 INF   "IMG_7717.MOV.json"
2024-11-29 09:31:50 INF Connection to the server *******************
2024-11-29 09:31:50 INF Server status: OK
2024-11-29 09:31:50 INF Connected, user: ********************
2024-11-29 09:31:50 INF Dry-run mode enabled. No changes will be made to the server.
2024-11-29 09:31:50 INF Can't set the takeout tag: no .zip file in the arguments
2024-11-29 09:31:50 INF scanned image file file=.:IMG_7717.HEIC
2024-11-29 09:31:50 INF scanned sidecar file file=.:IMG_7717.HEIC.json type=asset metadata title=IMG_7717.HEIC
2024-11-29 09:31:50 INF scanned image file file=.:IMG_7717.JPG
2024-11-29 09:31:50 INF scanned sidecar file file=.:IMG_7717.JPG.json type=asset metadata title=IMG_7717.JPG
2024-11-29 09:31:50 INF scanned video file file=.:IMG_7717.MOV
2024-11-29 09:31:50 INF scanned sidecar file file=.:IMG_7717.MOV.json type=asset metadata title=IMG_7717.MOV
2024-11-29 09:31:50 INF associated metadata file file=.:IMG_7717.HEIC json=IMG_7717.HEIC.json matcher=normalMatch
2024-11-29 09:31:50 INF associated metadata file file=.:IMG_7717.JPG json=IMG_7717.JPG.json matcher=normalMatch
2024-11-29 09:31:50 INF associated metadata file file=.:IMG_7717.MOV json=IMG_7717.MOV.json matcher=normalMatch
2024-11-29 09:32:37 INF server has same asset file=.:IMG_7717.JPG reason=An asset with the same name:"IMG_7717.JPG", date:"2022-01-01 11:29:45" and size:610.7 KB exists on the server. No need to upload.
2024-11-29 09:32:37 INF uploaded file=.:IMG_7717.MOV
2024-11-29 09:32:37 INF server has same asset file=.:IMG_7717.HEIC reason=An asset with the same name:"IMG_7717.HEIC", date:"2022-06-17 15:00:39" and size:1.5 MB exists on the server. No need to upload.
2024-11-29 09:32:37 INF Stacked file=.:IMG_7717.JPG
2024-11-29 09:32:37 INF Stacked file=.:IMG_7717.MOV
2024-11-29 09:32:37 INF Stacked file=.:IMG_7717.HEIC
2024-11-29 09:32:44 INF 
2024-11-29 09:32:44 INF Input analysis:
2024-11-29 09:32:44 INF ---------------
2024-11-29 09:32:44 INF scanned image file                      :       2
2024-11-29 09:32:44 INF scanned video file                      :       1
2024-11-29 09:32:44 INF scanned sidecar file                    :       3
2024-11-29 09:32:44 INF discarded file                          :       0
2024-11-29 09:32:44 INF unsupported file                        :       0
2024-11-29 09:32:44 INF file duplicated in the input            :       0
2024-11-29 09:32:44 INF associated metadata file                :       3
2024-11-29 09:32:44 INF missing associated metadata file        :       0
2024-11-29 09:32:44 INF 
2024-11-29 09:32:44 INF Uploading:
2024-11-29 09:32:44 INF ----------
2024-11-29 09:32:44 INF uploaded                                :       1
2024-11-29 09:32:44 INF upload error                            :       0
2024-11-29 09:32:44 INF file not selected                       :       0
2024-11-29 09:32:44 INF server's asset upgraded with the input  :       0
2024-11-29 09:32:44 INF server has same asset                   :       2
2024-11-29 09:32:44 INF server has a better asset               :       0
2024-11-29 09:32:44 INF 
2024-11-29 09:32:44 INF Dry-run mode enabled. No changes were made to the server.
2024-11-29 09:32:44 INF Check the log file: /home/helge/.cache/immich-go/immich-go_2024-11-29_09-31-50.log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@simulot @helgehatt