Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

There was a problem sending your highlights: received a non-200 status code from Readwise: code 400 #82

Open
justinryanc opened this issue Apr 25, 2023 · 22 comments

Comments

@justinryanc
Copy link

{"level":"info","msg":"Logger should be initialised now","time":"2023-04-25T09:51:11-04:00"}
{"level":"info","msg":"Backend is about to start up","time":"2023-04-25T09:51:11-04:00"}
{"level":"info","msg":"Logger should be initialised now","time":"2023-04-25T09:51:13-04:00"}
{"level":"info","msg":"Backend is about to start up","time":"2023-04-25T09:51:13-04:00"}
{"kobos_found":1,"level":"info","msg":"Detected one or more Kobos","time":"2023-04-25T09:51:13-04:00"}
{"device_id":"00000000-0000-0000-0000-000000000383","level":"info","msg":"Found an attached device","time":"2023-04-25T09:51:13-04:00"}
{"kobos_found":1,"level":"info","msg":"Detected one or more Kobos","time":"2023-04-25T09:51:13-04:00"}
{"device_id":"00000000-0000-0000-0000-000000000383","level":"info","msg":"Found an attached device","time":"2023-04-25T09:51:13-04:00"}
{"batch_count":1,"highlight_count":0,"level":"info","msg":"Successfully parsed highlights","time":"2023-04-25T09:51:15-04:00"}
{"level":"error","msg":"Received a non-200 response from Readwise","response":"{"non_field_errors":["No data provided"]}","status_code":400,"time":"2023-04-25T09:51:15-04:00"}
{"level":"info","msg":"Successfully validated token against the Readwise API","time":"2023-04-25T09:53:23-04:00"}
{"batch_count":1,"highlight_count":0,"level":"info","msg":"Successfully parsed highlights","time":"2023-04-25T09:56:02-04:00"}
{"level":"error","msg":"Received a non-200 response from Readwise","response":"{"non_field_errors":["No data provided"]}","status_code":400,"time":"2023-04-25T09:56:02-04:00"}

@marcus-crane
Copy link
Owner

marcus-crane commented Apr 27, 2023

Hi there,

Hmm, it seems that October didn't find any highlights on your device. I assume you do but I have to ask to confirm: Do you have highlights taken on your device? When selecting a Kobo, it should say how many highlights are found.

If you do, there are some next steps we can take to try and figure out what's going on

@GeorgeHahn
Copy link

I'm having a similar issue - it looks like something is causing my Kobo to wipe its highlights. I suspect this is a Kobo bug related to calibre-web sync. I reverted the conf edit for that and will keep an eye on my highlights.

@fnsne
Copy link

fnsne commented May 11, 2023

I'm having a same issue too.
I read the code start here .And don't know why the condition of cotentID/VolumeID is LIKE '%file:///%'.

func (k *Kobo) ListDeviceBookmarks() ([]Bookmark, error) {

there is no bookmark/content has contentID/volumeID that match the condition above In my Desktop kobo, and Clara 2E.

Will that be the key cause this problem?

Here is a snapshot of my bookmark table.
image

@fnsne
Copy link

fnsne commented May 12, 2023

I'm having a same issue too. I read the code start here .And don't know why the condition of cotentID/VolumeID is LIKE '%file:///%'.

func (k *Kobo) ListDeviceBookmarks() ([]Bookmark, error) {

there is no bookmark/content has contentID/volumeID that match the condition above In my Desktop kobo, and Clara 2E.

Will that be the key cause this problem?

Here is a snapshot of my bookmark table. image

And I found that is a commit after v1.3.1. So I take a try of a previous version.
Everything works well.

@marcus-crane
Copy link
Owner

I'm having a same issue too. I read the code start here .And don't know why the condition of cotentID/VolumeID is LIKE '%file:///%'.

func (k *Kobo) ListDeviceBookmarks() ([]Bookmark, error) {

there is no bookmark/content has contentID/volumeID that match the condition above In my Desktop kobo, and Clara 2E.

Will that be the key cause this problem?

Here is a snapshot of my bookmark table. image

And when I using the previous version(v1.3.1) today, everything works well.

Hi,

Seeing as those IDs are all GUIDs, are you highlighting books that you've purchased from the Kobo Store?

Sideloaded books all generally start with file:///

@fnsne
Copy link

fnsne commented May 12, 2023

I'm having a same issue too. I read the code start here .And don't know why the condition of cotentID/VolumeID is LIKE '%file:///%'.

func (k *Kobo) ListDeviceBookmarks() ([]Bookmark, error) {

there is no bookmark/content has contentID/volumeID that match the condition above In my Desktop kobo, and Clara 2E.

Will that be the key cause this problem?

Here is a snapshot of my bookmark table. image

And when I using the previous version(v1.3.1) today, everything works well.

Hi,

Seeing as those IDs are all GUIDs, are you highlighting books that you've purchased from the Kobo Store?

Sideloaded books all generally start with file:///

Yes, I purchased these books from Kobo Store.
So the v1.3.2 october is not support for highlights from Kobo Store?

@marcus-crane
Copy link
Owner

marcus-crane commented May 12, 2023

I'm having a same issue too. I read the code start here .And don't know why the condition of cotentID/VolumeID is LIKE '%file:///%'.

func (k *Kobo) ListDeviceBookmarks() ([]Bookmark, error) {

there is no bookmark/content has contentID/volumeID that match the condition above In my Desktop kobo, and Clara 2E.

Will that be the key cause this problem?

Here is a snapshot of my bookmark table. image

And when I using the previous version(v1.3.1) today, everything works well.

Hi,
Seeing as those IDs are all GUIDs, are you highlighting books that you've purchased from the Kobo Store?
Sideloaded books all generally start with file:///

Yes, I purchased these books from Kobo Store. So the v1.3.2 october is not support for highlights from Kobo Store?

It was never intended to support Kobo Store books in the first place but it just happened to work accidentally.

Readwise has an official Kobo integration that only works with Kobo store books and is what is recommended.

The main reason that I removed support for Kobo Store books is because of users who have both store bought and sideloaded books where they would get duplicate entries (one set from October and one set from Readwise) (#79)

I suppose if there's enough interest, I can always bring it back as an off-by-default toggle but yeah, the official Readwise integration is probably a better experience since you shouldn't need to connect your device to your computer as I understand it

https://readwise.io/changelog/kobo-import

@marcus-crane
Copy link
Owner

Ah, I forgot about this mention of the official integration not working from a while back: #78 (comment)

I've never tried it myself so I'd be interested to hear if people do have trouble with it.

@fnsne
Copy link

fnsne commented May 12, 2023

I think the official integration is wiered.
I faced the same trouble two week ago after it work very well for one or two month.
So I take a try of this project . And I also try to parse the database and turn it to a kindle's My Clipping.txt format.
But today I found the sync go back.
But I don't know why.
Maybe Readwise team fix this bug these days, or there maybe be a request rate limit on kobo.

Ah, I forgot about this mention of the official integration not working from a while back: #78 (comment)

I've never tried it myself so I'd be interested to hear if people do have trouble with it.

@melectronvolt
Copy link

melectronvolt commented May 26, 2023

Same problem here... :( Elipsa 2E but after restarting elipsa it works...
Maybe it's not a problem but when I start October I get this
image

Which refresh in
image

@marcus-crane
Copy link
Owner

@mevaere Ah, interesting. I have a brief flash of the same "Can't reach this page" on Windows myself. I mainly use macOS so I hadn't run the Windows version in a while nor have I used October recently either.

It could be related to general Windows 11 updates to WebView2 (which is what October uses under the hood) so I'll see what I can dig up.

As for the highlight detection issue, can you clarify whether you're only using books purchased from the Kobo store? The latest version of October disabled support for store books, as it was only intended for sideloaded content given that there is an official integration with Kobo supplied by Readwise for that use case.

If you're using sideloaded content though then I'll have to send you some further steps so I can try to narrow down what the exact cause might be.

@melectronvolt
Copy link

Clearly after restarting Elipsa 2E. Everything seems to work :

image

It doesn't detect the name of the kobo, but Calibre / Annotations is now able to fetch the annotations (it wasn't possible before).
On my side it's now usable :

  • Just WebView2 flashing white
  • Elipsa 2e and Unknow Kobo ;)

Thanks for the help.

@melectronvolt
Copy link

And everything is upload correctly on readwise :
image

Could-it be a problem on readwise ?

@marcus-crane
Copy link
Owner

Ah nice, I didn't realise that a new Kobo device had come out.

Yeah, as you saw, October will work fine. It contains a list of IDs and it looks up the one matching your Kobo in order to display its name but whenever a new Kobo comes out, it'll display "Unknown" until the list of IDs is updated to include new ones.

Hmm, perhaps there's something with brand new devices on their first run? I can try resetting my test device and see if any trouble occurs but glad to hear you've been able to upload highlights.

If 400 Bad Request occurs again, can you send me an email at [email protected] and I can try to run some tests on a copy of your Kobo database if you're comfortable with that.

@melectronvolt
Copy link

For sure, the problem is not specific to October, I had the same problem with calibre. So if problem come back I will send you a mail.

Thanks again for your app ;) very helpfull

@marcus-crane
Copy link
Owner

@mevaere While unrelated to this thread, I've just released v1.3.3 which should fix the "Connection Refused" error that flashes briefly. Apparently there were some changes to WebView2 (part of Windows used by October) that needed handling

@melectronvolt
Copy link

If you need some detail for the Kobo Elipsa 2E to don't appear "Unknown" just ask me ;)

@marcus-crane
Copy link
Owner

@mevaere I've just released v1.3.4 which should support detecting Kobo Elipsa 2E and Kobo Clara 2E.

I don't own either of these devices so I can't 100% test this out but I pulled the device IDs from the latest Kobo firmware so they should be correct.

CleanShot 2023-05-29 at 22 09 40@2x

@melectronvolt
Copy link

melectronvolt commented Jun 2, 2023

Porblem occured today.
image

If I use the stylet to draw on a book, it's an highlight for Kobo, it fail. Log give :
Just draw.txt

If now, I draw an highlight stuff normaly (without the stylet:
Fail :
Draw + HL.txt

After restart kobo + october: works
Ok.txt

Thanks a lot,

My database take 437 MB so are you want it ;)

@marcus-crane
Copy link
Owner

Hi @mevaere,

Hmm, I don't have an Elipsa 2E myself so I don't know how stylus drawings are saved in the database and whether they might be causing something to break but if you are fine with forwarding me a copy of your database to run some tests on, there are some caveats outlined here:

#86 (comment)

At almost 500MB, I'm not sure if it would fit in an email though 😅

@marcus-crane
Copy link
Owner

marcus-crane commented Jun 3, 2023

Hi @justinryanc @fnsne @mevaere, I've just released v1.4.0 which fixed some bugginess around querying that was introduced in v1.3.2.

It also adds a toggle in Settings for users who like to also sync store-purchased books. It's opt-in by default just so users are aware that it can cause duplication if the Readwise Kobo integration is also enabled.

Please let me know if this fixes up some of the issues you were seeing

@melectronvolt
Copy link

Fix found: I need to restart my Kobo reader to extract highlight or it doesn't work. Don't know why but that's how it works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants