Make planetary.name posts load consistently fast

[mplorentz]

The message cards on Planetary.name sometimes load really fast and sometimes they take over 10 seconds. Let's make them consistently fast. Some theories to why they are slow:

• the frontend slams graphql with blob requests for profile pics. Serving blobs directly from disk -> nginx might alleviate.
• The graphql server crashes and has to reboot, resulting in slow requests as it comes back up
• scuttlebutt stuff pauses for indexing
• (feel free to add more)

[mplorentz]

@mixmix as the person with the most widespread domain knowledge could you take the lead on diagnosing what is slow here and what we might do to fix it?

[mixmix]

@danlatorre this is good to know but probably a different issue? I'll make on for it

[mixmix]

Going through the graphql queries that this page makes one at a time
https://planetary.name/profile/@GCSmjLHmFC8m9t0ouJ6cY4tzlQdIbBaQfI0Uifw5cZA=.ed25519

1. Daniel's profile ✔️

all good, we need all this

2. Loading room data, members? 🔥

I can't see why we need members data on this page.
This amount of data should not be needed

Recommend : remove this query

3. Daniel's profile (AGAIN) 😢

Loading Daniels' data again... but more (need followers, following)

Recommendation

• combine this query with the one before
• there is HEAPS of unused data here, make a lighter query
  • we need the follower count
  • we need 2-3 images only (not description, URI, feedId etc etc for 71 people)

4. Loading 10 of Daniel's threads ✔️

Looks good to me. If we wanted we could probably load 6 or 7 as we have that auto-scroll load more.

5. Load random threads 🔥

Why are we getting 10 random threads? This seems like a mistake.
Recommend find and remove

[mixmix]

I haven't looked at the Vue code @chereseeriepa , but if you're not sure where those queries are coming from, or want to pair on how to split up the Vue routing/ pages differently I'm happy to pair.

Profile caching

We should add a cache over profiles in https://github.com/planetary-social/planetary-graphql/blob/main/src/graphql/resolvers.js#L22-L30 similar to the one we have in Ahau.

Needs

1. a cache which can invalidate itself if updates come through
   • Ahau cache: https://gitlab.com/ahau/lib/graphql/ssb-graphql-profile/-/blob/master/src/ssb/cache.js
2. wiring that into resolver
   • Ahau resolver: https://gitlab.com/ahau/lib/graphql/ssb-graphql-profile/-/blob/master/src/ssb/query/get-profile.js#L40-46

[mixmix]

@mplorentz That's my analysis. I'll talk with @chereseeriepa about generating cards from these.
Handing over assignment to her on this issue now

[mixmix]

Did some pairing with @chereseeriepa today, already seeing improvements in load times.
Gonna pair with @zachmandeville on graphql caching of profiles tmrw

[cooldracula]

Mix and I paired on this today and diagnosed one area that could be causing the issue, which is that the library that the graphql server uses has a vulnerability to DDOS attacks (see these docs ).
They recommend setting a “cache:bounded” option when configuring the server. These are caches that are being run under the hood automatically by the apollo server, and would be separate from other caches we’ve been discussing.
I set this option and pushed a new image to both ansible.fun and planetary.name. From my load tests they both seem faster, but curious how others find it.

note for later: upgrade server
The other recommendation from these docs is to upgrade the server to the next major version, where the caching system is stronger “out of the box”. I briefly looked at what would be entailed in doing this, and it might be a bit prickly getting all our dependencies to work together on the migration, so I saved that for another day.

[danlatorre]

Instead of creating new tickets I'm just adding on to this existing open ticket. This post is mainly for @chereseeriepa but since this is a compound stack of issues for QA/debug it'd be good for @mplorentz @mixmix and @cooldracula to have visibility on this.

Screenshare:
https://www.loom.com/share/32f44c631e174f81888f542e2d04b60e

In the screen cast I demo the 3 main UX QA issues to investigate, in a Lean timebox of a 1-2 hours for each issue. The outcome here is to determine the root issue and either fix if quick (a few hours or less) or have an estimate of the fix.

1. On a person's page the UI loads, you can see the cards, but SSB text and image content don't load.
2. On a person's page e.g. Paul Semel the images are broken/no loaded or they load but then viewing image in a new tab generates a 502 bad gateway error.
3. On a person's page, the profile data loads, but the person's feed doesn't load at all, no UI no data... we just a 3 dot ellipse animating.

[cooldracula]

For the third point(profile data loads, but not the person's feed), I wonder if it on accounts that have not posted anything yet?

I checked a couple of accounts, one where posts load and ones where we see the ellipsis:

• Daniel's page loads
• yeradis' page does not

on https://planetary.name/graphql/, you can run queries against our graphql server, like one to get the profile for a feed id, with its threads.
This is (roughly) the call being made by the frontend to the graphql server when visiting a profile.

  query  {
    profile: getProfile (id: $ssbID) {
      id
      name
      image
      description
      ssbURI
      threads {
        id
        text
      }
    }
  }

If you run the below, condensed query, you can see that Daniel's response has threads while yeradis' does not:

# Write your query or mutation here
query {
yeradis: getProfile(
  id: "@a2djG8NtXafuD7vSdymhfYd97HUr7snWfVd7gxSvXe4=.ed25519"
) {
  id
  name
  threads {
    id
  }
}
daniel: getProfile(
  id: "@GCSmjLHmFC8m9t0ouJ6cY4tzlQdIbBaQfI0Uifw5cZA=.ed25519"
) {
  id
  name
  threads {
    id
  }
}
}

Are there any cases we can find where the person has threads, but their page is only showing the ellipses? If not, maybe we'd just need a "this person has not posted yet!" style text if their threadcount is 0?

[mplorentz]

@cooldracula wow nice find. I checked a few people I had seen ellipses on and they all have a thread count of 0.